A Year of Updates about Operant and Classical Conditioning

A year ago and a couple of days ago, we got our dog, Argos!

Around then, I dived pretty deeply into learning about operant and classical conditioning, especially with respect to animal training. I read many books and blogs, attended ClickerExpo, and chatted with the trainers at our puppy socials. And after living with a puppy and a toddler for a year, I’ve had plenty of opportunities to experiment with ways to get behaviors I wanted from them. 

My very broad brush summary is that a lot of the technical points actually cache out in just being patient, caring, and creative. And, to me anyway, this is excellent news! I want to get better at those things anyway, and it (usually) feels good to practice them.

The most useful distinction in this area that I hadn’t really been making was between skill building and behavior choices

I’ll call something a choice if it would respond to Szasz’s “Gun-to-the-Head Test”. My dog wouldn’t eat butter off the counter if he knew it would be really costly, so I’ll count that as a choice. But he can only roll one way (I’ll call it to the left), so teaching him to roll on his back to the right counts as skill building. 

I summarized my heuristics for skill building in a Memocracy talk at Ephemerisle, and finally just put the notes from my talk into a blog post of its own because it seems like a separable topic.

Okay, so assuming you aren’t trying to help someone learn something new (like how to walk, talk, or give you a high five). Assume you have strong enough preferences over the behaviors the other person is already offering to try to influence their frequencies.

Plan A: Meet the Underlying Need

My first, best lever is still trying to meet an underlying need (or want?) that will get me the behavior I’m looking for. It actually did help Lydia ignore other kids’ water bottles to buy her a variety and let her play with them at home. It very clearly helps me get Argos to lie around calmly more of the time when I take him on long walks and run around the yard with him. Giving him stuff to chew helps him not chew Lydia’s toys.

In order to meet needs, I have to notice them, and one of the biggest things that help me do this is not blaming the subject OR myself. Either one takes the focus elsewhere. It also takes some experimentation to be pretty confident that you know what the underlying need is. 

More than anything else, maybe, meeting needs usually takes an absence of learned helplessness about meeting the need.

A lot of my recent thoughts about how to best meet needs have come from this unschooling website.

I don’t in any way consider myself an expert on figuring out how to meet people’s needs, though I do recommend a meta/intrapersonal approach. (Basically, meet your own needs too. Avoid unfunded behavioral mandates (HT: Mike Blume) by making it the job of the part of you that wants something to happen to figure out a way to meet the needs of parts that are blocking it. Or something kind of like that. I have a post building up in my head about my thoughts in this domain.)

Plan B: Assume New Tank Syndrome

My second lever actually covers quite a lot of the situations that aren’t solved by (straightforwardly) meeting needs, and that is helping the learner become more comfortable with an environment or stimulus. I covered already in this post, but I think it bears repeating and elaborating on. Very frequently, if it seems like Lydia and Argos aren’t processing what I’m saying, they’re are very focused on the environment, and spending lots of cycles processing that. 

The Behavior Adjustment Training framework helped me get clear on what to do to help a learner feel more comfortable with a stimulus. She tried a bunch of different approaches that I won’t summarize here, but what seemed to work the best and fastest was letting the animal explore the stressful stimulus at its own pace in an environment with other cool distractions for when it wants a break, only intervening to stop the animal from getting stressfully close to the thing.

In retrospect, this strikes me as being extremely commonsensical, but I think I now have a better model of why and how it works. This model results in greater confidence that, over time, behavior around a specific thing will get more conscious and flexible, with more room for input from other interested parties.

Karen Pryor mentions “new tank syndrome” in Don’t Shoot the Dog:

What is once learned is not forgotten, but under the pressure of assimilating new skill levels, old well- learned behavior sometimes falls apart temporarily. I once saw a conductor, during the first dress rehearsal of an opera, having a tantrum because the singers in the chorus were making one musical mistake after another; they seemed virtually to have forgotten all their hard-learned vocal accomplishment. But they were, for the first time, wearing heavy costumes, standing on ladders, being required to move about as they sang: Getting used to new requirements temporarily interfered with previously learned behavior. By the end of the rehearsal, the musical learning reappeared, without coaching. Dolphin trainers call this the “new tank syndrome.” When you move a dolphin to a new tank, you have to expect that it will “forget” all it knows until the new stimuli are assimilated. It is important to realize that berating yourself or others for mistakes in past- learned behavior under new circumstances is bad training. The mistakes will usually clear up by themselves shortly, but reprimands cause upset and sometimes tend to draw attention to the mistakes so they don’t go away.

I didn’t get how important it was though. At almost two, almost everything is “new tank” for Lydia. A circle of girls playing with attractive My Little Ponies at the playground is a “new tank”. Maybe the block we live on wasn’t a “new tank” last week, but it is now because her perceptual abilities are sharper and her interests are different. She’s not used to processing the neighbor’s flowers because she didn’t used to care about them.
For Argos, our neighborhood is still a new tank. I’ve been taking him on walks since the beginning, but the neighborhood is full of new smells, new dogs, new people, and new everything else. It’s not as true as it used to be, when he’d stare at bicycles. But it still mostly is. The new stimuli have not been assimilated. Which is fine. That’s part of why it’s exciting to go on walks. 
Argos and I have been finding a good rhythm on walks recently where he doesn’t pull me around too much (he’s big), but part of what’s helped has been… letting him pull me around quite a bit while he explores. He gets by now that he’s usually not supposed to interact with on-leash dogs much (because he’s not calm enough for it to go well, also because lots of dogs don’t like intact adolescent male dogs), and is actually pretty good at keeping himself pretty under threshold, in part by running away from dogs. Not in a scared way, in an I’m excited by I know I can’t really have it and now need to let off some energy way. Sometimes we still practice loose-leash walking around the neighborhood, and he can often actually do it for long stretches of time if nothing too stressful happens. (He’s a pro in the house, even when he’s excited because he knows he’s about to go on a walk.)
I’d read books and talked to trainers, but I hadn’t really gotten the thing about processing the environment being THE thing for loose-leash walking, assuming the dog can do it at home (which in our case took very little time to teach). I think this blog passage is what made it finally click for me:
Mindy has been in public a lot, but mostly in the suburbs, in shopping areas, in hotels and malls and such buildings. She hadn’t seen the press of a typical urban street during crowded times. So many people, so close together, with so much traffic, was a lot!
When one criterion is raised, another drops! so I didn’t worry about leash manners. She pulled, a lot, because she was so busy looking around and taking things in. I wasn’t worried about this; leash manners will come back when she’s able to think about them again. Fussing at her would only have frustrated us both and both kept her from processing all this new scenario and established a bad association with it. 

Miscellaneous Other Stuff

If I hadn’t been reading about operant conditioning, it probably wouldn’t have occurred to me to reward behaviors I was looking for even when they were totally accidental and out of context. Baby signing was the relevant use case. I made a point of getting super excited WHENEVER Lydia made a motion that was kinda like a sign, since then she became conscious of what she just did with her body. She could hook the skill of doing the movement on purpose up to the actual meaning later, and she routinely did.

Similarly, if there’s a particular thing I’m looking for from another person, like being really attentive to what I’m saying, or empathizing, or not being defensive, these days I’m more on the lookout for opportunities to comment appreciatively on the thing even when it’s not happening at the time I feel like I need it. Because then, the person is more aware of the thing as a conscious skill they can choose.

On a slightly different note, Argos is actually pretty great about trying to be careful with his body even when he’s moving very quickly… except that he doesn’t get that his tail is part of his body. I think lots of dogs don’t understand this. My plan is to try to teach him to intentionally knock balls off the table with his tail, in hopes that it will improve his awareness in this area. I don’t have super high hopes for the approach, but I’ll probably report back if it works.

Classical Conditioning

If you consistently do something a few seconds BEFORE something else happens, the learner will connect the dots and start anticipating the second thing after the first thing. This won’t be as reliably true if they happen too close together, simultaneously or in the other order. Knowing this made me more inclined to announce my intentions with Lydia verbally before showing body language that I was going to do the thing. (Saying “Can I pick you up?” before putting my arms out.) I know she would have figured everything out anyway, but verbal communication is a big enough convenience factor for me that I’m willing to invest in optimizing my behavior to help her understand me more quickly.

It’s worth worrying about poisoned cues. Basically, if something good OR bad might happen when you ask someone to do something (you’ll get mad if they don’t do it or do it wrong), then they’ll easily come to dread hearing you ask. Fear is terrible for learning. Avoiding poisoned cues falls under being nice, but I thought I’d mention it specifically.

tl;dr Be nice. Work on one thing at a time. Everything else is new tank syndrome.

Skill-Building Heuristics

After I gave a talk at Ephemerisle, some months ago, I got some requests for a summary in writing. Here it finally is!

My intuitive model of learning spits out that I learn best when I’m “challenged”. I’ve come to believe that a feeling of accomplishment and a desire to “consolidate” what I’ve learned (by taking a nap or some other sort of break) are good signs. I’ve also come to believe that frustration and failure are suboptimal. Sure, it’s virtuous on my part to be able to learn from situations that aren’t designed for my learning experience (most of them), and that means learning strategies to cope with frustration and failure. But when I’m the one designing the learning experience, for myself or someone else, I now see it as my job to minimize frustration and failure in the learner.

Errorless learning is a thing, and it seems to me to be a good thing. (At least in certain cases. I was going to say it was great for people, like amnesiacs, who relied on implicit memory more since I had remembered reading that theory, but while looking up citations, I also found this, which says that for memory, errorless learning can trade off against effortful recall, which seems to be important. Interesting. For memory, I think spaced repetition is quite optimized.)

An unfortunate side effect of learning that does involve failure and frustration is that you can end up getting misplaced aggression in the learner. I’ve definitely observed this in my dog, my daughter, and myself, though not to any serious degree.

I think TAGteach is a pretty good methodology. Here are my main takeaways:

  • Work on one criterion at a time. (Practice either speed or accuracy, for example.)
  • After giving whatever longer instructions you want, describe what the learner should do in at most five word.
  • Easily verifiable criteria are the best. (“Arms above head” beats “hold your arms up high”.)
  • If the learner can’t do what you’re asking in three (or so) tries, make it easier!
  • If you can use a marker sound (like a clicker, or saying “Yes!”) to point out the exact moment the learner got it right, this can speed things up quite a bit.
  • If you’re increasingly the criteria in one area, expect performance in other areas to degrade. This is normal and expected and will go away once you get the thing you’re working on straight. (If you’re trying to jump straighter and you’re not jumping as high as you were, that’s fine.)
  • Being in a new location (or with new equipment etc.) is an increase in criteria.
  • To best get fluency, work on precision before speed.
  • For learners who want to learn the thing, getting the information that they’re learning it is potent reinforcement. But if anything is making the experience unpleasant, it can be nice to additional actual treats when they get stuff right.

Punishment is very problematic for building skills. It has advantages, and can be very effective, sometimes resulting in one-trial learning. It also seems to generalize well, but threat kills curiosity and experimentation, which I very much want to preserve in any learners I work with. Here’s a Karen Pryor quote from Reaching the Animal Mind:

Clicker trainers have a strong gut reaction against punishment; and the more experienced we are, the stronger the feeling. Well, thanks to Panksepp, we have a scientific rationale for why mixing correction and reinforcement is harmful rather than helpful to the learning. It’s not just a moral issue; it’s common sense. Correction or rebuke switches the learner from the hypothalamus and its SEEKING mode to the amygdala’s path of avoidance and fear.

And even punishment that seems mild to the one administering it can be very bad for relationships. 

I will emphasize that, since learning more about learning, it’s become increasingly obvious to me how much people, including me, tend to assume that teaching means telling people they’re wrong and “correcting” them. As a tutor, I’ve found it especially hard not to fall into this pattern. My narrative has historically been that I’m providing maximum value to people when I’m showing them something they don’t know yet, and that equates to giving them problems they can’t do yet. These days, I’m changing my approach to something friendlier and more success-based :-).

I’ve also largely moved away from “teaching” as a paradigm recently, since I’ve been working on grokking unschooling, and that’s something I’ll try to write more about as my thoughts feel ready to be put into blog posts.

Appreciating Contempt

A couple of years ago, Will and I attended an event where someone asked us all to consider which emotion we had the hardest time owning and were most likely to resist and push away. We both gave the same answer: contempt.

At the time I remember thinking that contempt seemed mean and not that useful. I talked to some people about contempt, and don’t remember anyone at the event giving me a compelling reason to embrace it, though it’s possible I did hear good advice, but wasn’t in a place to process it.

And I’m happy to say that I’ve finally made some progress on appreciating contempt!

In particular, I noticed that a lot of my internal dialogue was actually pretty self-contemptuous. I seemed to be using self-contempt to notice when my own positions didn’t make any sense and straw man them.

Noticing when my own positions don’t make sense is awesome!

Using self-contempt to do that seems pretty efficient. If you’ve never asked yourself, “what would my enemy think about what I was doing,” I recommend trying it. It’s been eye-opening for me in the pass, and my worldview makes the claim that most people assume most people are more virtuous than is actually the case.

But then, viewing myself with contempt is also costly. It’s easy for me to miss how costly it is, because these thoughts are tinged with the cold kind of contempt contempt, and that tone can slip under my radar pretty easily. But I end up feeling small and not very confident as a result :-(.

And even though my thoughts aren’t perfect, they’re usually a better guess than than my best arguments against them.

Here’s an example of my contemptuous voice being mean:

“I think it worked out okay that I didn’t obsess about not having the dog jump on people. He’s naturally doing it less as he gets more comfortable.”

“OR you’ve created a behavior pattern that he didn’t have to have that makes everyone like him a little less and a generally pushy attitude that doesn’t serve him or anyone else very well.”

Sometimes, it says things that imply that I’m doing something more right than not.

“Meh, Argos (same dog) gets so frustrated when I try to teach him stuff. It didn’t seem as bad before, so maybe I’ve poisoned the process somehow.”

“OR you just thinking that because you’ve accidentally reinforced frustrated barking a few times in a row, that factor is very salient for you, and nothing else is all that different. You weren’t sure he’d ever learn the other stuff either, but he did.”

The self-contempt thing isn’t about me being a bad person, or about me having done the wrong thing. It’s M.O. is telling me that my thoughts, flattering or unflattering, are generally crappy and not to be trusted. 

Kinda costly, but also useful and overall truer than not. Thanks contempt!

Brief Clicker Expo Notes

I went to ClickerExpo in January!

The whole experience was terrific, and I think I learned quite a bit. I got a few new ideas, but not many. But I do think I understand many of the things I’d previously read about it more fully and in a way that’s more actionable. Real-life examples are invaluable when it comes to updating all parts of my mind.

I jumped around between talks quite a bit, and when I wasn’t compelled by any of the speakers I went to the hands-on “learning lab” clicker sessions with dogs. 

This post is a bit shorter than I had originally intended. I realized that I wasn’t getting it written so I lowered my criteria :-).

Here are some rough notes:

  • I was impressed with Karen Pryor as a speaker. I found her quite engaging–particularly her talk about how most scientists aren’t actually using the conditioned reinforcer properly. Yet another lesson in how crazy the world is, and in particular how messed up our scientific institutions are. Seems like things are slowly improving though in this area!
  • I saw some fun videos about some dogs distinguishing between one, two, and three objects fluently.
  • There was quite a bit about emotions. Everyone agreed that behaviors originally associated with angry, scared, or other upset emotions do not retain their original tone when captured with clicker training. In particular, the faculty noted that clicker training dogs to act ferocious didn’t work very well, and that this had been a challenge for people training dog actors for commercials. And there also seemed to be consensus that behaviors would acquire the emotional tone of the reinforcers used to train them. I updated that using multiple marker signals with different emotional tones was an excellent idea. (I’d heard of this idea before but hadn’t been compelled by it.)
  • I both got some good heuristics for good shaping and a bunch of hours observing good shaping. I think I (and most amateur shapers) should raise criteria more slowly and take more breaks). It’s hard not to be impatient and overenthusiastic! One thing I hadn’t considered was that jumping ahead too quickly with criteria increases both the likelihood that garbage behaviors will get chained in and that the learner will then get stuck at a particular level.Variability is worth preserving.
  • Verbal stimulus control is hard with dogs! Argos and I have been practicing since ClickerExpo to try to get him to listen to my actual words, but it was actually fairly heartening to see in the learning lab that many of the professional dog trainers hadn’t done any better with their dogs.
  • Kathy Sdao gave a lecture on Premack’s principle that I found quite inspiring. It didn’t contain any entirely new information, but it was a very welcome refresher. I had heard the idea of reframing distractions as the best possible rewards for dogs, but I hadn’t actually thought of it that way for Lydia. I can tell I have a lot to learn about this whole clicker baby/toddler business, and I do think I’m slowly learning it.
  • I also went to Kathy Sdao’s lab on cues. If anything, I was comforted that the professional trainers who had brought their dogs didn’t seem to have much better stimulus control than I have with Argos for verbal stuff. We’ve been working on that since I got back, but he’s still confused about sit vs. down, since he often goes down by sitting first, so it has historically worked well enough for him to guess. I’m trying to clean it up bit by bit.
  • I had heard that Clicker Expo was an unusually positive and rewarding atmosphere. Makes sense. Not sure i would give the same review, but I think I’m implicitly comparing it to events I futurist-type events I usually go to that are usually full of my friends and people in my general demographic. I did get a few genuine compliments each day though, which was nice!
  • I’m now part of a Clicker Parents Facebook group! I was going to start a list, but then I found out that already existed.

Since it’s been a few weeks now since I attended, I can confirm that it seems totally worthwhile that I went. It’s impossible to run controlled experiments with my n=1 parenting situation, but I can see that my relationship with Lydia has changed for the better in ways that seem directly attributable to stuff that I learned and internalized at Clicker Expo. That’s what I wanted! I’m now working on writing up the details of the changes I’ve made in perspective and how we do things.

Balancing Needs in Relationships

I don’t have time to write something long this week, but I did want to pose a question that’s been on my mind.

When someone you care about wants something and you have mixed feelings about whether to give it to them, what are your heuristics for deciding what to do?

Do you have an internal voice or feeling that you trust to guide your decisions? Do you try to calculate how much it matters to each person, and if so is that process explicit or implicit? Do you like the way you think about these questions?

It’s possible that the question I’m posing already assumes an unhelpful frame, and I think it’s true that there’s often low-hanging fruit in the category of win-win solutions. But I’d love to hear people’s answers anyway.

Any and all thoughts on the subject are welcome.

One Thought and Two Links

I skipped writing a blog post last week because I was tired and busy. I don’t feel up for writing a long one this week either, but I will share a thought and some links.

I often look at my (13 month old) daughter’s eyes when I’m trying to figure out what she is thinking. But something I don’t do often enough is look at what she’s looking at. I try, and it’s easy to get a vague idea by following her gaze. I but I can get a more precise idea by actually getting my head over by her head and following more closely. I would recommend the same for getting into the head of younger babies and animals.

Here’s a post I liked about frustration and learning.

And another one about why the story about extrinsic motivation replacing intrinsic motivation may be more complicated than usually discussed.

Other People’s Emotional States

Last week I posted as my Facebook status that: “Someone else’s emotional state is almost always a terrible optimization target.” I got a couple of requests to expand the thought to the length of a blog post, so here I go!

I’ve been thinking since I posted it, and I think there are two fairly separable reasons that I think the mental representation of another person in a particular emotional state isn’t a productive thing to focus my mind on. First, how other people feel isn’t something I can control. And second, it’s a metric that is easy and harmful to game.

Let me start by saying how I think intentionality works in the human mind. (These ideas are not original to me, but I also haven’t ever heard anyone articulate them in quite the way I do.) Intentions are represented as pictures of how we want the world to be. I think they usually have more influence over our behavior when they’re represented vividly. Pictures are almost always involved, words may be involved, feelings in our body are often involved. Smells and tastes may be involved too. They have size, color, and position. They may be moving or static. 

And ultimately, we try to make these pictures come true. Of course it’s somewhat more complicated than that. And we’re certainly imperfect at fulfilling our intentions, but that’s the basic idea. I won’t link to The Secret because that seems embarrassing and I haven’t actually read it, but I think that book says the same thing in more woo-woo language.

To break it down more, I think correspondence between our mental representations and our perceptions is reinforcing, and more so for the more vivid ones that are represented in many modalities. So approaching intentions shapes our behavior.

And it’s worth saying that the intentions that have most powerfully shaped our behavior aren’t necessarily ones that are aligned with our explicit goals. As the saying goes, “we’d rather be right than happy.”

[Read more…]

Memorizing Emotionally Salient Quotations

One of the more unusual projects I undertook a few years back was memorizing a bunch of quotations. In particular, I mostly picked ones that seemed emotionally salient.

I love it when I read something that lets me glimpse a truth about the world that I hadn’t quite noticed or thought of in that way. And typically, the clue that I’m realizing something is that I have an emotional reaction.

I did find that if I spent enough time with the quotations, I was able to grok them more fully. I could update my model of the world and internalize the meaning that I had been reacting too. I was still able to recognize the words as beautiful afterwards, but the character of my appreciation for them had changed to something softer.

Looking back at some of the passages I memorized at the time, I thought I’d share a few and what I think I learned from them.

[Read more…]

Suffering is Created by Resistance

I got an email a few days ago about a card from one of my Anki decks that says “Suffering is created by resistance.”

I wanted to explain what that statement meant to me. To start, in my mind there is a companion phrase: “Pain is not suffering.” 

Here’s what I’ve written about that statement. I hope it makes sense.

But suffering seemingly exists, and so if pain doesn’t account for it, what does?

This post is a more complete and articulate presentation of the idea that suffering results from conflict between two subsystems. 

And sometimes I do put it that way. I think there’s another card in the same Anki deck that says something about how internal conflict results in suffering. Saying that suffering is created by resistance is a slightly different framing, and one that I think is useful.

Physical suffering is a lot about tensing up, which seems to go along with the framing of suffering being resistance. Suffering comes from wanting the world to be different, and making some mental or physical motion that sort of kind of almost works on the margin but is actually ridiculous.

A very rough idea of how I think our brain works is that we hold in our mind a mental representation of how we want things to be, and then move towards that goal state. It seem to be strongly reinforcing to have our external experience match our internal pictures. But when we resist, we’re trying to hack this system. Like closing our eyes to prevent someone from seeing us.

Or trying to ignore and silence disturbing thoughts because that better fits our mental picture of health and happiness.

Pretending not to be angry because I don’t know what to do with anger. Slumping and having bad posture to dampen my feelings.

All that stuff feels bad, and I would call it resistance.

Feeling Better than Other People

This post will just be a quick note based on a conversation I had with someone recently on the subject of feeling superior to other people. I’ve been there before, as I suspect many people have. If you find yourself simultaneously thinking you’re somehow better than other people and thinking it’s wrong to think that way, here are some distinctions that may make the issue less loaded.

  • The question of how much net value you are providing for the world is a legitimate one, in my opinion. And the answer may be that you’re providing more than most, especially if you’re optimizing for it. 
  • If you’re feeling a sense of entitlement that seems less than pro-social, remember that most things aren’t zero sum. If you want more attention from others, influence in the world, money, etc., that’s legitimate! But these things don’t have to come at other people’s expense.
  • People are pretty different from each other. You may be more intelligent, powerful, aware, etc. than many people. And you are almost certainly less of all the above characteristics than others. It can be polite to ignore differences in ability between people, but there comes a time when you may also want an accurate assessment of where you are. Make it okay to think about these things as objectively as possible when you choose to focus on them. It’s okay that differences exist, and if you try to repress your desire to discern them, you might get messy side effects.. Sometimes these differences have far-reaching implications, and often not. 
  • The capacity to feel respect and compassion for all people is a really important one. Some thinkers say that we get our gut-level feeling of self-worth primarily from recognizing this particular capacity, and this seems plausible to me. Whatever causes you to lose track of this ability is worth reconsidering.

I think I had more thoughts there, but that’s a brief partial brain dump.