A Year of Updates about Operant and Classical Conditioning

A year ago and a couple of days ago, we got our dog, Argos!

Around then, I dived pretty deeply into learning about operant and classical conditioning, especially with respect to animal training. I read many books and blogs, attended ClickerExpo, and chatted with the trainers at our puppy socials. And after living with a puppy and a toddler for a year, I’ve had plenty of opportunities to experiment with ways to get behaviors I wanted from them. 

My very broad brush summary is that a lot of the technical points actually cache out in just being patient, caring, and creative. And, to me anyway, this is excellent news! I want to get better at those things anyway, and it (usually) feels good to practice them.

The most useful distinction in this area that I hadn’t really been making was between skill building and behavior choices

I’ll call something a choice if it would respond to Szasz’s “Gun-to-the-Head Test”. My dog wouldn’t eat butter off the counter if he knew it would be really costly, so I’ll count that as a choice. But he can only roll one way (I’ll call it to the left), so teaching him to roll on his back to the right counts as skill building. 

I summarized my heuristics for skill building in a Memocracy talk at Ephemerisle, and finally just put the notes from my talk into a blog post of its own because it seems like a separable topic.

Okay, so assuming you aren’t trying to help someone learn something new (like how to walk, talk, or give you a high five). Assume you have strong enough preferences over the behaviors the other person is already offering to try to influence their frequencies.

Plan A: Meet the Underlying Need

My first, best lever is still trying to meet an underlying need (or want?) that will get me the behavior I’m looking for. It actually did help Lydia ignore other kids’ water bottles to buy her a variety and let her play with them at home. It very clearly helps me get Argos to lie around calmly more of the time when I take him on long walks and run around the yard with him. Giving him stuff to chew helps him not chew Lydia’s toys.

In order to meet needs, I have to notice them, and one of the biggest things that help me do this is not blaming the subject OR myself. Either one takes the focus elsewhere. It also takes some experimentation to be pretty confident that you know what the underlying need is. 

More than anything else, maybe, meeting needs usually takes an absence of learned helplessness about meeting the need.

A lot of my recent thoughts about how to best meet needs have come from this unschooling website.

I don’t in any way consider myself an expert on figuring out how to meet people’s needs, though I do recommend a meta/intrapersonal approach. (Basically, meet your own needs too. Avoid unfunded behavioral mandates (HT: Mike Blume) by making it the job of the part of you that wants something to happen to figure out a way to meet the needs of parts that are blocking it. Or something kind of like that. I have a post building up in my head about my thoughts in this domain.)

Plan B: Assume New Tank Syndrome

My second lever actually covers quite a lot of the situations that aren’t solved by (straightforwardly) meeting needs, and that is helping the learner become more comfortable with an environment or stimulus. I covered already in this post, but I think it bears repeating and elaborating on. Very frequently, if it seems like Lydia and Argos aren’t processing what I’m saying, they’re are very focused on the environment, and spending lots of cycles processing that. 

The Behavior Adjustment Training framework helped me get clear on what to do to help a learner feel more comfortable with a stimulus. She tried a bunch of different approaches that I won’t summarize here, but what seemed to work the best and fastest was letting the animal explore the stressful stimulus at its own pace in an environment with other cool distractions for when it wants a break, only intervening to stop the animal from getting stressfully close to the thing.

In retrospect, this strikes me as being extremely commonsensical, but I think I now have a better model of why and how it works. This model results in greater confidence that, over time, behavior around a specific thing will get more conscious and flexible, with more room for input from other interested parties.

Karen Pryor mentions “new tank syndrome” in Don’t Shoot the Dog:

What is once learned is not forgotten, but under the pressure of assimilating new skill levels, old well- learned behavior sometimes falls apart temporarily. I once saw a conductor, during the first dress rehearsal of an opera, having a tantrum because the singers in the chorus were making one musical mistake after another; they seemed virtually to have forgotten all their hard-learned vocal accomplishment. But they were, for the first time, wearing heavy costumes, standing on ladders, being required to move about as they sang: Getting used to new requirements temporarily interfered with previously learned behavior. By the end of the rehearsal, the musical learning reappeared, without coaching. Dolphin trainers call this the “new tank syndrome.” When you move a dolphin to a new tank, you have to expect that it will “forget” all it knows until the new stimuli are assimilated. It is important to realize that berating yourself or others for mistakes in past- learned behavior under new circumstances is bad training. The mistakes will usually clear up by themselves shortly, but reprimands cause upset and sometimes tend to draw attention to the mistakes so they don’t go away.

I didn’t get how important it was though. At almost two, almost everything is “new tank” for Lydia. A circle of girls playing with attractive My Little Ponies at the playground is a “new tank”. Maybe the block we live on wasn’t a “new tank” last week, but it is now because her perceptual abilities are sharper and her interests are different. She’s not used to processing the neighbor’s flowers because she didn’t used to care about them.
For Argos, our neighborhood is still a new tank. I’ve been taking him on walks since the beginning, but the neighborhood is full of new smells, new dogs, new people, and new everything else. It’s not as true as it used to be, when he’d stare at bicycles. But it still mostly is. The new stimuli have not been assimilated. Which is fine. That’s part of why it’s exciting to go on walks. 
Argos and I have been finding a good rhythm on walks recently where he doesn’t pull me around too much (he’s big), but part of what’s helped has been… letting him pull me around quite a bit while he explores. He gets by now that he’s usually not supposed to interact with on-leash dogs much (because he’s not calm enough for it to go well, also because lots of dogs don’t like intact adolescent male dogs), and is actually pretty good at keeping himself pretty under threshold, in part by running away from dogs. Not in a scared way, in an I’m excited by I know I can’t really have it and now need to let off some energy way. Sometimes we still practice loose-leash walking around the neighborhood, and he can often actually do it for long stretches of time if nothing too stressful happens. (He’s a pro in the house, even when he’s excited because he knows he’s about to go on a walk.)
I’d read books and talked to trainers, but I hadn’t really gotten the thing about processing the environment being THE thing for loose-leash walking, assuming the dog can do it at home (which in our case took very little time to teach). I think this blog passage is what made it finally click for me:
Mindy has been in public a lot, but mostly in the suburbs, in shopping areas, in hotels and malls and such buildings. She hadn’t seen the press of a typical urban street during crowded times. So many people, so close together, with so much traffic, was a lot!
When one criterion is raised, another drops! so I didn’t worry about leash manners. She pulled, a lot, because she was so busy looking around and taking things in. I wasn’t worried about this; leash manners will come back when she’s able to think about them again. Fussing at her would only have frustrated us both and both kept her from processing all this new scenario and established a bad association with it. 

Miscellaneous Other Stuff

If I hadn’t been reading about operant conditioning, it probably wouldn’t have occurred to me to reward behaviors I was looking for even when they were totally accidental and out of context. Baby signing was the relevant use case. I made a point of getting super excited WHENEVER Lydia made a motion that was kinda like a sign, since then she became conscious of what she just did with her body. She could hook the skill of doing the movement on purpose up to the actual meaning later, and she routinely did.

Similarly, if there’s a particular thing I’m looking for from another person, like being really attentive to what I’m saying, or empathizing, or not being defensive, these days I’m more on the lookout for opportunities to comment appreciatively on the thing even when it’s not happening at the time I feel like I need it. Because then, the person is more aware of the thing as a conscious skill they can choose.

On a slightly different note, Argos is actually pretty great about trying to be careful with his body even when he’s moving very quickly… except that he doesn’t get that his tail is part of his body. I think lots of dogs don’t understand this. My plan is to try to teach him to intentionally knock balls off the table with his tail, in hopes that it will improve his awareness in this area. I don’t have super high hopes for the approach, but I’ll probably report back if it works.

Classical Conditioning

If you consistently do something a few seconds BEFORE something else happens, the learner will connect the dots and start anticipating the second thing after the first thing. This won’t be as reliably true if they happen too close together, simultaneously or in the other order. Knowing this made me more inclined to announce my intentions with Lydia verbally before showing body language that I was going to do the thing. (Saying “Can I pick you up?” before putting my arms out.) I know she would have figured everything out anyway, but verbal communication is a big enough convenience factor for me that I’m willing to invest in optimizing my behavior to help her understand me more quickly.

It’s worth worrying about poisoned cues. Basically, if something good OR bad might happen when you ask someone to do something (you’ll get mad if they don’t do it or do it wrong), then they’ll easily come to dread hearing you ask. Fear is terrible for learning. Avoiding poisoned cues falls under being nice, but I thought I’d mention it specifically.

tl;dr Be nice. Work on one thing at a time. Everything else is new tank syndrome.

Brief Clicker Expo Notes

I went to ClickerExpo in January!

The whole experience was terrific, and I think I learned quite a bit. I got a few new ideas, but not many. But I do think I understand many of the things I’d previously read about it more fully and in a way that’s more actionable. Real-life examples are invaluable when it comes to updating all parts of my mind.

I jumped around between talks quite a bit, and when I wasn’t compelled by any of the speakers I went to the hands-on “learning lab” clicker sessions with dogs. 

This post is a bit shorter than I had originally intended. I realized that I wasn’t getting it written so I lowered my criteria :-).

Here are some rough notes:

  • I was impressed with Karen Pryor as a speaker. I found her quite engaging–particularly her talk about how most scientists aren’t actually using the conditioned reinforcer properly. Yet another lesson in how crazy the world is, and in particular how messed up our scientific institutions are. Seems like things are slowly improving though in this area!
  • I saw some fun videos about some dogs distinguishing between one, two, and three objects fluently.
  • There was quite a bit about emotions. Everyone agreed that behaviors originally associated with angry, scared, or other upset emotions do not retain their original tone when captured with clicker training. In particular, the faculty noted that clicker training dogs to act ferocious didn’t work very well, and that this had been a challenge for people training dog actors for commercials. And there also seemed to be consensus that behaviors would acquire the emotional tone of the reinforcers used to train them. I updated that using multiple marker signals with different emotional tones was an excellent idea. (I’d heard of this idea before but hadn’t been compelled by it.)
  • I both got some good heuristics for good shaping and a bunch of hours observing good shaping. I think I (and most amateur shapers) should raise criteria more slowly and take more breaks). It’s hard not to be impatient and overenthusiastic! One thing I hadn’t considered was that jumping ahead too quickly with criteria increases both the likelihood that garbage behaviors will get chained in and that the learner will then get stuck at a particular level.Variability is worth preserving.
  • Verbal stimulus control is hard with dogs! Argos and I have been practicing since ClickerExpo to try to get him to listen to my actual words, but it was actually fairly heartening to see in the learning lab that many of the professional dog trainers hadn’t done any better with their dogs.
  • Kathy Sdao gave a lecture on Premack’s principle that I found quite inspiring. It didn’t contain any entirely new information, but it was a very welcome refresher. I had heard the idea of reframing distractions as the best possible rewards for dogs, but I hadn’t actually thought of it that way for Lydia. I can tell I have a lot to learn about this whole clicker baby/toddler business, and I do think I’m slowly learning it.
  • I also went to Kathy Sdao’s lab on cues. If anything, I was comforted that the professional trainers who had brought their dogs didn’t seem to have much better stimulus control than I have with Argos for verbal stuff. We’ve been working on that since I got back, but he’s still confused about sit vs. down, since he often goes down by sitting first, so it has historically worked well enough for him to guess. I’m trying to clean it up bit by bit.
  • I had heard that Clicker Expo was an unusually positive and rewarding atmosphere. Makes sense. Not sure i would give the same review, but I think I’m implicitly comparing it to events I futurist-type events I usually go to that are usually full of my friends and people in my general demographic. I did get a few genuine compliments each day though, which was nice!
  • I’m now part of a Clicker Parents Facebook group! I was going to start a list, but then I found out that already existed.

Since it’s been a few weeks now since I attended, I can confirm that it seems totally worthwhile that I went. It’s impossible to run controlled experiments with my n=1 parenting situation, but I can see that my relationship with Lydia has changed for the better in ways that seem directly attributable to stuff that I learned and internalized at Clicker Expo. That’s what I wanted! I’m now working on writing up the details of the changes I’ve made in perspective and how we do things.

Anki Cards for Don’t Shoot the Dog

I made some Anki cards for Don’t Shoot the Dog to help me better internalize the material. This deck isn’t intended to be a general overview of the most important concepts–just stuff that seemed important and I didn’t think I already fully understood. Your mileage may vary, but here it is!

Don’t Shoot the Dog.apkg

(Positive and Negative) Punishment vs. Negative Reinforcement

I just reread Don’t Shoot the Dog, which I recommend extremely highly. Here’s a good summary, but actually just read the whole thing because it’s worth it. There isn’t a kindle version, but if you google for a pdf that will work.

Anyway, I read Don’t Shoot the Dog a while back, then more recently I’ve been reading tons of stuff about modern dog training since we’re getting a dog this coming weekend (!!!). The stuff I’d been reading recently talked about the four quadrants of operant conditioning: positive punishment, negative punishment, positive reinforcement, and negative reinforcement.

In that context, positive means adding something and negative means taking something away. So, kicking a dog when it did something you didn’t like would be positive punishment. Walking away when the dog did something you didn’t like would be negative punishment. Giving the dog a treat for doing something you liked would be positive reinforcement. Pinching a dog’s ear until it did what you wanted would be negative reinforcement.

And then there’s extinction, which is stopping responding to something that previously worked. That can work too, if the behavior isn’t self-reinforcing. Parents use this when they leave babies to cry themselves to sleep. The baby is crying to attract the parents, so eventually the baby stops crying.

An important message from Don’t Shoot the Dog is that punishment does not reliably work, whether it’s negative or positive punishment. 

[Read more…]

Self-Improvement: What It Is And Why We Care

We like to talk about this concept we call “self-improvement” a lot. On the face of it, it’s a relatively simple and easy concept to understand: we are improving our selves. End of post!

…except it’s still a little bit vague what I mean by that, even in my own mind. Let’s forget about defining the “self” for a moment and just talk about “improvement”. By what standard are we judging improvement exactly? It’s not usually that clear cut. I might think adding delicious bacon to this dish is an improvement, but a vegetarian would beg to differ. Or to make it more personal, I might become a more assertive person, but to other people around me that might be relatively more off putting than allowing them to always get what they want.

Ultimately improvement ends up getting defined by my own standards. That’s one possible meaning of self-improvement: it’s my own improvement thank you very much! This is still only a partial answer, because we’ve passed the buck to the process that is setting our standards. I suspect that in many cases, we have an idealized vision of a human being in our minds, and we are trying to make ourselves look more like that vision. This can be a great motivator, and if human values are widely shared it will produce a great person. You could think of this as the virtue ethics model of self-improvement. [Read more…]

Learning Programming

My relationship with programming has a long and sordid past. We got our first computer when I was very young, and I was immediately transfixed by this devil machine. How did it possibly work?? This curiosity got me a handful of vague enough answers to temporarily satisfy me, and I went about playing Dune II. I remember at one point creating a file, adding .exe on the end, and telling people that I “wrote my first program”. When they asked me what it did I had no answer.

Years later I started to realize that I wanted more. I wanted to be able to command the machine. I knew this was possible to do through this thing called “programming”, though this was still vague. At one point I even said I wanted to be a programmer when I grew up. In retrospect, this skill seems valuable enough that I wish my parents had given me more of a push / support when I was younger – certainly we intend to introduce Lydia to Logo at a young age ourselves.

As it so happened, my dad did at one point give me a book about Java. I proceeded to do a few lessons, marvel at my ability to make the computer prompt me and spit out some text, and promptly forgot that it existed. [Read more…]