Oddly Influenced

Welcome to Oddly Influenced, a podcast for people who want to apply ideas from *outside* software *to* software. Episode 52: Emotions as concepts

Since the 1960’s, a popular theory of the brain divides it into three parts. At the bottom, there’s the “reptilian complex,” which does instinct, basically. Then there’s the “paleomammalian complex” or limbic system, which does emotions (among other things). And finally: the neomammalian or cerebral cortex, which is where language, abstraction, planning and all the stuff that makes you consciously *you* lives.

This turns out to be wrong. What’s right is up for debate. Here I’ll take the side of those who see the brain as largely or primarily a prediction engine. I introduced that idea in episode 49; here, I’ll elaborate on it and apply it to emotion.

This episode is something of a sequel to the last one. There, I claimed that memory isn’t just passive retrieval; rather, specific memories are *built* on demand. Emotions seem similarly passive – they just happen – but a lot of work goes on behind the scenes. It turns out – at least according to the claims of people like Lisa Feldman Barrett and Andy Clark – that becoming afraid (for example) is not dissimilar from thinking about a journey or a bicycle or money. The emotion “fear” is learned in a way similar to how you learn about money, and the mechanism underlying the brain state “I am afraid” is not that different from the one behind the brain state “I am thinking about bicycles.”

The subjective effects are very different, clearly – no doubting that – but how brain states turn into subjective experience is called “the hard problem of consciousness” and I declare it Out. Of. Scope.

≤ music ≥

I’ll start with three examples that need explanation. The first is a personal experience from Barrett’s book /How Emotions Are Made/.

“Back when I was in graduate school, a guy in my psychology program asked me out on a date. I didn’t know him very well and was reluctant to go because, honestly, I wasn't particularly attracted to him, but I had been cooped up too long in the lab that day, so I agreed. As we sat together in a coffee shop, to my surprise, I felt my face flush several times as we spoke. My stomach fluttered and I started having trouble concentrating. Okay, I realized, I was wrong. | am clearly attracted to him. We parted an hour later — after | agreed to go out with him again — and I headed home, intrigued. I walked into my apartment, dropped my keys on the floor, threw up, and spent the next seven days in bed with the flu.”

She mistook incipient symptoms of the flu for attraction, an emotion. How did that work?

In *his* book, /The Experience Machine/, Andy Clark describes an experiment. In it, a set of heterosexual males were asked to rate images of naked women for attractiveness while at the same time hearing the lub-dup lub-dup lub-dup of their own heartbeat.

Except, of course, the researchers lied. They sometimes played the sound of a different heartbeat, a faster one. The subjects who heard the artificially faster heartbeat rated women more attractive (on average).

How did that work?

The last example is the Capgras delusion, in which a sufferer believes that a loved one has been replaced by an imposter. The impersonation is very good, but the sufferer can usually spot the subtle flaws in the act and is frustrated or angry that others cannot.

To give an explanation for these, I’ll have to lay some groundwork, so here goes.

In previous episodes, I’ve talked about the perceptual system, which delivers information from outside the body to the brain. I’ve also talked about the proprioceptive system that reports on the position and movement of body parts, which I guess sort of straddles the line between outside and inside. Now I’m going to make use of the so-called interoceptive system, a term I was only able to remember by thinking of it as a portmanteau word combining “interior” and “perceptive.” (“Interoceptive” comes from the Latin for “inward looking”.)

The interoceptive system tells the brain of the state of the inside of your body: things like blood sugar level, body temperature, gut fullness, the tightness of capillaries, and so on. The brain has to attend to those things: if the bloodstream is getting low on water, the brain has to cause thirst, else its host will die.

Interoception has to have some overlap with emotions. Suppose you’re walking down the mean streets of Anchorage, Alaska (which has a slightly higher rate of violent crime than Chicago, Illinois). A man steps out of a dark, cold alley and points a gun at you. You instantly feel fear. Fear is partly a physical thing: your pupils dilate; your blood vessels direct more blood to the muscles and less to the skin; heart rate and blood pressure increase; your liver starts dumping glucose into the blood stream; your airways arrange to deliver more oxygen; and so on.

The mechanism seems intuitive: you become afraid, so the brain instructs the adrenal glands to flood the bloodstream with adrenalin, which causes the physical effects.

However, people like Barrett believes that causality is backward. Fear doesn’t cause physical effects; interoception of physical effects causes fear.

Consider Barrett’s date. The physical effects of incipient flu were close to her past experience of attraction, so her brain interpreted them as attraction. After all, she was on a date and felt a little flushed, her heart was racing a bit, and so on – it sure seems more likely that the date experience is producing attraction than that she *just so happens* to be getting sick. I mean, what are the odds?

So the brain reasons backward from symptoms to the most likely cause. (The technical term for this is “abduction”, a kind of logic distinct from deduction and induction.)

The problem the brain has is there’s no unambiguous mapping from a set of interoceptive signals to an emotion. Here’s Clark:

“multiple experiments find no neat, recurrent ‘bodily fingerprints’ for the different emotional states we seem to experience. There is no single set of bodily responses that is unique to sadness, or shame, or any other of the many emotional states we name in daily life.”

It seems Barrett’s weird date is not an exception: the brain is always having to pick *which* emotion corresponds to ambiguous interoceptive signals – much as it has to handle ambiguous and noisy signals from ordinary perception. It does that by using other information, specifically past experience. Barrett had been on dates before. Sometimes they led to attraction. So it seems abductively reasonable to assume it had happened again.

Similarly, the feeling of happiness depends on interoception, but it’s more likely to happen if you’re in the kind of situation where you’ve been happy before – or, importantly, in the kinds of situations that society tells you are associated with happiness.

Clark again:

“What so often seem to us to be raw feelings or emotions are in fact already highly informed guesses about how things are: guesses that are based (even though we are seldom aware of this fact) on a surprisingly wide range of evidence, expectation, and information.”

≤ short music ≥

So let’s move to the men whose opinions about attractiveness could be influenced by recorded heartbeats. What’s happening there?

Barrett and Clark believe the brain is a predictive engine of the sort I described in episode 49. I find this hard to think about because it’s so backward from my normal assumptions about cause and effect. For example:

While I was typing what I’m now speaking, I decided to turn and look at the auditorium on the opposite side of the University of Illinois quadrangle, something I’ve been seeing, off and on, since 1978. I know what the view looks like on a sunny summer day. So my brain could predict I was doing just that: seeing a big green lawn flanked by big green trees with a big red brick building at the far end.

That prediction was wrong because I was in fact seeing a laptop screen with text on it. This produced what’s dubbed “prediction error.” In this case, the error was huge, so it prompted corrective action. That is, some part of the brain decided to turn my head right until the prediction error from what I was seeing was sufficiently small. My brain faked a perception, then acted to make it true.

Such a high-level action is a good distance from the motor neurons and never reaches them. Instead, it reaches other parts of the brain, each of which interprets it appropriately.

So whatever controls my neck muscles specialized the top-level action into a prediction about proprioception: that my head is rotating. It’s not, so neck muscles are directed to make it so.

Meanwhile, there’s a prediction that the muscles that control the lens of my eye are relaxed to let me see long-distance. Wrong again, it turns out: they’re contracted to provide a near-field view of my laptop. Motor neurons are therefore instructed to relax the lens muscles until they “feel right.”

All of this happens very fast. At some point, the prediction error was small enough that the action could be considered completed, prompting a prediction that my head is no longer turning. Since it was, an action was taken to stop it.

This is crazy, but it’s a respectable theory for how the brain works.

The theory applies to the experiment where young men were ogling women while hearing a heartbeat, and therefore bumping up their perceived attractiveness. (The women’s, not the men’s.) Here’s Clark:

“[T]he finding starts to make sense if we consider that the brain should already be quite good at predicting the actual heart rate, so the false feedback leads to prediction error. That prediction error causes the subject to attend more strongly to the stimulus, making them experience it as somehow ‘important.’ It is that added salience that is then reflected in the increased attractiveness ratings.”

Or, to put it crudely: “my heart’s racing. She must be hot.” (Notice the similarity to Barrett’s experience.)

One weird thing here is that the brain is for some reason believing the evidence of the ears instead of just tried-and-true interoception. Clark doesn’t say how that works. However, consider the question of whether you feel like finishing some food in front of your face. That depends on the taste of the food, but also its smell (which is why food seems bland when you have a cold). It also depends on the state of your gut: is the brain getting interoceptive signals that say you’re full? It can even depend on whether you’ve just been told the dish contains cilantro and you *hate* cilantro. Now you can taste it, which makes the dish less appealing. Information has modified perception. Or: past experience with cilantro informs your perception of the present food.

It seems as if the part of the brain that matches expected perception against actual perception is fed some sort of complicated multi-dimensional or multi-modal summary of what’s going on. So it’s not so implausible that the brain combines the sound of heartbeat with interoceptive information to produce a summary that reports the heart beating faster than it actually is.

This explanation is slightly more plausible because you actually can hear your own heartbeat if, say, you press your ear against a surface. Maybe that faint-to-inaudible sound is part of the brain’s monitoring of the heartbeat – and maybe the sound is made more salient because it’s so much louder than normal. Which brings us to a new term:

“Precision.” This is not a great term, in my opinion. “Sensitivity” might be better. The idea is that prediction error is not a binary that either matches or doesn’t. It’s not even a numerical tolerance, as the word “precision” implies.

I’ll revisit the University of Illinois quad for an example.

It would be a mistake to think that, when I got my head turned, what was delivered to my brain was a high-resolution snapshot to be viewed by some little “me” inside my head. What I *saw* was the prediction, which my eyes had confirmed was close enough to the facts.

Suppose you then asked me if there were students lounging on the quad. I might just have answered “yes.” It was the afternoon of a nice summer day. There are always students on such days. No need to check.

Alternately, the brain might predict that there are students and instruct the eyes to check. But what does that prediction mean? It’s surely a mistake to translate it into words, but what choice do I have? So let’s say the prediction is just that there are one or more non-green fairly-contiguous splotches of a fairly small size against the green background. This requires more precise information-gathering than my previous task, but it doesn’t have to be too precise. For example, I don’t need a mental image that I can then mentally count, because I haven’t been asked for a count.

As it happens, there were five splotches, which I knew instantly. That’s because I was basically replicating an experiment from 1949, described like this by George Miller:

“Random patterns of dots were flashed on a screen for 1 / 5 of a second. Anywhere from 1 to more than 200 dots could appear in the pattern. The subject's task was to report how many dots there were. […] on patterns containing up to five or six dots the subjects simply did not make errors.”

This is called “subitizing.” It’s an example, I think, of how the data the brain works on is a combination of the prediction (“one or more splotches”) with the somewhat-more automatic perception that the number is five. The brain didn’t settle for a prediction and a prediction error that was just good enough. Because it was easy, it appears the prediction was updated with reality to better serve as the raw material for brain-work.

But if the number had been greater than whatever my subitization limit was, the augmented prediction wouldn’t suffice and I’d have to spin up a new task to move my eyes’ focus from blotch to blotch, counting as I went.

Even though I instantly knew the number of blotches, I didn’t know the number of *students*. As it turns out, there were two pairs and three singletons for a total of seven people. To determine that, I had to look at each blotch in turn and resolve it into the number of people… well, no – according to Barrett and Clark that’s not the right way to think of it.

Rather, my brain predicted the count for each blotch while or before my focus was moving toward it. That seems crazy, like some bizarre version of Twenty Questions: “Is it one? No? Well is it two? No…” and so on. But the brain will frequently make several predictions at once. I don’t know what my brain actually did, but I think it’s plausible that it made four simultaneous predictions:

- The blotch is single and undifferentiated. (That is, a single person.)
- Or the blotch is two separate smaller blotches (two people)
- Or the blotch is two overlapping blotches, perhaps a couple embracing or with one’s head lying in the other’s lap. The brain knows that’s a likely prediction based on past experience: lotsa romantic couples come to the quad.
- Or the blotch is three separate blotches.
- Or it’s more complicated and I’ll have to look more sequentially. (That is, I think I could recognize three non-overlapping sub-blotches instantly, but four seems beyond instant recognition, given the messiness of the possible images.)

Barrett has it that the different predictions have different prior or starting probabilities (based on experience) and talks about Bayes’ Theorem for resolving cases where two predictions are compatible with prediction error and precision, but I’m not going to go there.

Instead, I’ll move on to Capgras syndrome. One theory is that it’s due to a deficit in interoception and required precision.

The words you hear now are being written two days after my observation on the quad. As you may have noticed, I’m slow at composing scripts. In any case, in about an hour (as I write), I’ll walk out on our front stoop and sit down next to Dawn. I’ve been going from a no-Dawn-near-me state to a Dawn’s-in-view state more than daily for a bit over 37 years. My brain knows what to expect, interoception-wise. There will be a slight increase in heart rate, a little bump in galvanic skin response. That’s normal in the presence of a loved one.

According to the predictive brain theory, my brain will be actively predicting just that. But what if it didn’t happen? That’s a large and unexpected prediction error. Here’s Clark:

“These missing bodily responses […] are not consciously registered. But their sudden absence acts as evidence that the ever-whirring predictive brain needs to explain.”

We all have weird temporary mental experiences – deja vu is one of the more popular ones – but suppose my brain gets fixated on the interoceptive oddity. And suppose, while we’re looking out at Dawn’s easement garden, that I mention again how peonies are the least appealing possible flower because – well, they *are*, what with their droopy stalks and all. My brain – well trained from past experience – predicts Dawn’s patient “yes dear” reaction, expression, tone of voice, and so on.

But there’s some allowable prediction error. There has to be – her reaction isn’t identical, time after time. But, the theory goes, with Capgras syndrome, the precision – the allowable prediction error – contracts. For the first time, Dawn’s reactions are outside it. Here’s Clark:

“Meshing in the new evidence, the Capgras sufferer’s visual and auditory experiences become subtly reconfigured. Perhaps the person’s smile now seems slightly different, or their voice sounds a little higher. [These] subtly altered visual and auditory experiences that follow then put the Capgras patient in the strange position of seeming to have gathered additional perceptual evidence that something important has changed. The loved one ‘feels different’ and they also look and sound subtly different. This plausibly sets the scene for the emergence of the full delusional belief that the loved one has been replaced by a similar-looking (but not quite perfect) imposter.”

≤ music ≥

Where does this leave us?

There’s a good deal I don’t understand about the predictive brain. Barrett’s and Clark’s books are lighter on detail than I like, and the paper of Barrett’s I’ve been using is written for a specialist audience and has a good deal of detail about brain structure that I don’t understand the significance of. Both I think are particularly weak on dynamics of the sort I tried to give in my story of the Illinois quadrangle.

For example, I got hung up for some time on this question: if the brain constructs fear from interoceptive signals compatible with that emotion, what caused those body states in the first place? It has to be the brain, right? Which gives us a chicken-and-egg problem.

I think you could integrate Barrett’s account with Chemero’s approach in his book /Radical Embodied Cognitive Science/ (episode 43) and see fear as a growing and self-reinforcing dynamic loop involving the state of brain networks and organ systems. That is, a robber jumps out at you, which triggers some automatic reaction in the brain that doesn’t yet deserve to be called fear, which causes some changes to the gut (for example), which violates a prediction from the “am I afraid?” part of the brain, which leads to another prediction – one related to fear – taking hold, which has as one of its effects to push the gut to close down more, which is more consistent with a prediction of fear and on and on and on, round and round we go.

However, I’m a dilettante, and it’s not my job to fill in such details. Though I would be fascinated to interview or correspond with someone who really knows this stuff. Hook me up, listeners. In the meantime, let me just paint a summary picture by working up to the difference between “fear” and “dread.”

Emotions are special to Barrett because they’re largely driven by interoception, whereas skillful movement is largely centered around proprioception and understanding the concept of “doggie” focuses on the role of external perception.

However, none of these depend only on “-ceptions.” All require learning from experience. A child learns the concept of “doggie” by creating a “statistical summary” of encounters with doggies. A baby learns to control their limbs via a long series of flinging them around and seeing what happens. And “fear” is built from being in scary situations and learning to recognize both the interoceptive consequences and also what kinds of situations are scary..

Barrett thinks that concepts like “doggie” are composed of doggie-related predictions and doggie-related actions that have been collected together in a single neural network. Fear works the same. There’s a network that’s been trained up by observations of dangerous situations that coincide with the relevant interoceptive signals.

That is, we learn what fear is the way we learn what doggies are. That has some weird implications.

A big part of how we learn concepts is by cuing learning with names. What if no one ever actually uses any name for doggie around a child? What if everyone just points and says “Look at that?” with no further detail. How well will the concept be learned? (Please don’t try this with your own children, but if you do, let me know how it turns out.)

A metaphorical way to think of it is that the use of a name activates its neural network, preparing it to learn from the current situation. (Clark suggests something like this in his 1997 book, /Being There/, covered in the unnumbered episode just before episode 41.)

But it seems weird to think that never learning the word “fear” would make a different about how the brain reacts to scary situations. Would a pre-verbal child *feel* fear differently? Would a dog? However, that brings up the hard problem of consciousness, and I want to go somewhere else.

We have lots of words for kinds of fear: terror, horror, dread, and so on. And dread, for example, feels different than panic, doesn’t it? And the actions a feeling of dread prompts are different than those prompted by horror. Since Barrett believes concepts, be they “doggie” or “dread,” are defined by the predictions and actions of neural networks, does that mean “dread” is a separate network than “horror”?

In the paper I’m using, Barrett doesn’t mention such gradations of emotion, but she does mention a biological concept that’s perhaps applicable. It’s “degeneracy,” a strong contender for 2025’s Marick Award for “jargon that most misleadingly hijacks a common term but means something completely different.”

Degeneracy is, broadly, the capacity to get a given outcome in different ways. For example, the amino acid arginine is coded for by no less than six three-letter DNA sequences. If the first two letters are CG, the last can be any of the four possibilities. However, just for fun, apparently, the sequences AGA and AGG code for arginine too. But not AGU and AGC: those code for serine.

Barrett says the brain makes heavy use of degeneracy:

“Natural selection favors systems with degeneracy because they are high in complexity and robust to damage. Degeneracy explains why Roger, the patient who lost his limbic circuitry to herpes simplex type I encephalitis, still experiences emotions and why monozygotic twins with fully calcified basolateral sectors of the amygdala […] have markedly different emotional capacity, despite genetic and environmental similarity.”

I need to note that, in biology, complexity is a good thing – and for the same reason it’s a *bad* thing in software:

“Natural selection prefers high complexity systems [because] they can reconfigure themselves into a
multitude of different states.”

When it comes to concept formation, degeneracy allows:

“dissimilar representations (e.g. different sets of neurons) to give rise to instances of the same category (e.g. anger) in different contexts (i.e. many-to-one mappings of structure to function). […] “

…and:

“In emotion research, degeneracy means that instances of an emotion (e.g. fear) are created by multiple spatiotemporal patterns in varying populations of neurons.”

Note that degeneracy is *not* redundancy. It’s not like in databases, where you might have a “hot spare” database ready to take over if the primary server goes down. True redundancy is inefficient and expensive. So suppose you’re working with the concept of fear. There are N identifiable neural networks corresponding to fear. They do not learn about fear in lockstep: that is, the statistical summaries they embody will vary between them, as will the predictions they make, and the actions taken in response. They just have to be close enough to be useful as the body does its business.

These N degenerate representations of fear do not require N times as many neurons because, remember, your typical neuron is involved in many networks simultaneously. So instances of what we call the “same” concept – fear – will physically overlap. It’s likely that a lot of neurons are shared among all instances, some are used in several networks but not all, and some “belong” to only one network in the set (but are simultaneously in networks that have nothing to do with what we think of as fear).

Barrett puts it this way:

“conceptually similar representations reuse neural populations […] As a result, different predictions are separable, but are not spatially separate.”

That lets me speculate how the difference between dread and horror works.

Early in life, a child learns about fear. Fear is “implemented” by N (overlapping) neural networks, that differ in various ways, including which sorts of experiences they summarize. Now consider a new experience. Let’s just say, completely arbitrarily, that an 10-year-old didn’t do his report on the history of Illinois. He had a foolproof plan for dealing with the consequences: he would say he had too handed it in but the teacher must have lost it. Despite that, he spent the weekend after the due date fearful of what would happen on Monday.

Of the N neural networks devoted to fear, one was the strongest and let’s say it contributed most to the discomfort of those two days. Since the weekend was memorable, it further helped refine that network. Since the weekend was less relevant to the other networks, it refined them less or not at all.

At some point, the child made an association between the word “dread” and that experience. “Dread” thus becomes attached (or more strongly attached) to that particular network. Later that child, let’s just say, became a fan of cosmic horror of the sort associated with H.P. Lovecraft, which is fiction strongly associated with dread (rather than, say, shock or the gross-out). Those reading experiences will shape the “dread” network more than they do the other “fear” networks, so it becomes specialized to the feeling of dread.

In a sense, the child has learned a new concept, closely related to or associated with, the earlier concept of fear. Since such networks cause neural actions that eventually, via some unexplained mechanism, get translated into subjective experience, the feeling of dread becomes a specialized or different kind of fearful feeling.

That means that when that child grows up and becomes, let’s say, a podcaster and reflects back on the experience of his 55-years-ago self, he is projecting a feeling he’s now good at onto a person who hadn’t yet learned dread but was actually experiencing something closer to generic fear. Remember how, last episode, I distinctly remember driving seated on the left side of the car while in Ireland – even though I was actually on the right – because my brain mixed-and-matched stock images to construct a false memory? In the same way, this memory has had my current understanding of dread patched in.

Oops, I gave it away. That child was me.

That’s a weird thought: that I now remember a feeling and emotion I was at the time incapable of – but I don’t think it’s inconsistent with Barrett, as least insofar as far as I understand her.

≤ music ≥

So. That does it. A whirlwind tour of emotion research, revealing that abstract concepts and emotions look different from the outside but share a huge amount of implementation behind the scenes. Next time, I’ll tie this whole series together with a final episode on:

- how conversation works
- how metaphor fits in
- how metaphors for programming concepts fit in, and
- since I’ve gotten interested in the federated wiki, what this implies for hypertext explanations.

Thank you for listening.