Oddly Influenced

Welcome to Oddly Influenced, a podcast for people who want to apply ideas from *outside* software *to* software. Episode 41: The offloaded brain, part 1: behavior

The theme of this series is that we think too much. We’d be better off if we more often arranged for our environment to push us around. Be thoughtful a few times so you can be thoughtless the rest of the time. And, as embodied beings, we routinely solve problems using a bag of bodily tricks that minimize our need to think. Yet, at work, we act as if our only tool for problem-solving is our brain, as if we were the mythical “brain in a vat” or an “uploaded consciousness.”

I’m going to draw mostly on two books published in 2011: Louise Barrett’s /Beyond the Brain: How Body and Environment Shape Animal and Human Minds/ and Anthony Chemero’s /Radical Embodied Cognitive Science/. I also make some use of Andy Clark’s 1997 /Being There: Putting Brain, Body, and World Together Again/, as well as other references you can find in the show notes.

In this episode, I’m going to look at at animal and human behaviors that don’t seem particularly intelligent, such as catching balls and judging distance. I’m going to use them to extract “design principles” that you might take advantage of in your day-to-day work, or use to understand why certain techniques work as well as they do even though rationally other techniques are just as good.

Although the behaviors described here will be basic, I think the principles can be used to support work-time behaviors more complicated with catching balls. To keep this episode from running on too long, examples of that will be in the next episode. With luck, some of those examples will come from people other than me.

In the episode after that, I’m going to talk about behavior that at least seems more intelligent, like planning and learning and making models of the world.

≤ music ≥

I’m going to talk about certain animal behaviors that solve specific problems. All those behaviors evolved, and it’s really hard to talk about evolution without sounding like it has a *goal*, that there’s some designer who’s consciously setting out to solve a problem. So I’m not going to try. In fact, I’m going to lean into it and pretend that *you* are that designer. Purists will object, but I’m trying to extract design principles that you, as an actual human designer, can put into practice when designing your work environment, and it would be too painful and boring to be scrupulous about evolution.

As a designer, you’re going to have to deal with constraints in the normal engineering way. In fact, that gives me an idea. You see, a problem with the books I’m drawing from is that they’re about a single variant of cognitive science that has *two* names: either “ecological cognition” or “embodied cognition”. That bugs me because “ecological” shortchanges the role of the body, and “embodied” shortchanges the role of the environment. This is a field that’s all about the body, environment, and brain as a tightly coupled system, so neither name really fits.

Therefore, I’m going to address you as a EE, reminiscent of both “ecological” and “embodied”, but also of “electrical engineer”, which highlights the “working within constraints” bit. In fact, *electrical* engineer is particularly appropriate because, first, neurons deal with electricity and, second, the solutions I’ll be describing have a family resemblance to electrical circuits. They don’t look very much like programs running on a computer, which is the dominant metaphor in mainstream cognitive science. In fact, as a EE, your hope is to solve as many problems as you can without using computation: that is, stored state, algorithms, and domain models.

You see, your major constraint is that neurons are ridiculously expensive. If you’re sitting right now, as opposed to listening to this podcast while running a marathon – something I understand is very popular – your brain is consuming 20% of your body’s energy, despite being around 2% of your body weight. If I give you the task of solving a new problem, you – as a good EE – will want to solve it using as few neurons as possible. *Assuming you can’t repurpose neurons to do double duty* (I’ll come back to that), you’ll want to offload work to, for example, cheaper cells in the body, like muscle or bone cells. Or, if you can take advantage of properties of the environment without expending a lot of energy on movement, you’ll do that.

Coming back to the assumption about repurposing neurons. Mainstream cognitive science is inspired by the generality of the stored-program computer, the Turing Machine, lambda calculus, pick your favorite conceptual device that runs algorithms. Part of that inspiration includes what, in some object-oriented designs, is called a God Object: that is, the single object that knows everything about everything, so that any problem can be solved with code that uses the information already available from the God Object, just in a new way.

So, for example, mainstream cognitive scientists might assume that your brain contains a three-dimensional representation of the world around you, constructed by computing on the two two-dimensional arrays of information delivered by your eyeballs. A new problem can be solved by looking at that existing data in a new way.

EEs like you, however, are inclined to think that God Objects are too expensive. Instead, there will be a larger number of objects that represent just enough of the world to solve a certain class of problems. They can accommodate new problems, but not in a fully general way. For example, traces of their origin will show through, and they might have an air of what Brazilians call “gambiarra” and we English speakers call kludges.

≤ short music ≥

It’s high time for an example: stick insects. They have six legs, and walk. The legs have to move in synchrony or else the insect will keep tripping over itself. *Obviously* there have to be some neurons that coordinate the legs. You won’t be surprised, given how I emphasized “obviously”, that there aren’t any such neurons. The legs of stick insects are structured and physically interconnected in a way that makes coordination automatic.

Whichever EE designed the stick insect won the design competition against the one who used neurons to coordinate. That winner avoided a constant coordination expense by using a one-time expense: building the right bodily structure. And – who knows? – the winning legs might even be cheaper to build than legs designed for coordination.

Your own legs are more capable than stick insect legs, but they’re similarly autonomous. The bones and joints and especially springiness of human legs means that, once started stepping, the next step is both automatic and efficient. (The same is true for running, part of the reason we’re the long-distance running champs of the animal kingdom.) The show notes link to a video that shows a passive walking robot that looks like a metal model of a human from the waist down. Once started, it can walk on a treadmill forever, powered only by a slight downward slope. Nothing controls it but its construction.

≤ music ≥

Our first design principle is “Favor direct control links from perception to action.” The more usual jargon is “direct perception”, but I found that not helpful as I tried to understand. “Direct control link” is from Ron McClamrock, who describes how flies launch themselves into the air thusly:

“[Flies] don’t take off by sending some signal from the brain to the wings. Rather, there is a direct control link from the fly’s feet to its wings, such that when the feet cease to be in contact with a surface, the fly’s wings begin to flap. To take off, the fly simply jumps and then lets the signal from the feet trigger the wings.”

The brain tells the legs to jump, which automatically tells the wings to flap. The fly saves on the cost of coordinating the two.

That isn’t to say that there’s no coordination in direct control. If you try to stomp on a cockroach, it will start turning to scurry away within 58 milliseconds. The triggering direct perception is when two little pointy bits near its butt detect air accelerating beyond 0.6 meters per second squared. However, somewhat more than 100 neurons add contextual information – like if the cockroach is adjacent to a wall – that modulate the details of how it escapes. But the whole assembly is all about the single task of escaping, triggered by a key perception.

So you, as a EE, might add neurons to cope with necessary complexity, but you’ll always be on the lookout for ways to offload work onto the body. Crickets make a nice example.

Crickets, like you, have two ears on either side of their body. (In the cricket’s case, they’re on the forelegs.) Like you, the crickets can tell what direction a sound is coming from. Crickets use a different mechanism than your ears do, and it’s too hard to explain without drawing a picture.

Unlike you, crickets (as far as I know) are only interested in one sound. Female crickets want to sidle up to the male that’s chirping the loudest. Only a high-quality male can keep up loud chirping for long.

If the female is pointing off to the left of the male, the right eardrum will be moving at a higher amplitude. (That’s not just because it’s closer to the male. There’s some complicated jiggery-pokery involving sound waves arriving at the eardrum both directly, via the air, and indirectly, via a “tracheal tube” that runs from a hole in the cricket’s head, down into the body, and along the foreleg to the inside of the eardrum. That amplifies the amplitude difference between the left and right eardrums.)

There’s a direct connection from the eardrum’s sensory neurons to a small cluster of what I’ll call “calculational” neurons. It all ends up in two neurons, one for each side. The one with the strongest incoming signal fires first, signaling motor neurons to turn in its direction. So the female turns toward the male, as automatically as a fly starts flapping.

That sounds good, but wait: isn’t this assuming that the female exists in a world populated solely by crickets of her own species? What about all the the non-cricket noises? And how does she keep from getting seduced into a torrid but barren affair with a robust male cricket of some other species?

There are two mechanisms.

First, every species chirps at a different pitch. And the female’s tracheal tube is tuned to her own species. The chirps of other species are greatly attenuated. So a louder foreign cricket doesn’t *sound* louder to the female.

Second, a single cricket chirp is actually a series of “syllables”. It’s really chirp-chirp-chirp-chirp-chirp, just so fast it all blends together to us. And the spacing between the syllables is different for each species.

The female cricket’s calculational neurons are tuned to the species’ syllable spacing. I don’t understand the mechanism, but I think it may be similar to how rabbits’ smelling neurons are tuned to the rate at which rabbits do their sniff-sniff-sniff thing. I discussed that in the last episode, but briefly: a whiff of carrot starts rabbit neurons firing in a self-reinforcing cycle where the output of the neurons feeds back into the input. The reinforcing cycle will gradually die out unless repeated whiffs make it persist. The sniff rate of a rabbit is such that each new sniff amplifies the cycle, the way you can swing higher and higher in a playground swing by “pumping” with your legs at the right moments. When a threshold is reached, the signal “I smell carrot” goes out to the cortex.

I mention this circularity because EEs seem to use time and timing and feedback loops a *lot* in their solutions, whereas programmers like me want to avoid thinking about time and timing as much as we can.

≤ short music ≥

Programmers of my generation were very much in love with what we called “clever hacks”, where you make use of facts that seem incidental to the problem to solve it more efficiently. The cricket designer used a couple of clever hacks. The first? Chirps have to be at *some* pitch, after all; and tubes just *do* have resonant frequencies. Look what we can do if we make both the same number!

EEs like you might want to read Levy’s /Hackers: Heroes of the Microelectronic Age/ for inspiration. I also link to a nice talk by Guy Steele. And there’s “the Story of Mel” from a programming generation even older than mine (if you can imagine such a thing).

You might also want to read Ed Yong’s /An Immense World/. The reason is that, because of the clever hacks, there’s a sense that – when it comes to hearing – a female cricket really *does* live in a world that contains only crickets of her species. She doesn’t actively *ignore* other sounds; to her, they don’t exist.

The Estonian biologist Jakob von Uexküll coined the word “umwelt” for that kind of thing: to emphasize that different organisms live in radically different perceptual worlds – as do you yourself. Yong’s book is an exploration of the umwelts of various species.

≤ music ≥

The next design principle is: prefer composite values over atomic values. Or, in programming terms: avoid primitive obsession.

Starting in our algebra classes, we got used to solving problems whose values have types or units like length or duration or weight. Sometimes we combine them, like length and duration to give speed, but as a EE you’ll find that nature uses elaborate combinations you didn’t expect.

A nice example is the diving gannet. They’re sea birds that dive into the water from about 30 meters up. When they hit the water, they’re moving 100 kilometers an hour. By the time a fish notices anything, the gannet has gobbled it up. However,

“Such a dive represents an extraordinary coordination problem. Diving gannets must keep their wings spread for as long as possible in order to maintain and adjust their heading toward a target fish in windy conditions. But hitting the water with spread wings would be catastrophic: at sixty miles per hour, wing bones would break. The question here is how gannets manage to retract their wings at the last possible moment, so as to hit the water at the right location and avoid injury.”

It turns out (Chemero claims) that gannets have evolved a direct control link between a specific perception and its wings.

I expect you have no experience as a gannet, but I bet you can imagine riding a bicycle straight toward a brick wall. Let’s say you’re intent on smashing into an X spray-painted on it. As you approach the wall, you’ll see the X getting bigger and bigger, and experience the entire wall moving outward, “centrifugally”, from your focus point. This is called “looming”. “Texture elements radiate out from the center of your field of view as you move toward an object.”

Looming is a measurable quantity. You can also define a so-called “composite” variable called tau. It’s the ratio of the size of an image to the rate of change of that size. Tau can be interpreted as the time remaining until a stationary object (like the surface of the sea) is contacted. It doesn’t matter whether it’s you that’s moving or the object you’re looking at. Chemero writes,

“When you’re trying to cross the street, how far away in meters an approaching car is matters much less than how soon it will hit you. Second, note that tau need not be computed by the gannet. It is available at the retina. Tau, in other words, can be perceived directly.”

Gannets don’t care about speed or distance. They just pull their wings in when tau reaches a certain value.

You do something similar with regard to weight. Amazeen and Turvey made an object they called a “tensor object”. Imagine a short stick. It attaches perpendicularly to two crossed sticks that make an X. Each arm of the X has a metal ring that can be slid in and out. That doesn’t change the actual weight of the object, but it does change how heavy people report it being. The explanation is that the muscles in your wrist sense not the weight of the object but its moment of inertia, its resistance to being rotated. What the body perceives is the object’s *moveability*, not its weight. And what people answer when asked the weight is the moveability, despite the word they use. This can be seen by directly asking about moveability. Ask one set of people “On a scale of 100, with the control object at 50, how heavy is the object you’re holding?” and the other group “…how easy is it to move this object?” and the answers are “nearly identical”.

It’s important to keep in mind that our perceptions are tied to *tasks* or behaviors or activities. A neat example is a set of experiments by Witt, Profit, and Epstein. In the first, they in effect handed a subject a ball and said, “We want you to throw this ball at that target. How far away do you think it is?”

The catch is that some subjects got a ball that was “less throwable” than the one other subjects got, which was “easily throwable”. (Chemero doesn’t say what “less throwable” means: probably weight? Maybe size?)

Those subjects who got the less throwable ball estimated the target was farther away than the others. The hypothesis is that what people perceive when expecting to throw a ball is not *distance* but *throwability* – since throwing, after all, is the task at hand. Only when asked a question is the throwability translated – awkwardly – into a distance.

A bit more support for that hypothesis is a slightly different experiment. The subjects were given one of the two balls, told they were going to *walk* the ball up to the target, and asked the distance. The throwability of the ball didn’t make a difference. Because what the subjects were perceiving was *walkability*, which the ball wouldn’t affect.

A way you *can* influence walkability is by having some of the subjects walk on a treadmill until fatigued. Those people report a longer distance than do non-fatigued people, again regardless of the ball. So walkability is a composite of distance and the state of the body.

I want to highlight the difference between tau and something like walkability or throwability. Tau is an objective measure, like distance. It’s useful to any animal that needs to know the time-to-collision with a large object. We use it when a car is heading toward us; gannets use it when they’re heading toward the ocean surface.

Throwability is less broadly applicable. It doesn’t mean anything at all to a gannet, because gannets don’t have arms. And it’s going to be a big mishmosh of variables, including distance, weight of object, moveability of object, whether the object is too floppy to throw easily, and so on.

When designing direct control links for your workspace, you as the EE will find many more tasks that need big messy, partially subjective variables than tidy, objective measures. If you focus on tidiness or elegance or the things we like in software designs, you’ll end up in a situation akin to that classic management trap of being unable to measure what you want to measure, so you measure what you can and pretend you’ll get the same results. (Spoiler: you won’t.)

≤ music ≥

The previous two principles together imply a third that’s worth stating explicitly: discover or create affordances.

“Affordance” is a word coined by the psychologist James Gibson. It’s a somewhat tricky concept, but the summary is that an affordance is an opportunity for behavior.

The word is today used mostly in product design. If a door opens away from a user, the door handle should signal – or *afford* – pushing. It should be flat enough and wide enough that a hand can fit on it in a pushing motion, with the palm against a surface. If a door opens toward the user, it should afford pulling and not pushing. It should be clear how to wrap the hand around the handle so that the fingers can exert force for pulling.

Every time you’ve pushed on a door instead of pulling on it, some designer has afforded the wrong behavior.

As another example, consider walking or running. As you move, you pick up affordances about the stability of the ground in front of you and automatically adjust your foot placement. Barrett describes running on her habitual route and only belatedly realizing that there was one particular stretch where she left the pavement (which I think is British English for what we in the US call a sidewalk) to run in the street. Even once she realized it, she didn’t understand why, until…

“Then one day I saw it. […] As the street rounded a small green, the pavement became increasingly angled. Running along this angled pavement threw me out of my rhythm […] and so moving onto the flatter road surface allowed me to improve my current state of affairs with respect to running comfortably and efficiently, even though for most of the time I had been doing this, I had been aware neither that this was what I was doing nor why. Even once I knew what I was doing and why, it was often the case that I would end up switching to the road with no anticipation that I was going to do so, nor any recollection of the moment that I made the decision.”

As with the tensor object, the affordance was detected by her muscles’ continual monitoring of forces they’re counteracting, and also possibly by a disruption to the rhythm of running. (Your body is very intent on maintaining rhythm.) That is, your body has been tuned to detect relevant affordances.

It’s pretty noticeable when affordances are missing. Ice on the pavement does *not* afford what we wish it did, which is that you should walk canted forward, waddling like a penguin. That’s why, even if you know the pavement is icy, you may easily end up on your butt after walking normally. The automatic stabilizing mechanisms don’t work.

It’s important to keep reminding yourself that an affordance is perceived *in* the environment, but it’s not a property *of* the environment. It’s instead a message about the relationship between an environment and a task or behavior. The same stretch of ground between you and a target *means* something different if you’re preparing to throw than if you’re preparing to walk.

Moreover, an affordance is a property of the environment, the task, and the body. If the task is walking, the ground before the target will afford differently to a tired body than to a fresh one.

Affordances apply to all kinds of time spans. A gannet’s pulling in its wing is a single event. Walking is extended in time. You don’t scan all the affordances and plan your route; instead, your body is alert to affordances relevant to walking the whole way to your destination.

Even more extended is the task of attending to the chance of a predator-like motion in your peripheral vision. You spend every waking moment doing that. You can’t not do it. And you’ll automatically turn your head to examine a suspicious motion using the more detailed vision of your eye’s fovea.

When you do that, you’re ready to react to another affordance, that of shapes with left-right symmetry, like, for example, a tiger looking straight at you. In fact, there’s another principle there:

Just as perceptions can lead to automatic actions, automatic actions are frequently made to detect new affordances.

When it comes to picking out predators from the background, I’ve never read that our talent at recognizing faces comes into play, but it sure seems plausible. That allows me to push back against an impression that I may have given: that recognizing affordances is innate or hardwired. Many are actually learned. It appears that babies are born preferring shapes that have “top-bottom asymmetry, that is, they [are] top-heavy with more elements present in the top half of the stimulus than the bottom.” The elements in the top don’t need to look very face-like. In fact, when babies are shown pictures of faces, they show the same preference for ones with facial features that are scrambled or upside down as for faces with properly positioned features.

Barrett writes:

“This general preference for top-heavy arrangements becomes fine-tuned to become a specific preference for faces during the early weeks and months of life as a result of experience. Natural selection seems to have provided us with some very basic perceptual constraints on our visual processing abilities that are “experience-expectant”: that is, they require exposure to faces in order to narrow down the category of stimuli to which they respond. Given that a human infant will inevitably encounter a human face within moments of birth […], it is much more cost-effective, from an evolutionary perspective, for an organism to develop with only a very basic face-recognition mechanism and then let all the faces in the environment do the work of refining it.”

A baby quickly learns the most important affordance in her life: the face of her mother.

Learning will become important when you, the EE, create affordances. You can rely on learning, but only on the types of learning humans are good at.

≤ music ≥

The final design principle is: maintain invariants and rhythm. An interesting example of this is the “outfielder problem”. The “outfielder” is a position in the American version of cricket, called “baseball”. Since only about 45% of my listeners come from countries that play baseball, I’ll briefly summarize the game.

A player on one team, call the “pitcher”, throws a ball a good distance. It’s supposed to arrive within an imaginary rectangle, beside which stands a player on the opposing side, called the “batter.” The batter swings a long, thickish, smooth stick of wood to send the ball back in the rough direction of the pitcher. If the ball takes a long, high arching path, it’s called a “fly ball”. Certain people on the pitcher’s team – called “outfielders” – are positioned in various places far behind the pitcher in the hope they can run to the place the ball would hit the ground and catch it before it does. If they do, the batter becomes sad.

The outfielder problem is: how does the outfielder know where to go to catch the ball? It seems impossible for them to calculate the ball’s path using equations of motion, even approximately: the batter is too far away for the outfielder to perceive even the most crude estimate of the ball’s initial speed and angle. There doesn’t seem to be enough time or perceptual ability to both refine the estimates and to get to the right place in time.

Instead, it appears that what outfielders (and frisbee-catching dogs) do is maintain an invariant. The geometry of the situation is such that if you run in a path that keeps the ball as you perceive it moving in a straight line – both as it goes up and as it comes back down – you will naturally end up in the right place to catch it. If the ball starts curving backwards as you run along, you need to slow down. If it starts curving in the other direction, you need to speed up.

This model predicts the outfielder will *not* run in a straight line from the starting point to where the ball will hit the ground. Rather, to maintain the invariant of the ball’s perceived linear movement, they must run in a curving path, accelerating at first, then decelerating as they get closer to the catch. And this is in fact what happens. (It doesn’t hurt that the human visual system is much more sensitive to deviations from linear movement than it is to speed and angle, even though “speed” and “angle” seem more fundamental. Composite variables again.)

Walking or running are other examples of maintaining invariants. It’s true that, whatever measurement you took of walking, you wouldn’t find the pure constancy of maintaining a straight line of motion, but you would find a sequence of movements that repeat regularly and frequently.

And that’s invariant enough for me. It also makes sense in the context of the brain/body/environment picture. What frequently happens is that affordances push the body to move, and the movement produces different affordances, which push the body in a different way, and on and on until the first affordances pop up again and the cycle repeats. That’s how walking works.

Another example of maintaining an invariant might be Ms. Cricket. The invariant is that both eardrums vibrate at the same amplitude.That means the cricket can find her male by going straight forward. Since crickets don’t live on flat planes, but on a messy surface, I expect she’ll sometimes get shunted off track, then use her eardrums to get back to the “facing straight toward him” invariant.

≤ music ≥

Here, then, are your design principles:

* “Favor direct control links from perception to action.”
* “Prefer composite values over atomic values. Avoid primitive obsession.”
* “Discover or create affordances.”
* “Maintain invariants and rhythm.”
* “In addition to actions that achieve goals, also design actions that seek new affordances.”

Next episode, some examples of how they can be put to use.

Thank you for listening.