Oddly Influenced

Brian Marick 0:04
Welcome to Oddly Influenced, a podcast about how people have applied ideas from outside software to software. Episode 16: Glenn Vanderburg on engineering, software, and software engineering.

Brian Marick 0:22
The most important thing for you to know about Glenn Vanderburg is that around 2010, I was visiting a pretty high performing rent-a-programmer company. Basically, I invited myself to visit them for a week, pair with their programmers, and learn how they did things. While they were cordial, none of the Ruby programmers seemed to want to pair with me. Glenn took pity on me and invited me to help out with a project that was in a fairly new programming language called Clojure. The result was that Clojure became my default programming language for something over eight years and was essential to my most successful book.

Brian Marick 0:58
Those who aren't as fascinated by my life story as I am probably care more that he is one of the two most thoughtful people I know when it comes to the relationship between software and conventional engineering, the other being Hillel Wayne. He is also the person I described in episode three as "generally less excitable than me". As such, he pushed back on my comparison of software engineering to Monte Carlo methods in Episode 12. And so the only sensible thing was to invite him onto the podcast.

Brian Marick 1:30
This is not an argument. Remember, the idea behind the podcast is not to attempt to shoot down ideas, but rather to ask, "If that idea were true, what might that inspire us to try?" The point of the episode is to work with Glenn's ideas about the engineering in software engineering.

Brian Marick 1:47
So Glenn, if I become argumentative, as is my nature, feel free to shut me down. That said, Glenn, what about you have I left out?

Glenn Vanderburg 1:59
Well, I can be argumentative too, although I do try to be thoughtful, as you've said.

Glenn Vanderburg 2:06
I remember that week that you were visiting us pretty well, and that Clojure project we were working on. You and I had met in years prior to that at some OOPSLA conferences, I think. And in particular, you helped to lead the first OOPSLA essays track at, I think, OOPSLA 2005. And I submitted something, and it was the only essay accepted for that initial track. And I'm still really proud of that paper, [it] was actually some of the very first work that I did to seriously start to understand what kind of engineering discipline and rigor was really appropriate for our field. And that grew into some of the later work that you're referring to when you talk about me thinking about the relationship between Software Development and Engineering.

Brian Marick 3:06
I will try to put that paper up on the show notes.

Brian Marick 3:10
Okay, that said, what do we think about engineering (we being software people) that isn't true.

Unknown Speaker 3:20
Mostly, it's the same thing that everybody thinks about engineering that isn't true.

Glenn Vanderburg 3:26
Just to back up a little bit. You mentioned Hillel also. And Hillel and I came at this problem from two different directions. I came at it, as is my wont, from reading books. Hillel heard my talk on software engineering, and initially thought I was full of it, and decided to prove me wrong.

Glenn Vanderburg 3:48
And he went about it by going to interview people who had actual real experience in both traditional engineering fields and in software development. So I went to books, and he went to people and ended up having very similar conclusions, which I find immensely gratifying. [What] really makes me happy [is] that initially he was setting out to prove how wrong I was. I don't say that in a triumphal way. But I think it gives a lot of credibility to what he produced.

Glenn Vanderburg 4:22
So anyway, one of the things that I found as I started reading about traditional engineering and how it's actually practiced, and what he found as he interviewed people with experience in both domains, is that the very widespread stereotype about how engineers work and think, is dead wrong. There's almost no relationship to how engineering is actually practiced.

Glenn Vanderburg 4:50
The stereotype is what Herbert Simon called the rational model. You know: you analyze the problem, think about the requirements of your solution, and sort of in a step-by-step way work from that towards synthesizing a solution that meets those requirements. And then you're done.

Brian Marick 5:12
Is this /The Sciences of the Artificial?/

Unknown Speaker 5:15
Yeah, right. And as Frederick Brooks points out, in his book /The Design of Design/, that's just a very natural model for people to conceive of. It has independently been invented a number of times and passionately argued for as the right way to build systems, or design things, or whatever.

Glenn Vanderburg 5:39
But it's not how anything of any complexity gets built in the real world. The way things actually get built is to start with something small. You know, Gall's law: you start with a small system that works and iterate until it's a big system that works.

Glenn Vanderburg 5:56
Sometimes you have to do that on paper. But still, it's an iterative, incremental experimental process – an empirical process – rather than a defined step-by-step linear process, like the rational model.

Brian Marick 6:09
So paper... and perhaps to some extent, science and mathematics is what you fall back to, when you can't try it and see, cheaply.

Unknown Speaker 6:21
Sure, yes, that's right. The stereotype of engineering that most people believe comes from the most prominent engineering artifacts that we see all around us all the time. Not necessarily the most common engineering artifacts, but the most prominent ones: bridges and large structures. And in that kind of world, the cost of labor and materials to build a thing dominates so strongly that actual prototyping and physical testing is prohibitively expensive.

Glenn Vanderburg 6:55
And so it becomes more economical to do a lot of inspections and reviews and careful validation on paper in order to save actually building the thing and testing it.

Glenn Vanderburg 7:08
And that led a lot of people in the early history of software engineering research to conclude "that's what engineers do." And "that's the right way to do engineering." But in actual fact, in other traditional engineering disciplines – where either scale is not as important, so you can build small scale prototypes and trust that they are actually pretty representative of the full scale thing – or you have ways to synthesizing a prototype fairly cheaply, like, for example, circuit boards, things like that – engineering is actually much more iterative.

Glenn Vanderburg 7:41
There's much less reliance on any kind of inspection and validation by other engineers to figure out that you've got it right, because the goal is to reach robust solutions in economical ways, not necessarily [to] follow some abstract idea of what the right way to do it is.

Glenn Vanderburg 8:01
It's very much in line with Kent Beck's old driving metaphor for how a software project is built. You don't get somewhere by pointing your car in the right direction and making sure it's lined up right and then just pushing the gas. You watch and adapt and correct and steer along the way.

Brian Marick 8:19
It sounds like you're saying we do the kind of engineering that real engineers do. One thing that comes to mind is... I don't know that we have the same approach to satisfying constraints, where cost is a big constraint. We we don't seem to take costs into account, we kind of wing it.

Brian Marick 8:43
If what we're doing is already a kind of engineering, what is the advantage of paying attention to engineering?

Glenn Vanderburg 8:52
This gets right to the heart of some of the discussions we were having on Twitter that led to this interview. You're right, we don't have the same approach to costs as traditional engineering disciplines do. Although I will point out that [the track record] of engineering projects adhering to budgets is much more spotty than the popular wisdom would lead you to believe.

Glenn Vanderburg 9:15
But we don't have the same approach. Why is that? Well, one reason, I think, is because even though we have a lot in common with other engineering disciplines, we have our differences as well. And there are some fundamental differences that we haven't understood very well and don't yet have the right conceptual tools for dealing with.

Glenn Vanderburg 9:39
The other reason is that we were kind of misled. And this is wh ere it's going to sound like I'm playing right into your argument, but I'm not. We've been misled by a mistaken pursuit of "let's try to learn how to be engineers." You said, we just kind of wing it with regard to costs and don't even consider it sometimes." Well, growing out of the stereotype of how engineering works... That was actually very explicit in a lot of the software engineering circles in the 80s and 90s. Mayb e not in the actual work from serious academics, but I don't know how many discussions I was involved in where people said, more or less explicitly, this way of doing it – this rational model-based, linear, holistic way of viewing the design is the right way. And cost shouldn't be an object. You're aiming at some platonic ideal of the engineering process. And the goal is to make it right and solid and robust. And this is the right way to do it. So you shouldn't worry about cost.

Glenn Vanderburg 10:48
So yes, part of the problem is we mistakenly fixated on trying to learn how to do it like engineers do, but we were chasing a false target. Engineering processes have evolved to take costs into account. One part of the stereotype of engineering is that engineers use math a nd formalisms and mathematical models, because that proves that the designs are correct. But that's not why they do that at all.

Glenn Vanderburg 11:19
They do that because it saves money to use mathematical approximations of real materials. And mathematical models are always approximations of real materials. It saves money to do that, rather than going and doing a lot of physical testing.

Glenn Vanderburg 11:37
So the misconceptions about what engineering really is and how it works runs so deep in culture, in general, and especially in software, that we really got misled by those misconceptions, and pursued a false goal.

Glenn Vanderburg 11:54
The value I think, is that we should learn from real engineers, and how engineering is really done. And start to figure out how that applies and can help us manage costs in the software world.

Glenn Vanderburg 12:10
And a very valuable byproduct of us doing that is that it could help other engineering disciplines understand each other better, and learn from each other and from us. And this is an outcome of Hillel' s work that I wasn't expecting at all. Not only do we not understand other kinds of engineering, they don't understand each other. And there's almost no cross pollination between engineering disciplines. And that seems like a huge wasted opportunity.

Brian Marick 12:43
Yeah, I'm not sure computer programmers are the people to fix that, being how famous we are for telling other people what they're doing wrong, and how they ought to be doing their job.

Brian Marick 12:58
At this point, I sort of derail the conversation into some skippable discussion about reducing the cost of machinery (when it comes to web apps) and the cost of programmers in general. I do want to highlight something that Glenn said.

Brian Marick 13:13
My guess is, knowing you, that your focus on engineers doing their job better... their job is writing code that can change as the business needs change, because cost of change has historically been the cost people pay attention to. S o in a way, focusing on reducing cost of change is a focus on cost. The dominant kind of cost.

Glenn Vanderburg 13:44
That's right. And I just don't think there's any shortcuts to that. All the efforts to find ways to need fewer programmers, other than building better software, seems like wishful thinking and shortcuts.

Brian Marick 13:57
At this point, I talked about constraints and about how we as programmers don't seem to worry about constraints as much as other engineers do. I refer to Herbert Simon's idea of "satisficing" from his book /The Sciences of the Artificial/. "Satisficing" is finding a solution that's *good enough* along several different dimensions. Thinking of myself in the past, it seemed to me that I didn't do much conscious satisficing. Glenn pushed back.

Glenn Vanderburg 14:26
You know, I think there's actually a lot of that. A lot more than you might have been aware of. The constraints are maybe different than what we would think about in a physical engineering field. But they're there.

Glenn Vanderburg 14:36
I talked [in a deleted section] about trying to break programmers of the habit of micro-optimizing things, every little loop and every cycle, when that really wasn't important anymore. And those kinds of balancing of constraints are encoded in little bits of lore like Knuth's thing about "don't bother optimizing anything but the inner loop of your program" or Rob Pike's thing about "fancy algorithms are slow when N is small and N is usually small."

Glenn Vanderburg 15:03
We satisfy multiple constraints all the time, when we think about choosing architecture for background processing for web applications. There's so many ways you can optimize that. But for most purposes, you pick something that's fast enough and don't sweat it.

Glenn Vanderburg 15:19
Batch processing on a web application is an example of balancing those constraints. In a web application, you want to turn that request around as fast as you can. But there are things that might need to be done in response to it that take longer. So you build a system to offload that kind of work, and make sure you have transactional protections around it so that it doesn't get lost if it was important. And then you can report back to the web browser "Done!", even if you know it won't appear in the all the places that [it] needs to go for a few minutes.

Glenn Vanderburg 15:56
Eventual consistency is a way of balancing multiple constraints in a large system.

Brian Marick 16:03
That makes me think... One of the things you hear is that it costs fantastically more to put in a kilometer of subway in New York than it does in Paris. The most convincing explanation I've heard about that is that in Europe, you spend your career working on doing maintenance on a particular system. Whereas in the US, it's a one-shot [project]. You don't continue working with the same [system] over time.

Brian Marick 16:45
And a lot of the things that you're talking about seem to me to be the sort of tacit or water cooler style knowledge that gets passed along in France, say, but doesn't get passed along in the US so much. So that might be something that you would want to focus on: keeping people on [projects] for long enough.

Glenn Vanderburg 17:10
Yes. that is a real problem in the software world. And [while] I can't speak to other industries, it is a real problem in some parts of the software world where people don't stay on a project long enough to feel the cost of their mistakes – or their successes for that matter.

Glenn Vanderburg 17:33
I guess the classic example of that is the consultant. Different consulting firms have different ways of operating. And so depending on which consulting firm you work for, you might be the one who builds the thing to start with, and then moves on to another project and never gets to feel the cost of [what you did]. Or you might be in a "project rescue" kind of practice where you're the one who comes in and cleans up everybody else's mistakes and fixes a project that's in trouble. And there, you learn a lot more about the bad things you can do.

Glenn Vanderburg 18:06
But very few people stick around the same project for long enough to really see both sides of that and know "Hey, I'm to blame for all of it. And I need to learn from that."

Brian Marick 18:20
Yeah, Keith Braithwaite tells a story where he and maybe a consultant invented some very clever, quote unquote, elegant mathematically-based solution to some particular problem, only to discover that it was too elegant for anybody coming after them to actually understand it. So, an impediment to change. I hope I've got the story right. I'll try to ask him, but they eventually ripped it out and did the Simple, Stupid thing instead.

Glenn Vanderburg 18:58
Yeah, that's an easy thing to imagine, an easy thing to picture happening. I've seen a similar thing happen in the history of Ruby on Rails. I am still a fan of Ruby on Rails for a lot of kinds of things. But I started in it super early, when 1.0 still hadn't been released. And a lot of Rails is based on particular ideas about architecture and a lot of conventions. Some of those conventions are pretty explicit and others not so much. But when you're around and involved [when] a community of practice is growing around something like that, you hear those discussions about what should the conventions be and what are the right ways to do things. You internalize those, and you play [along] with those assumptions. And if you play [along] with those assumptions, Rails is is still a wonderful way to build web applications.

Glenn Vanderburg 20:03
In those early days, Chad Fowler gave a talk called "Active Record: Easy Living on the Golden Path." And I think about that phrase a lot when I'm using Rails. If you work with it, and work with those assumptions and the way it's intended to be, it's really wonderful.

Glenn Vanderburg 20:20
Well, then, a few years later, you start to see people coming to Rails who weren't a part of all those discussions – or any of those discussions. And they pick up a book. And they skip the chapter about the philosophical underpinnings and the the assumptions and the right ways to do things and just go look up the API docs and try to use it as a library with their architecture in mind. And I think on many Rails projects, that was the predominant mode from from 2012 or 2013 onward. And it just doesn't work that way. In contrast to Braithwaite example, I wouldn't describe rails as fantastically elegant or sophisticated or anything in that way. But it has, in addition to the explicit parts of its design, an underlying philosophy and attitude about how it should all work. And conveying that to newcomers is a real challenge, not least because it's hard to get them to understand why it's important.

Brian Marick 21:26
I always bring up my wife, professor of veterinary medicine. Some people think of medicine as being very science-based. But an awful lot of medicine is: you do things the way your teacher – in an apprenticeship sort of way – taught you. And that's both risky, because you get stuck in the old way, but also beneficial, because there's just so much that you have to do by habit. And if you tried to reason your way to those habits, it'd be inefficient at the very best and quite possibly impossible.

Glenn Vanderburg 22:04
And that's one part of the rationale, I guess, for the Dreyfus model of skills acquisition. Part of it is, yeah, shut up and learn these habits. You know, then you might be able to recognize times when you can break out of those habits, but for the most part, just do it like we've told you to do it.

Brian Marick 22:26
So as we're getting toward the end here, I'll bring out my tough question. There's a notion in American law, US law, called an attractive nuisance, which is if you have playground equipment or a swimming pool on your property, you are expected to understand that it will attract small children who can then kill or injure themselves. And you must take reasonable steps to prevent those children [from doing that], such as having a fence that's at least four feet tall around a pool, with a locked gate.

Brian Marick 23:11
I've been also reading about Lenin recently, and Lenin had that rationalist attitude. "I have solid science behind me," he thought, "and...

Glenn Vanderburg 23:26
I can analyze the situation and apply that science.

Brian Marick 23:29
"... and I will design a system." Both he and an architect / city planner called Le Corbusier, they both thought of themselves as designing machines. Le Corbusier famously described a home as "a machine for living." (Not an attitude that makes for a very livable home.)

Brian Marick 23:57
So my question is: since 1968 or '69, we've had this notion of software engineering that you refer to as "academic software engineering,"...

Glenn Vanderburg 24:08
Wish I had a better term for that.

Brian Marick 24:10
... in which people treat engineering as this rational process instead of the way you describe it. We've also got Le Corbusier and Lenin who are both doing the same thing. They have this idea of engineering, and they want to apply it to humanity.

Brian Marick 24:32
Maybe engineering is an attractive nuisance. Wouldn't we be better off just focused on what we need to do to do things better? And not let ourselves wander so close to the edge of that swimming pool when the past 60 years has taught us we can't swim?

Glenn Vanderburg 24:54
Yeah, I'm gonna say you're very close to something there but it's not engineering that's the attractive nuisance. It's the rational model. That's the attractive nuisance. The rational model was what we were chasing for decades in the software world, to our detriment. The rational model was what led Lenin and Le Corbusier down the wrong path.

Glenn Vanderburg 25:23
I think we see that all over the place. Again, calling back to Brooks's book, /The Design of Design/, you know, he talks about how attractive and even seductive the rational model is. And he tells the story of when he was in engineering school, engineering had been seen as a design discipline. But in the period after the World Wars, and especially the triumph of science in World War Two, there was a push back on that. And people wanted to see engineering as simply a branch of applied science. And so Nobel Prize winning physicist John Van Vleck was made the Dean of the School of Engineering where Brooks went to school and tried to see it in a very rational model kind of way. It seems so nice! And you see it popping up everywhere.

Glenn Vanderburg 26:16
And people just assume that's how you can build things. But it's not how you build things of any complexity, especially when – to come back to the Lenin and Le Corbusier examples – when people are part of those systems... In engineering processes, people are part of the system. And one of the nice things about the Agile crowd that you and I both know so well, [is that] from those old days, there was a lot of explicit acknowledgement [that] people are parts or components of these systems, both the system of designing and [the system of] building the software. And, in most cases, the milieu in which the software will run and operate. And you have to deal with the variation and vagaries and undependability of people.

Glenn Vanderburg 27:12
And so yeah, I think the rational model is an attractive nuisance. And anytime we see somebody approaching a complex problem-solving domain with that as their ground, we should try to disabuse them of that.

Brian Marick 27:31
So from a statistical process control or quality engineering perspective, people are components of the system with really high variance.

Glenn Vanderburg 27:47
Not just *among*, but also *within* individuals.

Brian Marick 27:50
And you can have two reactions to that. One is to squeeze the variance out of them. And the other is to design a system that is tolerant of high variation.

Brian Marick 28:08
Interesting...

Brian Marick 28:10
Anything else that you'd like to get off your chest?

Glenn Vanderburg 28:14
Well, our discussion has reminded me of a couple of things, and I will just send them along to you to include in the show notes for further reading. Okay. One is David Parnas. His paper, "A rational..."

Brian Marick 28:27
[crosstalk] "rational design process: how and why to fake it"

Glenn Vanderburg 28:30
"...design process: how and why to fake it? Yep, that's right.

Brian Marick 28:33
I love that paper.

Glenn Vanderburg 28:34
I do too. And I misunderstood it for a number of years before I finally broke through to realize what he was talking about.

Glenn Vanderburg 28:43
And also a talk by my friend, Neil Ford, from over 10 years ago, and I'll have to try to track it down. [Note: it's no longer available because of DCMA takedown.] But I think it's called "Constraints are Liberating." And it's an analysis of how, when properly approached, constraints in our systems can really unlock a lot of room for creativity.

Brian Marick 29:13
Thank you for talking with us, with me. And thanks to those who are listening to this. And now I will push the little red button that stops things.

Transcribed by https://otter.ai