Trains, Fruits, and Bombs: A Crash Course in Motivated Reasoning

On Trains

My sister is interested in genealogy, and several years ago unearthed a gruesome family story: on June 11th, 1880, my great-great-great-grandfather Anthony was caught between two railroad cars and crushed. The newspaper called it a "horrible death", and added that he left behind a wife and five kids "in destitute circumstances".

I don't remember my precise reaction to reading that quote for the first time, but I'm pretty sure it was something like, "Wow! Working for the railroad used to be a dangerous job. I'm glad we figured out how to make it safer."

More recently, I read Matt Stoller's Goliath, an excellent book about the history of monopolies and anti-monopoly activism in America. It contains a quote from WL Park, the general superintendent of the Union Pacific Railroad back in 1912. He spoke of the massive death toll in the railroad industry in disconcertingly rapturous terms: "One human being is killed every hour and one injured every ten minutes. There is a steady grinding and crunching of human flesh and bone under the juggernaut of modern car wheels. It is the price we pay for progress[.]"

I do remember my precise reaction to this quote. It was "fuck you, dude, you're not the one paying the price of progress".

But, you know, wealthy people say lots of awful things; Privileged Person Dismisses Struggles of Those Less Privileged—News at 11!

What brought everything together for me was a third quote, this time from Nancy Leveson's Engineering a Safer World:

In the nineteenth century, coupling accidents on railroads were one of the main causes of injury and death to railroad workers. In the seven years between 1888 and 1894, 16,000 railroad workers were killed in coupling accidents and 170,000 were crippled. Managers claimed that such accidents were due only to worker error and negligence, and therefore nothing could be done aside from telling workers to be more careful. The government finally stepped in and required that automatic couplers be installed. As a result, fatalities dropped sharply. According to the June 1896 (three years after Congress acted on the problem) issue of Scientific American:

Few battles in history show so ghastly a fatality. A large percentage of these deaths were caused by the use of imperfect equipment by the railroad companies; twenty years ago it was practically demonstrated that cars could be automatically coupled, and that it was no longer necessary for a railroad employee to imperil his life by stepping between two cars about to be connected. In response to appeals from all over, the U.S. Congress passed the Safety Appliance Act in March 1893. It has or will cost the railroads $50,000,000 to fully comply with the provisions of the law. Such progress has already been made that the death rate has dropped by 35 per cent.

My great-great-great-grandfather didn't die because we lacked the knowledge to make railroad work safer. He died because some robber baron decided making him safer would cost too much money—and Congress went along with it (for a while). Superintendent Park wasn't just a dismissive rich dude. He was using the language of inevitability to obscure an active choice his company was making—the choice that other people, people like my great-great-great-grandfather, would be the ones to pay the price.

On Fruits

The language of inevitability is everywhere, and can be used to justify anything. There's a great interview with Ada Palmer where she talks about how, in pre-revolutionary France, the distinctions between classes were seen as natural and inevitable. One of the pieces of evidence marshaled was, of all things, fruit:

"Food had been one of the proofs of nobility, because in pre-revolutionary France if you are a servant, and your master and you trade lunch, you both get sick. We understand that this is because your stomach only produces the enzymes necessary to dissolve the things that you're used to eating, and so just as a vegetarian who suddenly has a pork chop will be sick because their body isn't used to producing the material for it [...] similarly it was true that if a peasant and his master traded lunches they would both get sick. So they had this idea of noble people eat noble fruits like oranges and apples and things that grow on the tops of trees, and and lower class people are meant to eat onions and garlic and things that grow in the ground.

[...] When cantaloupe and other new world plants were discovered [...] [that] contradicts very directly everything that Aristotle and his apprentice Theophrastus said about fruits and how the noblest fruits which are round and golden and similar to the sun grow on the tops of trees, and only base, bad, 'lower class' fruits grow on vines that lie abjectly on their bellies on the ground like a defeated prisoner of war. And so when they find this round golden sun-like thing that grows on limp vines on the ground it's one very important crunch in overthrowing Theophrastus and Aristotle [...] when the hierarchy of plants is overthrown the hierarchy of food is overthrown which [raises] the question: is there a biological difference between the servant and the master, or is it just arbitrary? If we were that wrong about food are we also that wrong about class?

Like the previous example, this can be summarized as 'powerful people make inaccurate claims about how things have to be to justify keeping things the way they are'. But I think there's something more complicated going on here. So let's look at a third example.

On Bombs

In 1933, Ernest Rutherford dismissed the idea of nuclear weapons as "moonshine". If you're thinking that maybe Rutherford wasn't qualified to give an opinion, try googling "father of nuclear physics". Rutherford was one of many luminaries who thought this way. Albert Einstein once described the pursuit of nuclear fission, a crucial step towards making nuclear weapons, as “shooting birds in the dark...in a country where there are few birds”.

By 1939, only six years later, Einstein was urging Franklin Roosevelt to begin the Manhattan Project. Physicists were now sure that someone was going to build the bomb and they wanted their side to do it first. What was recently an impossibility had become an inevitability. As Nazi physicist Carl Friedrich von Weizsäcker put it: “The atom bomb exists. It exists in the minds of some men. According to the historically known logic of armaments and power systems, it will soon make its physical appearance."

Einstein and Rutherford weren't trying to make a profit when they made their earlier statements downplaying the possibility of nuclear weapons. And they weren't lying about their beliefs. I mean, sure, can we ever truly know what's in another person's heart? Yet I'm confident that there was no massive conspiracy among the world's physicists to hide the possibility of nuclear weapons. These dorks put on plays at their yearly meetings! Winston Churchill suspected Neils Bohr of spying for the Russians because he thought no one could possibly be that idealistic and naive!

It seems clear that Einstein, Rutherford, Bohr, etc sincerely believed that nuclear weapons were a fantasy. So why were these literal geniuses so utterly wrong?

I think they really, really wanted nuclear weapons to be a fantasy. Then they could just focus on the science and not have to worry about war and death and politics. Lise Meitner led the discovery of nuclear fission and was often called, to her dismay, "the mother of the bomb". She later wrote that before the bomb, and before the rise of Hitler, "one could love one’s work and not always be tormented by the fear of the ghastly and malevolent things that people might do with beautiful scientific findings.”

For many people, including scientists like Meitner and Einstein, the ability to love their work without reservation is more valuable than profit to a robber baron.

On Motivated Reasoning

In other words, this is motivated reasoning. And in all three cases, the power and reputation of the people making these biased judgments helped create a general, common sense understanding that there was no way to prevent railroad accidents, no chance of nuclear weapons ever being made, and also that oranges are somehow morally superior to cantaloupes.

My takeaway is not that motivated reasoning exists—I already knew that—but that the language of inevitability/impossibility is a red flag pointing to its presence. So when I hear people use that language, or when I catch myself using it, I'm going to try to ask:

• What evidence is there that this is actually inevitable/impossible?
• What does the person making the assumption gain from believing that?
• What might they be trying to avoid, that they'd feel compelled to do if the assumption wasn't true?

And here are a few assumptions I've personally held, that I'm hoping to test out this process with:

• the creation of general AI in the next 10-20 years is impossible
• the electoral college will never be reformed or replaced

The goal of this exercise is not necessarily to shift my beliefs but to practice the meta-work of noticing my own biases and approaching assessments in a different way when I know I'm biased.