8. Knowing Our Weaknesses

To that end, I would like to offer an exceedingly simplistic model of the human mind so that we can begin to think about how our knowledge-gathering or belief-making process works and the various ways in which it can go wrong. The model I shall offer could be called the “Guesser-Checker-Storymaker” model.

The basic idea in this model is that there are three departments in our mind, and each of them has a different job to do. The job of the Guesser is to make all sorts of guesses about what is in our environment and even about what we ourselves are doing. It’s a wild and creative department, always brainstorming new ideas that come seemingly out of nowhere. The job of the Checker is to use our senses to try to determine if any of the guesses coming from the Guesser have any connection to what we can see, hear, smell, taste, or feel. The main job of the Checker is to filter out the wilder guesses coming from the Guesser and obtain a smaller set of guesses that seem possible given what we are experiencing. Finally, the job of the Storymaker is to take the plausible guesses and the information from our senses, and our memories as well, and then try to fold them all together into a coherent story about what is happening and what it all means. It provides the “finished copy” of what we think we know.

The three departments share information and affect one another. So, the Guesser might ask if we are seeing a duck on the pond. The Checker swivels the eyes toward the pond and examines more closely. On the basis of what the Checker sees, the Guesser makes more specific guesses such as whether we are seeing a Mallard or a Pintail—or maybe a goose or just a clump of sticks. Meanwhile, the Storymaker is rapidly putting together an account: “I see an object that might be a duck, and I have seen ducks here in the past, so it is not unlikely, but it’s not perfectly clear…” and this story, as it is being made, further affects the guesses that are being made and the ways in which the senses are being used. “Is this the same duck I saw here last Tuesday—the one with the funny feather sticking out of its head?…” Over time the three of them settle on a story—“Behold! I see a duck!”—before moving on to new guesses and new jobs to do.

Most of the time the system serves us quite well. Most of the time, we get things right. But there are also many ways in which the system can malfunction, leading to surprising results. Imagine what happens when the Checker does not have good access to the sense instruments—perhaps because the senses are “offline” (as they are when we are sleeping) or because their functioning has been affected by poor conditions, prismatic glasses, or hallucinogenic drugs. The Guesser keeps guessing away with all sorts of wild guesses about what’s going on, and the Checker does its best to confirm or disconfirm the guesses. But its functioning is impaired, so it is not very accurate. The Storymaker tries to keep up with the Guesser and the Checker, trying to weave together a coherent story from the information being provided. The result is a dream, or an LSD trip: objects keep turning into different objects (as the Checker keeps confirming wild guesses), and the story seems to make some sense at the time, but later on (when we are relaying the story to others), it will seem very strange and incoherent as the plot seems to keep changing. For example, at first I am looking at a duck on a pond, but then the duck is actually my brother, and we need to get to the airport because we are late for a flight. But the airport has no doors, and I am burdened with an enormous orange suitcase…

Or imagine a very intense and dangerous experience like getting into a fight or getting mugged. All three systems are working quickly and furiously, fueled by adrenaline, looking for immediate threats and escape routes and frantically coming up with the best idea of what to do next. In the heat of the moment, the Checker may not take the time to notice what the other people are wearing or whether they have a beard or whether they are tall or short, the Storymaker may not be keeping record of the precise order of events as they unfold, and the Guesser may be screaming out all kinds of wild ideas in the hope of producing something that will help. Later on, when we tell our friends or the police about what happened, it may be hard to remember exactly how things went down, what the other person looked like, what they were wearing, or why we said the things we said. Eyewitness testimony is notoriously unreliable for this reason: the memories are made under extremely adverse circumstances.

Or imagine a Storyteller that does not pay attention to the information that is being provided by the other two departments. The Storyteller is totally occupied in putting together a story, perhaps one of events from the past or imagined events of the future, and it is putting together all sorts of details and consequences and flourishes. The Guesser is guessing away, as usual, and the Checker is checking away, as usual, but none of the information is being taken up by the Storyteller since the Storyteller is totally absorbed in its own project. Then, a message suddenly comes through from the Checker: it appears we have been asked a question, and everyone around us is looking at us expectantly. What has been going on? We have no idea since the Storyteller has not been paying attention. We have been daydreaming. Things happening before our very eyes and ears have left no impression on us at all, though nothing was wrong with our Checker, and perhaps we were even nodding along with what other people were saying, though not keeping track of what was being said.

Generally, our cognitive system works very well. After all, it has served us well enough to allow us to survive this long in our evolutionary story. But its complicated nature means that it can malfunction from time to time. Moreover, it may be “engineered” to perform some tasks very well and others not very well at all. For example, perhaps our cognitive system works really well at processing information and maintaining social life in a small group of hunters and gatherers living on a savanna, but perhaps it does not do so well when asked to memorize passages and recite them backwards or to estimate probabilities.

Even apart from the various sorts of malfunctions described above, it seems our thinking system does tend to make recurring sorts of mistakes in ordinary circumstances. We can call these mistakes fallacies, or ways of processing information that do not reliably yield true beliefs. We will consider seven such fallacies and also use our simplistic model of the mind to diagnose how these patterns of mistakes come to be made. Then, we will see if we can draw an interesting general conclusion from these fallacies and some guiding advice for steering clear of them.