This is from Kahneman’s Thinking, Fast and Slow:
As you consider the next question, please assume that Steve was selected at random from a representative sample:
An individual has been described by a neighbor as follows: “Steve is very shy and withdrawn, invariably helpful but with little interest in people or in the world of reality. A meek and tidy soul, he has a need for structure, and a passion for detail.” Is Steve more likely to be a librarian or a farmer?
Apparently, most people, when presented with this question, answer librarian. Kahneman concludes that this is because of a resemblance heuristic (a common stereotype of librarians is being shy and orderly). He argues that the correct answer is farmer, because there are a lot more farmers than librarians.
While his conclusion might be correct, I argue that there are several other alternative explanations.
Imprecise questions
What does it mean that Steve was selected at random from a representative sample? Does he mean that Steve was randomly selected from the population of adults in the US? One could interpret his question as: “is it more likely that a person, selected at random from the population of shy, withdrawn, helpful, meek people who have a passion for details, but little interest in the world of reality, works as a librarian or a farmer?”
If that is the correct interpretation, why does he make it personal by saying that a neighbor was describing Steve? If a neighbor is choosing to gossip about someone, that might offer clues into what the person’s profession is. Are librarians or farmers more likely to be gossiped about?
If a neighbor is talking about someone being shy and needing structure, and that is the stereotype of a librarian, then it very well might be more likely that Steve really is a librarian (precisely because people notice stereotypical traits in people and like to talk about it). So people who said librarian very well might be correct. Further, it could be that the neighbor is wrong about Steve. That is the neighbor’s opinion about him. Maybe the neighbor assumes those things about him because he is a librarian. Maybe it is the neighbor who made an error.
One could interpret his question as: “is it more likely that a person who is described by a neighbor as shy, withdrawn, helpful, meek, with a passion for details, but little interest in the world of reality, works as a librarian or a farmer?” What is the correct answer to that question? Does anyone know?
Social questions
Most of the time, when we are asked a question, it is in a social setting. In those settings, most people are not very precise and we have to interpret what they mean. If you always interpret questions literally, you will often make errors. In a social setting, if someone asked me the question about the farmer vs librarian, I would probably interpret their question as follows: “if you compare the percentage of librarians who are shy, withdrawn, helpful, meek, with a passion for details, but little interest in the world of reality, with the percentage of farmers who have those traits, which one do you think is higher?” I think this is what most people would mean if they asked a question like the one Kahneman posed.
Kahneman made his question social and imprecise by describing a gossiping neighbor. People are used to the imprecision that comes with being asked questions in social settings. They interpret these questions and answer the question that they think the person is asking. Because of that, it is unclear whether librarian or farmer is truly the correct answer (it depends on what question the reader was intending to answer).
In research settings, questions are typically more precise. Researchers often spend a large amount of time editing survey questions to make them as precise as possible. The problem here is that it’s a research question that is as sloppy as a question in a social setting. So what can one conclude from it?
Read Full Post »
N=1 