Dogmatism makes you a worse predictor

TETLOCK: That experts thought they knew more than they knew.That there was a systematic gap between subjective probabilities that experts were assigning to possible futures and the objective likelihoods of those futures materializing.

DUBNER: Let me translate that for you. The experts were pretty awful. And you think: awful compared to what? Did they beat a monkey with a dartboard?

TETLOCK: Oh, the monkey with a dartboard comparison, that comes back to haunt me all the time. But with respect to how they did relative to, say, a baseline group of Berkeley undergraduates making predictions, they did somewhat better than that. Did they do better than an extrapolation algorithm? No, they did not. They did for the most part a little bit worse than that. How did they do relative to purely random guessing strategy? Well, they did a little bit better than that, but not as much as you might hope.

DUBNER: That “extrapolation algorithm” that Tetlock mentioned? That’s simply a computer programmed to predict “no change in current situation.” So it turned out these smart, experienced, confident experts predicted the political future about as well, if not slightly worse, than the average daily reader of The New York Times.

TETLOCK: I think the most important takeaway would be that the experts are, they think they know more than they do. They were systematically overconfident. Some experts were really massively overconfident. And we are able to identify those experts based on some of their characteristics of their belief system and their cognitive style, their thinking style.

DUBNER: OK. So now we’re getting into the nitty-gritty of what makes people predict well or predict poorly. What are the characteristics then of a poor predictor?

TETLOCK: Dogmatism.

DUBNER: It can be summed up that easily?

TETLOCK: I think so. I think an unwillingness to change one’s mind in a reasonably timely way in response to new evidence. A tendency, when asked to explain one’s predictions, to generate only reasons that favor your preferred prediction and not to generate reasons opposed to it.[1]