The Strong Demand for Charlatans

Chris Dillow says past performance is not indicative of future results:

The strong demand for charlatans, by Chris Dillow: In the improbable event of ever being invited to give a commencement address, my advice to graduates wanting a lucrative career would be: become a charlatan. There has always been a strong demand for witchdoctors, seers, quacks, pundits, mediums, tipsters and forecasters. A nice new paper by Nattavudh Powdthavee and Yohanes Riyanto shows how quickly such demand arises.

They got students in Thailand and Singapore to bet upon a series of five tosses of a fair coin. They were given five numbered envelopes, each of which contained a prediction for the numbered toss. Before the relevant toss, they could pay to see the prediction. After the toss, they could freely see the prediction.

The predictions were organized in such a way that after the first toss half the subjects saw an incorrect prediction and half a correct one, after the second toss a quarter saw two correct predictions, and so on. The set-up is similar to Derren Brown's The System, which gave people randomly-generated tips on horses, with a few people receiving a series of correct tips.

And here's the thing. Subjects who saw just two correct predictions were 15 percentage points more likely to buy a prediction for the third toss than subjects who got a right and wrong prediction in the earlier rounds. Subjects who saw four successive correct tips were 28 percentage points more likely to buy the prediction for the fifth round.

This tells us that even intelligent and numerate people are quick to misperceive randomness and to pay for an expertise that doesn't exist; the subjects included students of sciences, engineering and accounting. ...

It's easy to believe that this happens in real life. For example, the people who are thought to have predicted the financial crisis of 2008 are invested with an expertise which they might not really have. ... This paper ... suggests ... people are too quick to perceive skill and thus to pay for something that doesn't exist. The demand for forecasters and tipsters substantially exceeds the real ability such pundits actually have.

Why does this happen? Daniel Kahneman:

Kahneman: ...Psychologists distinguish between a "System 1" and a "System 2," which control our actions. System 1 represents what we may call intuition. It tirelessly provides us with quick impressions, intentions and feelings. System 2, on the other hand, represents reason, self-control and intelligence. ...

System 2 is the one who believes that it's making the decisions. But in reality, most of the time, System 1 is acting on its own, without your being aware of it. It's System 1 that decides whether you like a person, which thoughts or associations come to mind, and what you feel about something. All of this happens automatically. You can't help it, and yet you often base your decisions on it. ...

System 1 can never be switched off. You can't stop it from doing its thing. System 2, on the other hand, is lazy and only becomes active when necessary. Slow, deliberate thinking is hard work. It consumes chemical resources in the brain, and people usually don't like that. …

Spiegel: By studying human intuition, or System 1, you seem to have learned to distrust this intuition…

Kahneman: I wouldn't put it that way. Our intuition works very well for the most part. But it's interesting to examine where it fails. ... In the stock market, for example, the predictions of experts are practically worthless. Anyone who wants to invest money is better off choosing index funds... Year after year, they perform better than 80 percent of the investment funds managed by highly paid specialists. Nevertheless, intuitively, we want to invest our money with somebody who appears to understand, even though the statistical evidence is plain that they are very unlikely to do so. ... The experts are even worse because they're expensive.

Spiegel: So it's all about selling snake oil?

Kahneman: It's more complicated because the person who sells snake oil knows that there is no magic, whereas many people on Wall Street seem to believe that they understand. That's the illusion of validity …

Spiegel: Do we generally put too much faith in experts?

Kahneman: I'm not claiming that the predictions of experts are fundamentally worthless. … Take doctors. They're often excellent when it comes to short-term predictions. But they're often quite poor in predicting how a patient will be doing in five or 10 years. And they don't know the difference. That's the key.

Spiegel: How can you tell whether a prediction is any good?

Kahneman: In the first place, be suspicious if a prediction is presented with great confidence. That says nothing about its accuracy. You should ask whether the environment is sufficiently regular and predictable, and whether the individual has had enough experience to learn this environment. ...

One implication of this is that we use rules of thumb rather than rational, deliberative thought (i.e. rational expectations) for many of our decisions.