Persuasion with statistics: Mark Thoma's point that apparently strong econometric results are often the product of specification mining prompts Lars Syll to remind us that eminent economists have long been wary of what econometrics can achieve.
I doubt if many people have ever thought "Crikey, the t stats are high here. That means I must abandon my long-held beliefs about an important matter." More likely, the reaction is to recall Dave Giles' commandments nine and 10. (Apparently?) impressive econometric findings might be good enough to get you published. But there's a big difference between being published and being read, let alone being persuasive.
This poses a question: how, then, do statistics persuade people to change their general beliefs (as distinct from beliefs about single facts)?
Let me take an example of an issue where I've done just this. I used to believe in the efficient market hypothesis. ...[explains why that changed]...
He concludes with:
The above is not a story about statistical significance (pdf). Single studies are rarely persuasive. Instead, the process of persuading people to change their mind requires diversity - a diversity of data sets, and a diversity of theories. Am I wrong? Feel free to offer counter-examples.