« Is Inflation a Threat to China's Stability? | Main | The Other Half »

Wednesday, October 17, 2007

Comment on "Tests of Statistical Significance in Economics"

This comment scrolled down the sidebar fairly quickly, so many of you may have missed Deirdre McCloskey's response to your comments to the post "Tests of Statistical Significance in Economics." As you'll recall, the post highlighted comments by Andrew Gelman on a debate between McCloskey/Zilliak and Hoover/Siegler over the use of classical statistical methodology in economics. Here's the comment:


I can't keep all your monickers straight, so let me "reply" in a general way. Ziliak and I are to get the page proofs of a book about all this in a couple of week, out in I think December: The Cult of Statistical Significance (University of Michiagn Press).

(1.) There we tell how R. A. Fisher resisted calls to make his procedures relevant to scientific hypotheses (by Neyman and Pearson, for example, or by the inventor of the t test, Gossett; or by later exponents of statistics with loss functions such as Wald). He stuck with 5% and tests of one kind of scepticism, supposing (as is very frequently not the case) that the only source of scientific error is one of inference from a sample to a population.

(2.) But the basic point, which some of you-all are not quite seeing, is that good fit is not the same thing as importance. In fact, usually it has nothing to do with importance. Yet SPSS-sciences have thoroughly confused the two. As one of you said, irrelevant reporting of standard errors (when for example the issue is not one of sampling anyway!) is used as a referee-enforced entry fee to journals, and makes no contribution to the scientific oomph of the argument.

(3.) It's rather important to understand (see the book or any of the articles we wrote) that we are making a very old and elementary point. So it just won't do to say, "Wow, this is crazy. Who are these people anyway?" We're merely reporting on scores of the leading minds on statistics from the beginning who make the same point. See for example Bill Kruskal's devastating article on "significance" in The old Encyclopedia of Statistics. It's not our point. It's Savage's point, or Freedman's point (Freedman, Pisani, and Purvis).

In other words, some fields---not physics and engineering and chemistry---have made a dreadful mistake, which vitiates most of their statistical work.


Deirdre McCloskey

    Posted by on Wednesday, October 17, 2007 at 02:07 PM in Economics, Methodology | Permalink  TrackBack (0)  Comments (19)


    TrackBack URL for this entry:

    Listed below are links to weblogs that reference Comment on "Tests of Statistical Significance in Economics":


    Feed You can follow this conversation by subscribing to the comment feed for this post.