Comments on 'Any P-Value Distinguishable from Zero is Insufficiently Informative'TypePad2015-05-20T16:29:51ZMark Thomahttps://economistsview.typepad.com/economistsview/tag:typepad.com,2003:https://economistsview.typepad.com/economistsview/2015/05/any-p-value-distinguishable-from-zero-is-insufficiently-informative/comments/atom.xml/Matt Young commented on ''Any P-Value Distinguishable from Zero is Insufficiently Informative''tag:typepad.com,2003:6a00d83451b33869e201bb08329cdd970d2015-05-22T08:00:55Z2015-05-22T08:00:55ZMatt YoungI do it different. I have a large collection of data, and a machine that generates a test statistic. So,...<p>I do it different.<br />
I have a large collection of data, and a machine that generates a test statistic. So, I ask my data to walk through the machine, backwards, and identify the null hypothesis with increasing accuracy. When the data has done its task, then I can fill in the number line and that number line is the data's most likely hypothesis. Got this idea from a movie, it works.</p>
<p>The data organizes itself into a sequential sets. The first set goes through backwards and delivers some b^(-1), b > 1, as its guess of the null hypothesis. The second group goes through and generates b^(-2), and so forth. If my large group had N elements, then the number of notches on my number line between 0 and 1 should be close to log base b of N. </p>Richard H. Serlin commented on ''Any P-Value Distinguishable from Zero is Insufficiently Informative''tag:typepad.com,2003:6a00d83451b33869e201bb083205d7970d2015-05-21T06:31:50Z2015-05-21T06:31:50ZRichard H. Serlinhttp://richardhserlin.blogspot.com/Honestly, I think you just say, that a p-value is the probability that the true parameter is EXACTLY zero, given...<p>Honestly, I think you just say, that a p-value is the probability that the true parameter is EXACTLY zero, given the data and the assumptions of the test -- data gathered 100% correctly and accurately, model specified 100% correctly, and so on.</p>
<p>So, if the true value is $0.00000000001, then with enough data you could get a really small p-value that it's EXACTLY 0. Because, in fact it is not exactly zero, it's just economically insignificantly different from zero. To see the economic significance just look at the confidence intervals. It really doesn't seem that hard if it's explained well.</p>
<p>That said, no one in my PhD program ever explained this directly and well. I had to figure it out for myself. </p>Richard H. Serlin commented on ''Any P-Value Distinguishable from Zero is Insufficiently Informative''tag:typepad.com,2003:6a00d83451b33869e201bb0831c629970d2015-05-20T20:22:28Z2015-05-20T20:22:28ZRichard H. SerlinI always thought confidence intervals were a great antidote to this. The p-value may be 1% for whether it increases...<p>I always thought confidence intervals were a great antidote to this. The p-value may be 1% for whether it increases income, but show people a 99% confidence interval around negative $1.94/year and positive $2.97/year, and people immediately understand that the effect, at least based on this data, is nothing.</p>
<p>Confidence intervals should really be given regularly, rather than rarely, even though skilled people can construct them approximately in their heads.</p>cawley commented on ''Any P-Value Distinguishable from Zero is Insufficiently Informative''tag:typepad.com,2003:6a00d83451b33869e201b7c78db42d970b2015-05-20T18:25:44Z2015-05-20T18:25:44ZcawleyGlad he cleared that up ...<p>Glad he cleared that up ...</p>