« North Korea: Engage or Isolate? | Main | Methodological Issues in Economics »

Thursday, November 30, 2006

Ray Fair: Interpreting the Predictive Uncertainty of Presidential Elections

A colleague emails to let me know about a new paper on presidential elections by Ray Fair (thanks Jeremy). The paper, which is forthcoming in the Journal of Political Economy, looks at the problem of determining the degree of uncertainty in predictions of election outcomes and argues that the standard errors released along with traditional polling data miss "an important type of predictive uncertainty..." Fortunately, the uncertainty "can be estimated using data from political betting markets." To adopt this betting market approach, the paper imposes a ranking assumption (explained below) to restrict the number of possible admissible outcomes. The paper then shows that if this assumption is correct, and there is empirical evidence from betting markets provided in support of the assumption, it implies that political parties "should spend all their money on a few states, which seems consistent with their actual behavior." So now you'll know why your state was either skipped or bombarded with advertising last presidential election:

Interpreting the Predictive Uncertainty of Presidential Elections, by Ray C. Fair, September 2006, International Center for Finance, Yale University, WP No. 06-25, Cowles Foundation, WP 1579: 1 Introduction A common way of assessing the uncertainty of election predictions is to use the standard errors that are released by polling organizations. Almost all polling organizations release both a mean prediction and a standard error of the mean prediction. This paper argues that there is an important type of predictive uncertainty that is not captured by these standard errors and that can be estimated using data from political betting markets. Section 2 presents this argument and uses data from the Intrade[1] political betting market to provide estimates. The idea is that there are a number of possible "states" or "conditions" of nature that can exist on election day, of which one is drawn on election day. The uncertainty is which condition will be drawn.

A "ranking" assumption about dependencies across U.S. states is presented in Section 3 that greatly restricts the possible conditions of nature than can exist on election day. ... As restrictive as this assumption seems, it will be seen that the assumption is strongly supported by the Intrade data. Section 4 is concerned with the question of how the two political parties should behave regarding campaign spending across states if the ranking assumption is correct. They should spend all their money on a few states, which seems consistent with their actual behavior. This result is contrary to results in the literature that are based on the assumption of independence across states, where there is some spending in all states.

2 Predictive Uncertainty

Conditions of Nature on Election Day A standard error from a poll measures sampling uncertainty. The larger the sample size, the smaller the standard error. Consider for sake of argument that every eligible voter in a state was asked the day before the election whether he or she was planning to vote and for whom. This would yield a mean vote share with a standard error of zero.[2] On this score, there would be no uncertainty left. The argument here is that there is still uncertainty...

There are many reasons people might do something different on election day than they told the pollster on, say, the day before the election they would do. The weather may be different than they expected, which may affect their decision on whether to vote. They or family members may wake up ill or cranky, which may change their decision to vote or for whom to vote. They may have lied to the pollster and voted for a different candidate than they said they would. They may have changed their mind as they were in route to the voting booth, perhaps because they had not thought much about the election until then or because of a conversation they had a few hours before with someone they trusted. Reasons like these have been advanced many times in discussions of polling results, and the main point here is simply that they may pertain even on the day of the election. This type of uncertainty would not be captured even with a poll of every eligible voter on the day before the election. ...

Measurement Using Intrade Data It should be clear that polling data cannot be used to measure the type of uncertainty that is the concern of this paper. This uncertainty exists even if there is no sampling error. Fortunately, political betting markets do provide a way of measuring this uncertainty. ...

3 The Ranking Assumption: A Restriction on the Possible Conditions of Nature

The Ranking Assumption The ranking assumption is easy to describe. Rank the states by pi [the market's assessment of the probability that a particular candidate, e.g. Bush, will win state i] ... using the Intrade data. The assumption is then that there is no condition of nature in which Bush wins state i and loses a state ranked higher than i. If, for example, Texas is ranked higher than Massachusetts, then in none of the n conditions of nature does Bush win Massachusetts and lose Texas. There may be conditions in which Bush wins Massachusetts (Kerry makes some serious error), but in these conditions Bush also wins Texas.[5] It is common in previous work to assume some form of independence. ... [footnote 5: Ed Kaplan has pointed out to me that given a ranking like in Table 1, under the ranking assumption there are only 52 possible outcomes... This compares to 251 possible outcomes, about 2.25 million billion. A remarkable economy of outcomes has been achieved by the ranking assumption!] ...

The ranking assumption does not, of course, directly concern different sets of the n possible conditions of nature. It simply puts restrictions on the n possible conditions of nature that exist on election day. If state i is ranked ahead of state j, then in no condition of nature does Bush win j and lose i. The concept of different sets of the n possible conditions of nature is not needed.

The Ranking Assumption and the Intrade Data It will now be seen that the ranking assumption is strongly supported by the Intrade data. ...

5 Conclusion This paper has argued that there is an important type of election predictive uncertainty not captured by polling standard errors. It can be measured using prices from political betting markets...

The ranking assumption puts severe restrictions on the possible conditions of nature than can exist on election day, but it is supported by the Intrade data ... and by the actual outcome of the 2004 election. It will be interesting to see how it does in the 2008 election.

If the ranking assumption is correct, the stochastic simulation results in Section 4 show that the two political parties should spend only in a few states. The larger the variance of the estimation errors, the larger is the number of states in play, although even for large variances the number of states in play is small.

    Posted by on Thursday, November 30, 2006 at 04:13 PM in Academic Papers, Economics, Politics | Permalink  TrackBack (0)  Comments (2)

    TrackBack

    TrackBack URL for this entry:
    https://www.typepad.com/services/trackback/6a00d83451b33869e200d8350348bd69e2

    Listed below are links to weblogs that reference Ray Fair: Interpreting the Predictive Uncertainty of Presidential Elections:


    Comments

    Feed You can follow this conversation by subscribing to the comment feed for this post.