Social Science Statistics Blog

« Call for Papers: Conference at Harvard on Networks in Political Science | Main | Nickerson on “How (and how not) to Study Voter Registration Experimentally” »

9 January 2008

Polling the New Hampshire Primaries

New Hampshire voted last night, and managed to set off another frenzy of introspection among pollsters and pundits. On the Democratic side, public polls released after Iowa showed Obama leading Clinton by an average of about 10 points, but in the end Clinton of course edged out a narrow victory. The polls were much closer on the Republican side, but the "miss" on the Democratic side has already produced much concern about "New Hampshire's Polling Fiasco". Perhaps the witch-hunt that ensues whenever polls appear to be inaccurate in a major election should be viewed as a positive sign about the acceptance of survey research in the media and electorate; at the very least, these kinds of things keep a fair number of our colleagues gainfully employed. From my perspective, it would have been nice to have polls that were more consistent with the eventual outcome since we were planning to use them as examples in an undergraduate class; they will still be examples, but now the focus will be more on total survey error.

Why did the poll results diverge from the outcome? Several hypotheses are floating around. Jon Krosnick from Stanford has an opinion piece pointing to the ballot order effect; Hillary Clinton won the random draw to end up near the top of the ballot. There is certainly a lot of evidence that ballot-order effects matter, but my sense of the literature is that these effects tend to be smaller for better-known candidates, and it is hard to imagine a candidate better known than Hillary Clinton. Dan Ho and Kosuke Imai have written two articles on elections California that take advantage of randomization to estimate ballot-order effects:

Estimating Causal Effects of Ballot Order from a Randomized Natural Experiment: California Alphabet Lottery, 1978-2002

Randomization Inference With Natural Experiments: An Analysis of Ballot Effects in the 2003 California Recall Election

More comment has focused on the possibility that Obama suffered from the "Bradley effect", in which some white voters say that they will support a black candidate when responding to poll questions but end up voting for a white candidate at the ballot box. There is not much academic literature on this supposed effect; here is a Pew Research Center note from last year; ironically, it is titled "Can you trust what polls say about Obama's electoral prospects?"

Finally, many observers have pointed to political prediction markets as either a supplement or alternative to traditional polls for predicting election outcomes, on the idea that these can incorporate other sources of information and require participants to put their money where their mouth is. They didn't do so well, either, as Jon Tierney notes on his New York Times blog, although the market prices did begin to move during the day. There is an interesting research agenda regarding the relative merits of polls and markets (and how markets integrate the information from various polls); Bob Erikson and Justin Wolfers, who are leading contributors to this literature, have an interesting exchange on this question on Andrew Gelman's blog (posted a week before New Hampshire, but even more interesting today).

Posted by Mike Kellermann at January 9, 2008 10:57 AM