The 2008 U.S. presidential primary period saw a great degree of polling controversy, crystallized by Hillary Clinton’s win in the New Hampshire primary despite polls suggesting she would lose to Barack Obama. Afterward, the American Association for Public Opinion Research (AAPOR) conducted an inquiry into what exactly took place and why. As scholars from the University of Michigan and Temple University note, there is relatively little in the political science literature systematically analyzing the performance of polls during the presidential primary period.

A 2009 study published in Public Opinion Quarterly, “Dynamics of Poll Performance during the 2008 Presidential Nomination Contest,” extends the AAPOR analysis and looks to draw general lessons about measuring public opinion during the party nominating contests. The study, by Michael W. Traugott of the University of Michigan and Christopher Wlezien of Temple University, incorporates 258 polls in 36 Democratic contests and 219 polls in 26 Republican contests. The scholars’ chief hypothesis is that candidate momentum “can produce underestimates in measured poll support because pollsters will have difficulty capturing late surges.” Because the Democratic primary extended longer than that of the GOP, the study was able to explore the effects over time.

The study’s findings include:

• In Democratic contests, the winner’s vote share exceeded his or her poll share in in 29 of 36 states. Moreover, in five of the seven contests where the winner’s poll share exceeded the vote share, Clinton prevailed. “The mean vote-poll gap was 5.3 percentage points when Obama won and only 0.4 when Clinton won. In one sense, the pattern implies a ‘reverse Bradley effect,’ with the polls understating Obama’s ultimate vote share.” This likely indicates a “surge” in support for the lesser-known Obama as voters encountered him.
• The larger the percentage of a state’s black population, the more Obama’s vote share exceeded his poll share and the less Clinton’s vote share exceeded her poll share. The result “confirms speculation in the media about stimulated black turnout and its impact on Obama vote margins that was rife in the wake of the South Carolina Democratic primary.”
• In Republican contests, there was also tendency for early polls to underestimate vote share. This ended on Super Tuesday, however, when John McCain’s mean vote share was equal to his poll share.
• Overall, the findings show that the “polling problems in New Hampshire in 2008 were not the exception, but the rule. The media devoted unusual attention to the polling problems in New Hampshire because they came early and suggested that the Democratic primary winner would be Obama instead of Clinton. But the pattern of underestimating the winner’s vote share was present in most of the primaries.”

“Polling in a low information environment like the party primaries is not easy,” particularly early on, the authors state. “But the performance of the polls improves only slightly as the field is winnowed. This is because other forces come into play, including strategic voting in terms of both participation and the expression of candidate preferences. As a result, by the measures that we employed here, the performance of the polls improved only a little as the campaign wore on.”

Further variables need to be explored, the researchers explain, including “the impact of the Republicans’ tendency to employ a winner-take-all allocation of the delegates at stake in the primaries as opposed to the Democrats’ use of proportional allocation.”

Tags: polling, presidency, Iowa/New Hampshire, campaigns and media