America Association for Public Opinion Research

May 4, 2017

AAPOR Releases Report: An Evaluation of 2016 Election Polls in the U.S.

CHICAGO, IL, May 4, 2017 - The American Association for Public Opinion Research (AAPOR) released its report: An Evaluation of 2016 Election Polls in the U.S. This report is the product of a committee convened in the Spring of 2016 with the goal of evaluating the accuracy of 2016 pre-election polling for both the primaries and general election, reviewing variation by different poll methodologies, and identifying differences from prior election years.

“The committee succeeded in identifying several factors that led to the polling errors in 2016. Fortunately for the profession, most of those factors are fixable,” said Courtney Kennedy, Director of Survey Research at Pew Research Center, chair of the committee.

“There were two different stories regarding the performance of pre-election polls last year. National polls were actually quite accurate. But at the state level, the poll errors were quite large. We were able to identify several reasons why that was the case,” said Kennedy.

National polls were among the most accurate in estimating the popular vote since 1936. Collectively, they indicated that Clinton had about a 3 percentage point lead, and they were basically correct; she ultimately won the popular vote by 2 points.

State-level polls showed a competitive, uncertain contest but clearly under-estimated Trump’s support in the Upper Midwest.

“The committee found that there were multiple reasons for the polling errors in 2016. Two factors for which we found some of the strongest support were real late change in voter preference and the failure of many polls to adjust their weights for the over-representation of college graduates, who tended to favor Clinton in key states,” said Kennedy.

There are a number of reasons (not a silver bullet) as to why polls under-estimated support for Trump. The explanations for which the committee found the most evidence are:

  • Real late change in vote preference during the final week or so of the campaign. About 14% of voters in Michigan, Wisconsin and Pennsylvania decided on their presidential vote choice in the final week, according to the best available data. These voters broke for Trump by near 30 points in Wisconsin and by 17 points in Michigan and Pennsylvania.
  • Adjusting for overrepresentation of college graduates was critical, but many polls did not do it. In 2016 there was a strong correlation between education and presidential vote in key states. Voters with higher education levels were more likely to support Clinton. Furthermore, recent studies are clear that people with more formal education are significantly more likely to participate in surveys than those with less education. Many polls – especially at the state level – did not adjust their weights to correct for the overrepresentations of college graduates in their surveys, and the result was overestimation of support for Clinton.
  • Some Trump voters who participated in pre-election polls did not reveal themselves as Trump voters until after the election, and they outnumbered late-revealing Clinton voters. This finding could be attributable to either late deciding or misreporting (Shy Trump) in the pre-election polls. A number of other tests for the Shy Trump theory yielded no evidence to support it.
“In talking about polling last year, we have to strike a balance. It is important to correct the record and dispel the narrative that polling writ large is broken. We found that not to be the case,” said Kennedy. “At the same time, we must not whitewash what happened. There were errors. We must acknowledge and learn from them.”

“AAPOR is grateful to this committee for all of their hard work,” said Roger Tourangeau, AAPOR President. “As the leading professional organization of public opinion and survey research professionals in the U.S. AAPOR is working to ensure that quality measurement of public opinion.”
# # #

The American Association for Public Opinion Research (AAPOR) is the leading professional organization of public opinion and survey research professionals in the U.S., with members from academia, media, government, the non-profit sector and private industry. AAPOR members embrace the principle that public opinion research is essential to a healthy democracy, providing information crucial to informed policymaking and giving voice to the nation's beliefs, attitudes and desires. It promotes a better public understanding of this role, as well as the sound and ethical conduct and use of public opinion research.

"An Evaluation of 2016 Election Polls in the United States"

November 9, 2016

AAPOR to Examine 2016 Presidential Election Polling

Washington, DC, November 9, 2016 – Election years present particularly high profile moments for public opinion and survey research. This is a time when polls dominate the media and the accuracy of polls can be confirmed or refuted by the actual poll vote outcome.
The polls clearly got it wrong this time and Donald J. Trump is the projected winner in the Electoral College.  Although Clinton may actually win the popular vote, her margin is much lower than the 3 to 4 percent lead the polls indicated.  And many of the state polls overestimated the level of support for Clinton.
There is much speculation today about what led to these errors and already the chorus of concerns about a “crisis in polling” have emerged as headlines on news and social media sites. As final results continue to be tabulated it would be inappropriate for us to participate in conjecture.
Pre-election polling is critical to the industry. Such polling can support the democratic process and it offers a very public opportunity to showcase the benefits, and weaknesses, of survey research. Therefore, understanding and being able to articulate the overall outcomes of election polling, the changing methodologies being used, and the potential for variation in the accuracy of polls is vital for the industry.
As it has done in the last several elections, AAPOR has already convened a panel of survey research and election polling experts to conduct a post-hoc analysis of the 2016 polls. The goal of this committee is to prepare a report that summarizes the accuracy of 2016 pre-election polling (for both primaries and the general election), reviews variation by different methodologies, and identifies differences from prior election years.
The committee was convened in April 2016 and is chaired by Courtney Kennedy (of the Pew Research Center) and includes Scott Clement (Washington Post), Kristen Olson (University of Nebraska-Lincoln), Claire Durand (University of Montreal), Lee Miringoff (Marist College), Doug Rivers (YouGov), Josh Clinton (Vanderbilt University),  Mark Blumenthal (SurveyMonkey), Chris Wlezien (University of Texas), Kyley McGeeney (Pew Research Center), Evans Witt (PSRAI and President of NCPP), Charles Franklin ( and University of Wisconsin), and Lydia Saad (Gallup). The committee should have completed its work by May of 2017.
About AAPOR: The leading professional organization, representing over 2,000 public opinion and survey research professionals. AAPOR understands that journalists play a critical role as gatekeepers for the flow of opinion information. We will continue to provide free resources such as the free online course on Understanding and Interpreting Polls, Election Polling Resources and many others  to help sort through the deluge of survey and polling data.
# # #