Theory of Change #005: Can polls be trusted? (With Courtney Kennedy of the Pew Research Center)
Election polls nationally were fairly accurate in 2016, but big errors at the state level indicate that conducting surveys will more difficult in the future
If you read, listen or watch the news today, it’s impossible to avoid public opinion polls. They are literally everywhere. The president’s approval rating, what people think about impeachment, even what the best fast food restaurant is.
But as omnipresent as opinion surveys are, a lot of the math and science that goes into them is relatively unknown to many people. There are also a lot of questions about how polls work and how they should work. Why do most polls include more Democrats than Republicans? Do political “independents” actually exist?
There are also a lot of misunderstandings about opinion surveys. Many people, think that they got the 2016 presidential election wrong. But that’s not quite true. In fact, the national polls did a pretty good job of predicting what the vote would be. But some of the state polls, did get it wrong, and the Constitution awards the presidency to the candidate with the most Electoral College votes—not the popular vote.
To get to the bottom of these issues, Theory of Change host Matthew Sheffield spoke with Courtney Kennedy, the director of survey research at the Pew Research Center. In the conversation, Kennedy talks at length about the profusion of survey research companies and the rise of polling aggregation operations like those at the FiveThirtyEight, RealClearPolitics, and the New York Times.
While she is skeptical about some pollsters’ practices, she also wonders whether it make sense to lump low-quality polls with those from organizations with much stricter standards.
She also warns that trying to create election forecasts that assign a candidate’s percentage chance of winning based on polls may actually suppress voter turnout.
Kennedy also discusses how polling operations have had to change how they conduct research in light of the rise of spam phone calls. One surprising change that the Pew Research Center has made is to go old-school by recruiting poll respondents via letters in their mailboxes.