Walking my dog. On the Metro. In line at a sandwich shop. People keep coming up and asking me about “the polls.” What do the numbers mean? Should they be worried about the election? If a set of swing state polls is released, the odds are by the end of the day I will have been asked by a friend, a relative, a neighbor, or a stranger, or several, “Did you see that poll in Nevada? Why was there a shift of three points since the last one? How could Pennsylvania be going in a different direction? And North Carolina, really? Do you think that’s accurate?” If they start referencing Nate Silver, Nate Cohn, or any of the other pollster celebs…I want to scream.
Polls, to be hyperbolic about it, have ruined American politics. Okay, a lot has ruined American politics. But polls have certainly made American politics less enjoyable. Many of those who follow politics—and not enough citizens do—have become slaves of polling, overly obsessed with these surveys and palpitating over the slightest changes. I’m not unsympathetic. This election is prompting more anxiety than most. The oft-repeated mantra that the 2024 race could determine whether the United States remains an imperfect democracy or slips toward a more authoritarian form of governance is true. Thus, every iota of data related to the face-off between Donald Trump and Kamala Harris appears loaded with relevance and consequence. Still, the hyperfixation on polls is unwarranted and distracts us from other important aspects of this most important election.
George Gallup (sitting), director of the American Institute for Public Opinion, and the institute's chief statistician, Edward G. Benson, working at an office in Princeton, New Jersey, in 1941.
The below article first appeared in David Corn’s newsletter, Our Land. The newsletter comes out twice a week (most of the time) and provides behind-the-scenes stories and articles about politics, media, and culture. Subscribing costs just $5 a month—but you can sign up for a free 30-day trial.
Walking my dog. On the Metro. In line at a sandwich shop. People keep coming up and asking me about “the polls.” What do the numbers mean? Should they be worried about the election? If a set of swing state polls is released, the odds are by the end of the day I will have been asked by a friend, a relative, a neighbor, or a stranger, or several, “Did you see that poll in Nevada? Why was there a shift of three points since the last one? How could Pennsylvania be going in a different direction? And North Carolina, really? Do you think that’s accurate?” If they start referencing Nate Silver, Nate Cohn, or any of the other pollster celebs…I want to scream.
Polls, to be hyperbolic about it, have ruined American politics. Okay, a lot has ruined American politics. But polls have certainly made American politics less enjoyable. Many of those who follow politics—and not enough citizens do—have become slaves of polling, overly obsessed with these surveys and palpitating over the slightest changes. I’m not unsympathetic. This election is prompting more anxiety than most. The oft-repeated mantra that the 2024 race could determine whether the United States remains an imperfect democracy or slips toward a more authoritarian form of governance is true. Thus, every iota of data related to the face-off between Donald Trump and Kamala Harris appears loaded with relevance and consequence. Still, the hyperfixation on polls is unwarranted and distracts us from other important aspects of this most important election.
Polls don’t matter. Or maybe they do. It depends on your definition of “matters.” By all measurements, this is a close race. What else do you need to know? The candidates are within a few points of each other in the national polls and the swing state polls. But the difference is usually within the reported margin of error. That means the poll that has just caused you heartburn may not have any value in terms of telling us what will happen on Election Day.
And get this: That margin of error may not even be accurate.
While doing a little (but not much) research for this rant, I came across a useful article from the Pew Research Center, which does a lot of polling. It was published this summer and called “Key things to know about U.S. election polling in 2024.” The piece made the usual points. In 2016 and 2020, polling underestimated Trump’s performance. (Polls on average overestimated Hillary Clinton’s strength by 1.3 percent and Joe Biden’s by 3.9 percent.) The 2022 nonpartisan polls—meaning those taken by the media and research centers and not by campaigns and political groups—were more accurate than people may have assumed after the mythical “red wave” did not materialize. Polling methodologies have shifted to keep current with changes (such as the decrease in the use of landlines and a low response rate). Pollsters have improved how they weigh demographic variables to obtain representative samplings.
What was most interesting in this article, though, was what it said about the margin of error: “The real margin of error is often about double the one reported.” What? Read that again. Double the margin of error. “A typical election poll sample of about 1,000 people,” Pew tells us, “has a margin of sampling error that’s about plus or minus 3 percentage points.” That’s usually the number you see associated with a poll. Three percent. That doesn’t seem so bad.
But there are other errors. If you must know, they are called noncoverage error, nonresponse error, and measurement error. I’m not going to go into the technical details here. But this is the bottom line from Pew: “The problem is that sampling error is not the only kind of error that affects a poll. Those other kinds of error, in fact, can be as large or larger than sampling error. Consequently, the reported margin of error can lead people to think that polls are more accurate than they really are…Several recent studies show that the average total error in a poll estimate may be closer to twice as large as that implied by a typical margin of sampling error. This hidden error underscores the fact that polls may not be precise enough to call the winner in a close election.”
So are you really going to pull your hair out over a poll with a margin of error of 6 points? C’mon. Get a grip.