Should You Worry About the Polls?
September 30, 2024
No. Do not allow yourself any more anxiety than you already have in your life. The polling will be all over the place - or at least will appear that way. If you must follow the polls regularly, look at the averages. Individual polls can be way off, but the averages will smooth them out. However, that does not mean they will predict the outcome. Polling is not prediction. In fact, there are other ways that polls can be wrong, even where the topline result might be correct. Consider Tom Bonier’s point in this tweet:
Tom has a really good point here that is almost never discussed. Getting the topline outcome of the horse-race correct is not indicative of a poll that was “right.” If the pollster had the underlying demographics of the turnout wrong, then the topline outcome of the poll was just lucky. No one thinks to ask about that because in the end all anyone cares about is who wins. But, if we are going to take polling seriously for what it can offer us before the election outcome is known, putting a voter turnout model together is important - probably even more important than getting the outcome correct. The outcome, after all, is subject to all sorts of error even where the turnout model is correct. A poll that said Biden would win the popular vote with 51% with a 3% margin of error would have been spot-on. It would also have been “right” if Biden got 48%.
Tom took a look how AtlasIntel - the pollster who “got it right” in 2020, and has Trump winning this year - did with the voter turnout in 2020.
Georgia: Trump +2 (he lost by 0.3); Black voter share underestimated by five points
Pennsylvania: Trump +1 (he lost by 1.2); Demographics pretty much on the nose
Michigan: Biden +2 (he won by 2.8); Overestimated Black share
Arizona: Trump +2 (he lost by 0.4); Badly underestimated both Hispanic and youth turnout
Wisconsin: Biden +2 (he won by 0.4); Got race and age turnout correct
AtlasIntel consistently showed Trump winning voters under 30 by big margins, which did not happen.
Tom’s conclusion: Keep all of this in mind when freaking out about the polling this year.
Last week, I was having a pint at my favorite pub and talking to a friend who asked me something about the election. Some guy overheard us and started going on about how the polls are always wrong and they are just making it up and lying. He gave the usual “they” were wrong in 2016 and 2020. The “they” he and other people usually are referring to when people say this is the election forecasts, not the polls. It’s important to understand that. Election forecasts are not polls. Polls look at a snapshot in time. Forecasts are (typically probabilistic) predictions of election outcomes, often using polling data but often using a lot of other factors as well.
Polls are probably always wrong if what you are looking for is absolutely accuracy across everything they assume and observe. I suppose the real challenge for election polling is that in the end, the election produces a very specific outcome. When marketers uses survey research to see if there is a market for some brand of laundry soap, they do not need precision to decide whether or not there is. If only 48% of the country will buy their brand instead of 52%, that probably is not a problem for the soap company. They should sell a lot of product. However, if a pollster was this far off in a two-way race on the eve of the election, they would have been roundly jeered as having been wrong.
Pollsters are not “lying” as that guy-at-the-pub was claiming. Well, not the good ones anyway. Just because they miss doesn’t mean they are being dishonest. However, we have seen in the last few cycles a phenomenon where new or unheard of outfits calling themselves pollsters have peppered the field with suspicious polls (usually not being transparent at all about methods, demographics, or even the questions they asked) that have leaned further to the right than the rest of the polling. They were the reason why pundits were telling people there might be a “red wave” in 2022. And this weekend we discovered that Rasmussen Reports, a notoriously bad pollster that 538 has recently banned, has in fact been working for - to at least in cahoots with - the Trump campaign.
For people like me who spend too much time considering such things, it is not difficult to spot the disingenuous or suspicious pollsters. The media should be able to do it too, but they continue to assume everything is in full and equal balance - especially when the Republican pundits they pay start crowing about these “polls” and how a red wave is coming. For the average person out there, the flood of crappy polls and seemingly (or actually) opaque forecasting can be dizzying and hard to wade through. So if it’s already starting to make you anxious, maybe it’s time to look away.
It may be early to say this, but I am going to. Stop looking at the polls and the forecasting unless you enjoy the game of it. (I assume a lot of you who actually read this newsletter enjoy the game of it, but if it stresses you take care of yourself first!) Feel free to check in with me once or twice (or more) a week to see if there is anything going on that you might be interested in. Here’s why. The polling is going to be wrong and the forecasts don’t tell us anything useful. It’s safe for you to understand that Harris has a real chance to win this election. Go out and organize & mobilize voters. Be mindful that some organizers will scare you about how close the race is. They seem to think scaring volunteers is a way to motivate them. Just ignore it and remember why you are volunteering. You are already there; you don’t need further motivation.
Soon, we are going to have a lot of early voting data to consider. It was this data that told us in 2022 that Democrats were going to do much better than the pundits were predicting. While we won’t know who voted for whom, we get a really good idea of what voter turnout will look like. It will also reveal something about voter enthusiasm as well. For starters, here’s a taste for you. As of close of business on Friday, Pennsylvania had received 1,409,889 applications for mail ballots. Here is the partisan breakdown:
Democrats: 62.5%
Republicans: 26.5%
Others: 11.0%
Finally, there is a fantastic new tool out there for you traveling volunteers and impact donors. Vote Maximizer comes from Sam Wang and the good folks at the Princeton Election Consortium. It can tell you where you can make the most impact with your volunteer time or donations. There is a video tutorial on the site that explains how the tool works. It’s not just good for federal races, but for state and local races as well. It may put my trifecta state analysis out of business, but this tool is much better for your own personal use. It is probably the only useful thing you - as an ordinary volunteer - can do with polling data. If you use Vote Maximizer, let me know how it works for you!
I will be on the trail in Philadelphia soon, but I will continue to write while I am there. I don’t get paid for campaign work; I volunteer. If you want to help me out, please get a paid subscription. It really does help and I appreciate it!