Notes from APSA: Election Forecasting for 2024
September 10, 2024
Cincinnati. Tuesday morning. September 10th. Last week, I attended the American Political Science Association (APSA) in Philadelphia. Relevant to this blog was the panels I attended on election forecasting. In this post, I am giving you my thoughts on what I heard at these panels. I was not privy to the drafts of their papers, so I can only go on their presentations. Also, I am fairly critical of the models so I am not naming names here. It would a little unfair of me to do so because, despite my criticism, these folks are all doing a lot of hard work in an area where it is so hard to get it right. I would have liked to read some of their papers before writing this post, but with the Harris-Trump debate tonight and early voting just about to start in Pennsylvania, the subject matter here would likely get thrown on the back burner if I don’t write about it today.
At last week’s APSA conference, I attended a couple of very interesting panels on forecasting the 2024 election. Several different forecasts were presented. To be clear, these are academic forecasts not the popular ones done by 538, Silver Bulletin, New York Times, The Economist, and Princeton. The models presented at APSA largely, but not exclusively, included some combination of fundamentals and historical trends.
Presidential approval played a large, or at least an important, role in just about every model presented. Since the measure changes over time, researchers have to pick a moment in time to capture. For one of the forecasts, that time pegged Biden’s approval when it was 36%! The challenge - which I raised in both panels - is whether presidential approval really tells us anything anymore - or at least whether it tells us what it used to. Presidential approval used to be a good indicator of how well the incumbent’s party would perform in mid-term elections. That did not work in 2022 and it hasn’t stopped Democrats from over-performing in nearly every special and off-year election since 2020.
It could be a sign of the times or it could be a function of the candidates. Let’s return to Biden’s 36% approval rating that one forecaster used. I can say with confidence that at no point was Biden ever going to get anywhere near as low a vote total in 2024 as 36%. In fact, it is hard to believe that he would have gotten less than 45%. There is still good reason to believe that he could have won the election had he stayed in (although I think he made the correct decision withdrawing, to be clear). In previous years, a 36% approval rating during an election year would spell defeat for an incumbent president. Not this year. Why is that?
That’s still unclear, but what we can say with some measure of confidence is that a lot of Democratic voters were comfortable expressing disapproval for Biden while still intending to vote for him. On the other hand, there seems to be no significant number of Republicans who would ever express disapproval for Trump yet still vote for him. What this creates is an illusion that in terms of electoral support that Biden was more unpopular than he actually was - both in absolute terms and relative to Trump.
A participant at one of the APSA panels noted that presidential - and perhaps a lot of political - approval ratings appear to be flat now. They do not move very much, no matter what happens. This could be a function of political polarization. That may be true, but I still think the degree to which Biden’s rating was low was a function of Democrats willingness to express disapproval of the party leader. Some may have done that to try to signal to Biden that he should not run again (even though they would probably support him if he did), others wanted to express disapproval for policy decisions, and some others were really not going to support him.
There is no similar phenomena at work with Republicans. Nearly all Republicans report they approve of Trump and nearly all Republicans report they do not approve of Biden. If Democrats were this blindly loyal in answering a simple survey question, his approval rating would have been much higher all during his presidency - and it was for a brief period at the beginning of it.
A further complication is that the models do not take into consideration the popularity of the opposition candidate, in this case Trump. In the past that has not mattered. For the most part, every party nominee was essentially no different than the generic nominee. But Trump is different than any nominee before him. First, he is a former president who has supporters that claim he is still president. Second, he is a convicted felon and is currently under felony indictments in at least three jurisdictions. Third, he not only increases turn-out among his supporters, but among his opponents. A lot of people are motivated to vote against him. It is hard to see how any of this will not factor into the outcome this year. Yet, none of the forecast models incorporate any way to capture this.1
Yet another complication is how we are to think of Harris’s candidacy. She is the sitting Vice President, but she is running as - and because of who her opposition is has a legitimate claim to be - the change candidate. Her approval ratings are higher than Biden’s. In fact, once she became the presumptive nominee her approval ratings went up. Does this tell us anything about using Biden’s approval rating to build a forecast? Her standing in the polling was a significant improvement over Biden. In fact, Biden was losing the race and she is winning it. How can that not matter?
A big problem in forecast models is something that is problematic in many social science - and particularly political science and economic - models: confounding issues are eliminated from the model because either they cannot be easily incorporated or because they make the model less explanatorily powerful. A lot of models are built to explain things that already happened, and then are used - by the researcher or by others - to predict future events. Taking current public opinion out of the model seems to be unwise for predicting something has volatile as an election, but aside from presidential approval rating election forecasts tend to do just this.
Forecasters will tell you that their model has been right a lot of the time. But when it comes to presidential elections this is deceptive. Most elections have been easy to call - at least as far as who would ultimately win. Here are some examples from the past 60 years: 1964, 1972, 1984, 1988, 1992, 1996, 2008, 2012 (yes, that was easy), and even 2020. No one needed any math or modeling to call the winners in those years. I think it was probably even easy to call 1968 and 1980 (again, only in terms of who would win). However, working on campaigns for so long I get nervous every year - and that’s because in politics you are always right until you are wrong. Conventional wisdom always fails at some point - and often you do not see it coming. The same is true with modeling.
One of the forecast models boasted that it picked the results in 90% of the states correctly over time - and 95% in 2020. This sounds impressive until you recall that anyone can pick 75-80% of the states correctly in a given election. Getting 90% correct means missing five states. Getting 95% correct means missing two or three states. If those states included Wisconsin, Michigan, and Pennsylvania in 2016 then that was not such a great forecast after all, was it?
Considering this, is there any good reason to do this kind of fundamentals/historical forecasting? If you want to know who is going to win in November, I don’t think this modeling is very helpful. But, if you are trying to understand the impact that campaigns or other exogenous factors have on elections, it is useful. All of the approaches generally do this: they look at those indicators that have historically pointed to electoral outcomes and use them to predict who should win next if those indicators are as important today as they were in the past. A good model can help a campaign understand what kind of political environment and challenges they face in a particular election and perhaps what they might need to do to overcome them. As for analysis, when the election is over and the forecast was wrong we can examine why and see if there are any lessons we can learn about how to run campaigns or what’s changed in our politics from previous years.
Grimmer, Knox, & Westwood think no forecast can be accurate right now because presidential elections are so rare and we have had so few.2 For them, we can’t know enough about how US presidential elections predictably work until we have dozens, perhaps hundreds, of additional elections to consider. While, they may be right, it’s not clear that forecast models will ever be able to predict every election. Social scientists sometimes try too hard to mimic physics. The human variable in human activity will always be, well, variable. We just cannot make any sweeping assumptions about how events or approval ratings impact electoral outcomes.
All of the forecasts I saw presented at APSA have Trump winning the election. But, to be clear, these models do not take into consideration who the specific candidates are. Theey would have DeSantis beating Harris, Haley beating Newsom, and Hogan beating Sanders. The specific candidates do not matter in these forecasts; what they are really telling us is that in this political environment the out-party - in this case the Republicans - should win. That alone should give us some pause. And some of these forecasts have the victory by what I think are ridiculous margins, especially when you consider the guy actually running is Trump.
One forecast has him beating Harris 54-45 and winning over 330 electoral votes. The electoral vote number is difficult because, as we have seen all too often in this century, it is possible to win the popular vote and lose the electoral vote. But does anyone really believe that Trump can win 54% of the popular vote? He has not reached 50% in any approval rating that I am aware of since 2015. In his two national campaigns, he did not win more than 47% of the vote either time. He is currently polling at or near his 47% ceiling (it sure seems like it is; we’ll see in November). Part of this may be factoring out minor party candidates to have a pure two-party race. That still would give Trump a majority of the vote in a multiparty race. It is very hard to see that happening, even if he wins the electoral vote. It’s these kind of smell tests that should tell researchers there might be something really off in their models.
However, to be fair - we do not know the future. If we wake up on November 6 and Trump has beaten Harris by 9 points and won over 330 electoral votes, this researcher would prove us wrong. And we might have learned something from his model.
Still, think of all that has to change in the next 60 days for a result like this to occur. Trump has to win large portion of the independent vote and a not insignificant portion of the Democratic vote to amass that much support. Harris has unified Democrats and independents are trending in her direction. Is a bad debate going to do it? No. Biden’s debate problems were not that he had a bad night, but that he did so in a way that reinforced an existing narrative that he was too old for another term. And even then, Trump did not gain more than a couple points in the national polling. Biden probably would not have lost to Trump 54-45, so why would Harris?
Who are these perusable Trump voters? I don’t see any significant number of them in the electorate. I don’t think the Trump campaign does either, which is why they are focusing on remaining negative and discouraging people from voting. They want the less-than-absolutely-committed voter to walk away because they know Trump cannot win that person over at this point - and, because it has to be part of the strategy, he is not even going to try. We will likely see Trump’s strategy to drive up Harris’s negatives at tonight’s debate.
The election forecasts you will more likely be reading and hearing about this fall will be from the Times, 538, and Silver Bulletin, among others. But you may hear more about these academic forecasts - particularly since they are so bullish on Trump winning. I would not let it worry you. It is unlikely that these models are able to capture all the unusual dynamics at play this year. Even these academic forecasters were quick to remind us at different point in the panels that campaigns matter. So, get out and organize, mobilize, and vote!
To be fair, at least one of the forecasters admitted they did try to figure this out but could not come up with a way to do it.