Professor’s Long-Accurate Presidential Election Model Picks Biden. How Do We Judge This Model?

How impressive does this headline sound: “Forecaster who’s predicted every presidential winner correctly for 40 YEARS reveals who he thinks will win 2024 election”? We’ll use this to look at forecasting elections and showing hypothesis testing is never justified.

The headline is from an article on Allan Lichtman, a historian at American University, and his model that predicts presidential elections.

A historian who has correctly predicted every presidential election since 1984 has declared that ‘a lot would have to go wrong’ for Joe Biden to lose to Donald Trump – in November.

Allan Lichtman, a professor of history at American University in Washington, DC, devised a system, which he terms ’13 Keys’, and wrote a 1980s book explaining the idea.

He says the technique enables him ‘to predict the outcome of the popular vote solely on historical factors and not the use of candidate-preference polls, tactics or campaign events.’

Now “40 YEARS” is a long time, isn’t it? It sounds like the model has gone on for quite a while moving from success to success. Yet, when you count, it’s only 10 predictions, which sounds less impressive.

But not unimpressive. Ten out of ten is a pretty good hit rate.

The natural question to ask is if this model has real skill, or whether its success can be accounted for by something else, like luck.

To answer that question, let me introduce the Award-Eligible Patent-Worthy Briggs Presidential Election Model. It works like this:

1. If the Incumbent is running, he wins;
2. If the Incumbent is not running, the Opposite Party wins.

Opposite from whichever party held the office in the term before, that is. (Well, there is in reality one big Party, but they divide themselves by shirts and skins every four years for the joy of the game.)

Here is how the model works since 1960, with X = Success, and O = Failure.

1960, X
1964, O (assassination; incumbent dead)
1968, X
1972, X
1976, X
1980, O (stagflation; Reagan beats Carter)
1984, X
1988, O (Bush sr. rides Reagan’s coattails)
1992, O (Clinton over Bush sr.)
1996, X
2000, X
2004, X
2008, X
2012, X
2016, X
2020, O (fortification; covid)
2024, Predicts Biden

The Briggs Model doesn’t do quote as well as Lichtman’s over the same time period, but in its favor it is substantially simpler. The notation also suggests a quick adjustment, which is:

3. Invert the prediction if a substantially major event occurs.

That model is then almost perfect, as long as we count Kennedy’s assassination, Carter’s stagflation, and covid plus the fortification under Trump as substantially major events. It misses only Bush senior’s curious win. Problem with that modification is that we’d have to predict, in advance, what these “major events” are. After-the-fact justification is weak and not believable.

Anyway, the Briggs Model is unthinking. It performs the same service “Persistence” does in climate and weather forecast models. Persistence is the simple model that always predicts the next time period (year or day) will be just like this time period. Thus if today’s high is 70 F, we predict tomorrow’s high will be 70 F, and so on.

In spite of its simplicity, Persistence is quite a good model, especially for climate. This is because many things don’t change fast in the atmosphere. Persistence is the model we all use without really thinking too much. For instance, it’s usually true summer is hotter than winter. An excellent model, that.

Suppose as a rival I have a super scientific ascendant academic model made by Top Minds, using science, math, and even, Lord help us all, Artificial Intelligence. That model ought to be able to beat Persistence. And that same kind of model for elections ought to be able to beat the Award-Eligible Patent-Worthy Briggs Presidential Election Model.

If these super models can’t beat Persistence, then there is no reason to trust those super scientific ascendant academic AI models made by Top Minds.

It’s not at all clear global climate models can beat Persistence, but we’ll leave that topic for another day.

Lichtman’s model, which uses a mix of “short term economy”, “long term economy”, “policy change”, “scandal” and nine other metrics, seems to beat the Persistence-like Briggs model. It’s only seems because I can’t check Lichtman befor 1984.

There have been 16 elections since 1960, and the Briggs Model got 11 right, or 69%. Since 1984, ten elections and 7 right, or 70%. Lichtman got 100% since 1984; since 1960 unknown. Both models make the same prediction for 2024, so if you’re trying to choose which one to use, there is no choice.

The next question people ask is “Did Licthman get lucky or did he correctly ascertain some of the items in the causal path of the election?”

This is where the temptation to use “hypothesis testing” comes in. But don’t do it! Here’s why.

Let’s compare Lichtman with a coin-flip model. Our coin has one side “R” and the other “D”. We flip, and we remain ignorant of the causes of the flip (blog, Substack), and that is our model. Here is one such outcome after flipping the coin 10 times.

Year, Model, Outcome
1984, R, R
1988, R, R
1992, D, D
1996, D, D
2000, R, R
2004, R, R
2008, D, D
2012, D, D
2016, R, R
2020, D, D

Well looky that. The coin flip model nailed it. It predicts (I just checked) R for 2024. Which model would you rely on now?

Of course, somebody else’s coin flip model might not have done so well as this one¹. If this “coin” is some guy at the end of the bar’s pick every year, you might begin to trust it. We can’t know if this guy really does have insight, not judging by his results. We have to look outside the results, as it were. That’s what “hypothesis testing” purports to do.

What we really want is the probability each model’s guess for 2024 is correct. And that means we must have a fourth model which gives us that probability.

All probability, and therefore all probability models, are conditional on the assumptions you make.

Based on the usual assumptions, the coin flip model has a 50% chance of being right. Yes! This is the correct probability, and not some godawful-p-value-like excrescence. We don’t care about whether the coin is causal in its predictive powers. We already know it isn’t. We therefore do not need to test. No hypothesis test is sane here. Just you think that through if you don’t understand it. If you get it, you understand no hypothesis test is ever justified.

Based on assuming the causal conditions have remained the same in their correlative character regarding presidential elections, which we need to assume to use the Briggs or Lichtman model, then the probability the Briggs model will be correct (using the model deduced from its past performance only, a math trick we’ll go over in Class soon enough) is 67%, whereas the probability the Lichtman model will be correct is 92%.

The modified Briggs model also predicts D (Biden), on the additional assumption that the election will be fortified and because of the general lunacy of our political class. That modified model has a 89% chance of being correct—-but only if all the other assumptions about the nature of the political process since 1960 are the same.

If you don’t think they are the same, if you think “our democracy” has changed in fundamental ways related to causes of presidential elections, then you cannot use the Briggs model in any form, nor the Licthman model. You can use the coin flip, because its assumptions always hold. If you think the nature of presidential elections has changed, then you are on your own. You have to create a fifth model.

¹The probability the coin flip guesses 10 right is 1/1024. But if, say, 10,000 people use that model, then it’s nearly certain at least one of them will nail all 10 predictions. If you have a million coin-flip-like models, far from impossible, then about 1,000 of them will get 10 elections right.

Subscribe or donate to support this site and its wholly independent host using credit card click here. Or use the paid subscription at Substack. Cash App: $WilliamMBriggs. For Zelle, use my email: matt@wmbriggs.com, and please include yours so I know who to thank.