Before the 2009 season began, I developed a very simple predictive1 model based solely on each NFL team’s historical record. Here is what was said about the model:
I used data from 2002 until 2008. In 2002, the NFL changed the league structure (they increased the number of divisions), so this felt like a natural point of demarcation. All data weighted equally. No account of the fact that teams are constrained to winning a certain number of games has been taken. For example, suppose there are only two teams in the entire league: it is then impossible that both can win (or lose) all their games. All ties (only one) have been counted as wins.
Below, the results of those predictions.
This table shows, for each team, the probability of winning 0 games, 1 game, …, 16 games. It has been sorted so that the team (the Patriots) with the highest probability of winning 16 is first, and the team (the Lions) with the lowest probability is shown last. All probabilities are rounded: probabilities less than 1% are shown as 0. The most likely number of games won is in bold.
The actual numbers of games won is shown by an orange background.
| 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Patriots | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 2 | 5 | 10 | 16 | 21 | 21 | 16 | 8 | 2 |
| Colts | 0 | 0 | 0 | 0 | 0 | 1 | 2 | 4 | 8 | 13 | 18 | 20 | 17 | 11 | 5 | 2 | 0 |
| Titans | 0 | 0 | 0 | 0 | 0 | 1 | 2 | 5 | 9 | 15 | 19 | 19 | 15 | 9 | 4 | 1 | 0 |
| Steelers | 0 | 0 | 0 | 0 | 0 | 1 | 2 | 6 | 10 | 15 | 19 | 19 | 14 | 9 | 4 | 1 | 0 |
| Eagles | 0 | 0 | 0 | 0 | 0 | 1 | 3 | 6 | 11 | 16 | 19 | 18 | 14 | 8 | 3 | 1 | 0 |
| Giants | 0 | 0 | 0 | 0 | 1 | 2 | 5 | 9 | 14 | 18 | 19 | 15 | 10 | 5 | 2 | 0 | 0 |
| Chargers | 0 | 0 | 0 | 0 | 1 | 2 | 5 | 10 | 15 | 18 | 18 | 15 | 9 | 5 | 2 | 0 | 0 |
| Panthers | 0 | 0 | 0 | 0 | 1 | 3 | 6 | 11 | 16 | 18 | 17 | 13 | 8 | 4 | 1 | 0 | 0 |
| Packers | 0 | 0 | 0 | 0 | 1 | 3 | 7 | 12 | 16 | 19 | 17 | 12 | 7 | 3 | 1 | 0 | 0 |
| Broncos | 0 | 0 | 0 | 0 | 1 | 3 | 7 | 12 | 16 | 19 | 17 | 12 | 7 | 3 | 1 | 0 | 0 |
| Seahawks | 0 | 0 | 0 | 0 | 1 | 4 | 8 | 13 | 17 | 18 | 16 | 12 | 7 | 3 | 1 | 0 | 0 |
| Ravens | 0 | 0 | 0 | 1 | 2 | 4 | 8 | 13 | 17 | 18 | 16 | 11 | 6 | 2 | 1 | 0 | 0 |
| Bears | 0 | 0 | 0 | 1 | 3 | 7 | 11 | 16 | 18 | 17 | 13 | 8 | 4 | 1 | 0 | 0 | 0 |
| Buccaneers | 0 | 0 | 0 | 1 | 3 | 7 | 11 | 16 | 18 | 17 | 13 | 8 | 4 | 1 | 0 | 0 | 0 |
| Jaguars | 0 | 0 | 0 | 1 | 3 | 7 | 12 | 17 | 18 | 17 | 12 | 7 | 3 | 1 | 0 | 0 | 0 |
| Cowboys | 0 | 0 | 0 | 1 | 4 | 8 | 13 | 17 | 18 | 16 | 11 | 7 | 3 | 1 | 0 | 0 | 0 |
| Vikings | 0 | 0 | 0 | 1 | 4 | 8 | 13 | 17 | 18 | 16 | 11 | 7 | 3 | 1 | 0 | 0 | 0 |
| Falcons | 0 | 0 | 0 | 2 | 4 | 9 | 14 | 18 | 18 | 15 | 11 | 6 | 3 | 1 | 0 | 0 | 0 |
| Saints | 0 | 0 | 1 | 2 | 5 | 9 | 14 | 18 | 18 | 15 | 10 | 5 | 2 | 1 | 0 | 0 | 0 |
| Chiefs | 0 | 0 | 1 | 2 | 5 | 9 | 14 | 18 | 18 | 15 | 10 | 5 | 2 | 1 | 0 | 0 | 0 |
| Cardinals | 0 | 0 | 1 | 2 | 5 | 10 | 15 | 18 | 18 | 14 | 9 | 5 | 2 | 1 | 0 | 0 | 0 |
| Dolphins | 0 | 0 | 1 | 3 | 7 | 12 | 16 | 18 | 17 | 13 | 8 | 4 | 1 | 0 | 0 | 0 | 0 |
| Redskins | 0 | 0 | 1 | 3 | 7 | 12 | 16 | 18 | 17 | 13 | 8 | 4 | 1 | 0 | 0 | 0 | 0 |
| Jets | 0 | 0 | 1 | 3 | 7 | 12 | 16 | 18 | 17 | 13 | 8 | 4 | 1 | 0 | 0 | 0 | 0 |
| Bills | 0 | 0 | 1 | 3 | 7 | 12 | 17 | 19 | 16 | 12 | 7 | 3 | 1 | 0 | 0 | 0 | 0 |
| Bengals | 0 | 0 | 1 | 3 | 7 | 12 | 17 | 19 | 16 | 12 | 7 | 3 | 1 | 0 | 0 | 0 | 0 |
| Texans | 0 | 0 | 1 | 3 | 7 | 12 | 17 | 19 | 16 | 12 | 7 | 3 | 1 | 0 | 0 | 0 | 0 |
| Rams | 0 | 0 | 1 | 3 | 7 | 12 | 17 | 19 | 16 | 12 | 7 | 3 | 1 | 0 | 0 | 0 | 0 |
| 49ers | 0 | 1 | 3 | 7 | 13 | 17 | 19 | 17 | 12 | 7 | 3 | 1 | 0 | 0 | 0 | 0 | 0 |
| Browns | 0 | 1 | 4 | 9 | 14 | 19 | 19 | 15 | 10 | 6 | 2 | 1 | 0 | 0 | 0 | 0 | 0 |
| Raiders | 0 | 2 | 7 | 13 | 18 | 20 | 17 | 12 | 7 | 3 | 1 | 0 | 0 | 0 | 0 | 0 | 0 |
| Lions | 1 | 5 | 12 | 19 | 21 | 18 | 12 | 7 | 3 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
As expected, the model was crudely useful, at best. It did capture a certain rough order in the standings, and it did better than the naive model which simply said “All teams will win 8 games.” But it would not have been of much, if any, use to a gambler, except as a starting point.
Do not forget that the model’s predictions were conditional on the belief that all that mattered was the last seven years’ performance, a belief which is clearly false. No account was taken, for example, of how the players have changed over that time period, nor did I use any information about the schedule. This model is meant to be a baseline with which to compare the performance of other, more sophisticated models.
If you are a gambler, or somebody who actively tracked the season, we would love to hear how your pre-season guesses compared with our model.
The next level of complexity is to account for the conferences and divisions, which do much to constrain the possible number of games won and lost. That is, each team’s schedule should be part of a better model. My guess, looking at the results of this simple model, is that incorporating this step will give a fairly decent boost to our predictive skill.
So, as the Lion’s always say (before each season begins), just wait ’till next year!
I also made this prediction: “the probability that at least one team wins 0 games is about 2%. The probability that at least one teams win 16 games is about 3%.” The two rare events had low probability, and no team won or lost all games.
1Note: these results are from a predictive and not from the more usual parametric model. You won’t discover how to calculate this model from ordinary statistics textbooks.
Using a predictive model gives us at least three advantages: (1) a full account of the variability of results (parametric models always under-predict the variability); (2) statements with respect to the observables (actual numbers of games won and lost); (3) probabilities for every possibility (each team can win from 0 to 16 games, and our model can quantify each possibility).
Uh, the Packers actually won 11 games
Interesting that out of 32 teams only two, the Dolphins and Raiders, exactly matched the predictive result. Woulda thunk that number should be 5>.
Kozak,
Oops. Typo; it’s fixed.
Another inconsequential correction (unless you’re a charger fan!): Chargers went 13-3.
Great blog, by the way.
49er,
What it shows is how important—or unimportant—a team’s previous record is. There’s something to the record, it’s not totally useless, but more information is necessary. Not surprisingly.
ChargerFan,
You won’t believe it, but I actually checked the list twice before I sent it out. Two mistakes so far!
Thanks.
Matt
Would a similar chart of the playoffs [as we now know them to be set] be useful or interesting? That is, a chart consisting of playoff result records [if any] of those specific teams during the same time span? I’m thinking even though the format is only a four game progressive elimination tournament, a predictive model could still provide valuable insight as to the Super Bowl’s final results.
OK, statistically accommodating the automatic playoff ‘byes’ might be a problem. At least for me.
The English football pools depended upon the idea that a draw. identical scores, in a match was a truly random event. Much like cavalrymen being kicked by horses and such like.
There are many variants but the most popular is to pick the eight matches with draws out of some 50 odd matches all played on the same day. The pool is just that with after costs and profits etc. of about 40% divided between the winners as in a sweepstake. I forget the exact proportions but even three draws out of the selected eight paid a small sum. Back then the full eight probably won you many hundred thousand pounds, literally millions today.
And like modern politicians they did love to call your stake your investment. And of course the winnings were tax free. Something of an incentive when income tax rose to a maximum level of 103% and even the poorest paid a few percent.
When I was young and could ‘borrow’ core time of about as much computing power as you have in your Blackberry nowadays I did a little analysis of this. True the incidence of a draw over many matches between two specified teams was and is as far as I could tell random, but the probability of a draw occurring in a match between those two specified teams is not compared to the other matches they might play with other teams: or indeed other teams might play.
In short though the incidence of a draw cannot be predicted the average chances of a draw do vary with the teams playing. Based on then some fifty years of results. Today, although they are a bit infra dig, the Pools companies have changed the scoring system to specify a no score draw, that is 0-0, as opposed to a score draw, 1-1, 2-2, and so forth.
An interesting adjustment designed to weight the odds against the player. As usual.
At least the lottery is truly random. And as far as I can tell impossible to fix.
Which doesn’t mean they don’t try to seduce with their marketing. The latest is the UK version of the Euromillions in which various countries participate so the maximum pay out could be up to 100 million pounds or so, if the Jackpot is not won it rolls over but there is a limit to the number of times that is allowed.
In the UK a ticket cost £1: 50 but now costs £2 with the extra 50p going to a separate draw guaranteed to pay the winner 1 million pounds. I won’t bother to elaborate the odds except to say the system is designed to produce only one winner and that infrequently.
Now what was it about those number houses in the US?
Kindest Regards.
You are trying to forecast the number of wins in a season. Most people try to forecast individual games against the point spread.
When looking at forecasting games against the point spread, IMO, the major problem encountered is in the data used to make the forecast. The teams won/lost record from past years has little to due with this year’s (or next year,s) team and its performance. The past game data (yards, pass attempts, yards per pass, time of possession, etc. etc. ) is most likely the RESULT of the score, not the CAUSE of the score in any given game. Therefore it is not much help in forecasting future scores. Even the scores of games can be misleading because the coaches don’t care about the margin of winning, they just care about winning.
There is a site (http://www.thepredictiontracker.com/prednfl.html) where people post weekly predictions, many of them computer generated, and there is no one that does consistently well.
Robert Burns:
I think Matt’s chart refutes your characterization. On average I think his program demonstrates a teams record is a significant but not conclusive factor in its present success. At least by my read.
Robert Burns,
49er is right. The teams’ past performance has something to do with predicting current performance; it is clearly of some help to know the old records. But you wouldn’t find it a lot of help in forecasting the next game’s point spread (per se; you’d have to build a different kind of model).
Briggs,
I think your claim,”It did capture a certain rough order in the standings, and it did better than the naive model which simply said ‘All teams will win 8 games’,” is false.
Taken together the charts show:
1) If I counted right, the Patriots moved from 1rst to a 4-way tie for 7th place. It’s rough only in the sense that the Patriots rank managed to stay below 10 and shows the unsurprising result that the top half wins more games than the bottom half.
2) the average number of expected games won for any randomly selected team is around 8.
3) the variability is too high to warrant its use for betting on rank. Two wins or losses out of 16 will radically change the standings. I realize you weren’t originally trying to predict rank.
I don’t think you model is significantly better than the naive one. It would comprise one of the lesser indicators if it has any value at all when it comes to rankings.
If you were trying to predict rankings, a better approach would be to develop a model that gives the probability of ranking. Your model suffers from insufficient data. My rough rule of thumb is to aim for getting a minimum count of 10 in each cell. Obviously hard to do for very low probability events (like winning all or none).
Robert Burns, “At least the lottery is truly random. And as far as I can tell impossible to fix.”
In its early days, it was discovered that the Pennsylvania state lottery had been rigged by the announcer. Apparently, he had been soaking some of the balls in water thus changing the probability of selection. He went to prison for it. Never underestimate the power of an insider.
That aside, how is changing the rules of a game the same as fixing it? Wouldn’t the change simply mean it’s a different game? If you go by your apparent definition of “fixed” all state lotteries have been “fixed”. For example in Maryland, the lowest takeout for Keno is 40% (the highest is around 75%) which makes it virtually impossible for player profit on the game.
49er
I was talking about about picking a winner in a game against the spread, and there is no evidence that previous years straight up won lost record is a predictor for an individual game versus the spread.
briggs
My point is that there are very few known statistical relationships (at least to me) to build a model to predict games against the point spread.
By the way, I checked your predictions against the Las Vegas win total lines and found that you agreed with the line on 7 teams (no bet), would have won with 11 teams and would have lost with 14 teams.
Robert Burns,
What about counting touchdowns and field goals per game? Not sure how the bookies go about it. It may not matter to them though if they can change the spread on the fly.
I guess that will teach Briggs to quit messing artound with the NFL!
DAV,
The exact skill of the model depends on the use to which it is put and the resulting gains and losses that would have been met had the model been used.
If it was used to predict the exact number of wins, then it would have very minor skill against the model which said, “All teams will win exactly 8 games.” If it was used to bet each team’s number of wins plus or minus one, then it did much better (not that anybody would have that as a goal: I use it to illustrate a point).
One of the goals of the model, as I said, was to examine how much importance there was in just examining a team’s prior record. As we can see, there is some usefulness, but not much.
Robert Burns,
Looks like I won’t be heading to Vegas to become a professional gambler.
49er,
I’m sticking to baseball in the future.