The most enjoyable part of this project for me was implementing the NFL’s incredibly elaborate tie-breaking procedures in code: taking a plain English description of the rules (that requires some interpretation) and porting it to JavaScript.
We should be open-sourcing the simulator implementation (the model, but not the UI) soon so you can see how it works.
There was also a fun moment when I realized I could make the simulator 10x faster by computing only the seed of the selected team, rather than the full order of teams in the playoffs. This avoided a lot unnecessary sorting and tie-breaking!
This is a great project. Well executed, and looks like a blast to work on. I love the breakdown of what would be required for the Seahawks to miss the playoffs if they split their final two (it requires the Packers and Lions to tie their final game -- among other things).
This is awesome! Kudos to you and the team (I assume you had help?). Yeah, I'm sure the tie breaking procedures drove you crazy. Imagine the rules committee meetings when they discussed all of them. :) How did you manage the complexity of the rules in your mind? I'm curious as to how you went from rules => mental model/entities => code. Did you use a formal process?
Thanks, and yes, this was a collaboration with Kevin Quealy and Shan Carter.
The development process was incremental: first create a simple representation of the league (conferences, divisions, teams) and schedule (games, outcomes), then implement the necessary metrics (win percentage, strength-of-victory, etc.) and filters (e.g., head-to-head, common games), then finally the tie-breaking algorithm itself.
The incremental approach let me test the parts independently and reduce the amount of hair-pulling debugging unlikely scenarios. Though, there was still plenty of that… Fortunately the NFL has an official playoff computer I could use to test my implementation (for a single, complete scenario).
This is really cool, but using coin flips really colors the answers. For example, if you pick the Broncos to win out, then it says they have a 75% chance of being the #1 AFC seed. 75% is obviously the chance Patriots will lose one or both of their last games if the outcome is random, but that’s extremely misleading; the Patriots might lose, but they will be heavy favorites in both games.
Having an option to predict based on Elo ratings would be nice, but even using existing records to predict game outcomes would be a much better simulator. Additionally, betting spreads are available for many games too.
I get why this is hard, but it would be a fun addition to have a setting to change how the likelyhood of an outcome is computed.
Agreed. It’s more of a what-if machine than an attempt to make the most accurate prediction.
Incorporating Elo ratings into the simulator is actually easy: the difference between the two ratings tells you the probability of one team winning. We may do this in the future, but at least for now we opted for a simpler model whose main goal is to let you explore what’s possible.
For example, if you pick an outcome on one game that goes greatly against the ratings (e.g., the Seahawks lose to the Rams), maybe you have some reason to disagree with the ratings and you don’t want them to influence the simulated outcomes of other games.
I'm wondering at which week does your simulation become usable?
I've work on this problem as a side project before but with the goal of being able to make this kind of prediction from Week 1. I would be able to compute various probabilities about who would win Divisions, but figuring out anything outside of a division would become computationally prohibitive. In my attempt I used a probability distribution based on historical performance of the home team instead of giving each team an equal chance of winning.
Since they use a coin flip to determine who wins each game, running the simulation on week 1 would give each team an equal chance of making the playoffs.
Actually it would be interesting to know if the NFL rules / schedule give some teams advantages from the start.
If games are a coin flip, I believe the answer to your question is "no". Every team plays exactly 6 division games, 6 conference games, and 4 non-conference games. Those are the only non-points-scored, non-coin-toss variables in the playoff algorithm, so if no one has played a game yet, every team is identical.
I love this tool, have been using it obsessively on football sunday :)
Question (or suggestion): would it be possible to choose the desired outcome? By that I mean, my team (the Lions) is already in the playoffs, so all changes I make show 100's in all columns. But what I'm really trying to narrow down is whether they will win the division, or whether they will have home field advantage throughout. Perhaps once a team has met a particular goal (i.e. 100% chance of making the playoffs for example) the remaining games could be ranked vs. their percentage chance of reaching the next-higher goal?
Where did the number 62,000 come from? Are they eliminating a class of simulations that involve differentiating results of games between two teams that are both out of it? Wins by Jacksonville?
Never really thought of it as a 3^n problem before. Interesting. Thanks for this!
It’s just an arbitrary number that’s hopefully large enough that everyone sees approximately the same result. It would be nice to calculate the exact variance of our estimate, though. (An earlier version continued simulating indefinitely but we thought it was better to stop at a certain point.)
The most enjoyable part of this project for me was implementing the NFL’s incredibly elaborate tie-breaking procedures in code: taking a plain English description of the rules (that requires some interpretation) and porting it to JavaScript.
http://www.nfl.com/standings/tiebreakingprocedures
We should be open-sourcing the simulator implementation (the model, but not the UI) soon so you can see how it works.
There was also a fun moment when I realized I could make the simulator 10x faster by computing only the seed of the selected team, rather than the full order of teams in the playoffs. This avoided a lot unnecessary sorting and tie-breaking!