Dear friends,
As I write this letter, the vote count is underway in yesterday’s U.S. presidential election. The race has turned out to be tight. In their final forecast last night, the political analysts at fivethirtyeight.com suggested an 89 percent chance that Joe Biden would win. What did that mean?
In repeated trials, such as dice rolls or cohorts of patients with potentially fatal illness, it’s easy to define the probability of a given event. We have a set of possible universes, and the probability is the fraction of those universes in which the event occurs. We can also ask if a set of probabilistic predictions is calibrated. If so, then out of all the events predicted to occur with an 89 percent chance, around 89 percent of them — neither many more nor many fewer — actually occur. We want our learning algorithms’ probabilistic outputs to be calibrated, and there is a body of literature on this topic.
But an election is a one-time event. What does a probability mean in this case?
When fivethirtyeight.com says that Biden has an 89 percent chance of winning, I mentally append the phrase “under a certain set of modeling assumptions made by the fivethirtyeight team.” The analysts made a set of assumptions under which they built a number of different universes — some that went for Biden, some Trump — and found that Biden won in 89 percent of them. It’s important to remember that these universes are artificial constructs built on the assumptions that Nate Silver and his team chose.
I find that organizations such as fivethirtyeight.com generally make reasonable assumptions. For example, one assumption might be that a state’s vote tally for a given candidate follows a Gaussian distribution, with mean and variance estimated from the polling data. Yet every model has flaws and fails to capture some effects. A model might assume that each state’s outcome is independent of other states — but what if there are pervasive problems with the postal service delivery of mail-in ballots, or systematic biases in polling that result in undercounting some demographics? That’s why, while I consider election polls to be useful, I don’t take their predictions at face value.
Even though every model is flawed, good ones allow us to understand the world better. No one knows with certainty if it will rain tomorrow, but my decision to carry an umbrella will differ depending on the probability. That’s why I use probabilities to quantify uncertainties when I make decisions.
I find that if you think in probabilities consistently, you’ll start to develop an intuitive feeling for what the numbers mean. When someone tells me something has an 89 percent chance of happening, I’ve heard similar statements enough times in enough different contexts to have an intuition for what might happen next.
Like many others, I stayed up late watching the election results trickle in, worried about the future of the U.S. and the potential global impact of this momentous election. Whatever the outcome, let us commit to keep on fighting for fairness, justice, and human decency, and to do our utmost to bring the greatest possible good to the greatest number of people.
Keep learning!
Andrew