Polls, Damn Polls, And Statistics

What, exactly, do all these presidential election forecasts do? And with just two weeks left, why are we still clicking them so compulsively?

Election forecasts are everywhere.

BuzzFeed News; Getty

It’s late October, which means — if you’re like us — you’ve probably developed a repetitive stress injury from constantly refreshing your favorite election-forecast website. FiveThirtyEight, the RealClearPolitics, New York Times, the Huffington Post, Talking Points Memo and more; every political site is trying to predict the 2016 presidential election.

Tracking them obsessively can feel like a way to cope with our election-related anxiety — though it probably does more to fuel that anxiety than to allay it.

Even now, when every site puts Hillary Clinton well in the lead, trying to sort out what these projections actually mean can produce a high degree of WTFness.

BuzzFeed News; Getty

As data reporters at BuzzFeed News, we are here to help.

BuzzFeed News

The sites all say they are forecasting the election, but they have different methodologies and, at times, substantially different results.

BuzzFeed News

Most forecasts base their projections largely on polls. Even among that group, however, there are different ways to weigh and adjust the responses and calculate the degree of uncertainty. And that's not the only way to predict an election. PredictWise relies not on polls but on betting markets, which can react much faster to big news events — or choose to ignore them completely.

However the different sites come up with these numbers, they all represent the odds that a candidate will win on Nov. 8, right?

BuzzFeed News; Getty

Not quite: they represent the odds that a candidate will win on November 8, given what we know today.

Let’s take Nate Silver’s FiveThirtyEight “polls-only” forecast. Today, it gives Clinton about an 87% chance of winning. That is: If we found ourselves in this situation — these same poll results, this many days before the election — one hundred times, Clinton would be elected president in 87 of those scenarios, and Trump in 13.

One hundred times? But we’re only going to do this once. (Thank goodness!)

BuzzFeed News

Yes, but it’s not the first time that the U.S. has elected a president. No previous election, obviously, has been exactly like this one, but forecasters can use past polling and election results to get a sense of how different situations played out.

But we haven’t had many elections with big-league polling data. Can we be that confident?

BuzzFeed News

It’s true: Silver’s forecast, for example, is “trained” on just 11 election cycles — 1972 through 2012. (Other use even fewer.) But something working in forecasters’ favor is that the U.S. election is actually a combination of lots of smaller, state-level elections. (Hello, Electoral College!) That gives forecasters’ algorithms a healthy chunk of data to crunch.

Why have the forecasts changed so much over the course of the election, even week to week or day to day?


Well the obvious reason is that people change their minds. More than you might think. In 2012, CNN’s exit polls found that 21% of voters made their decision in October or November. Nine percent of voters decided in the “last few days.” And even a small change in public opinion in a swing state can have a big impact on the overall election. Remember Florida in 2000?

Another reason: Polls only capture a small slice of voters.

BuzzFeed News; Getty

A big national phone poll might only have a couple thousand responses. These respondents aren’t perfectly representative of Americans overall, so pollsters have to find ways to extrapolate the initial results to the overall population of the United States.

But there’s no single, agreed-upon way to do that; last month, The New York Times gave the same raw data to four different pollsters, who each drew wildly different conclusions.

One more reason: $#!% happens.

BuzzFeed News; Getty

This election has already seen a few “October surprises,” and there could still be more to come. Polling data can't predict major scandals, international warfare, Beyoncé dropping an album, or any other event that might sway the election.

That said, the closer we get to the election, the harder it is for the underdog to catch up with the frontrunner, so most predictions will — all other things being equal — get more confident over time.

Still, no matter how the numbers stack up, it’s all just an educated guess until the votes are counted.

Which means we'll all still keep pointlessly clicking "refresh" on those polling sites, right?

BuzzFeed News; Getty