William M. Briggs

Statistician to the Stars!

On Nate Silver’s Republicans-Take-The-Senate Prediction

By law, statisticians must look like this.

Yet the first bringer of unwelcome news
Hath but a losing office, and his tongue
Sounds ever after as a sullen bell,
Remember’d tolling a departing friend.

Henry IV, part II

So there was Nate Silver, statistician par excellence, wearing the oak leaf cluster and crown of laurel, holding a purple slide rule, riding his chariot triumphantly through the blogosphere commemorating his famous victory over Uncertainty. He had predicted a high probability Barack Obama would be re-elected to the presidency.1

The Media loved him for his divination and showered him with much praise, honors, and gold.

But riding on the chariot with Silver was one of Uncertainty’s vanquished generals who whispered into Silver’s ear, “All glory is fleeting.”

Boy, was he right. And in spades.

Because Silver has again ventured forth into battle, facing his old enemy, but this time his augury is unwanted: “GOP Is Slight Favorite in Race for Senate Control.”

The same Media who once loved him is now hot on Silver’s tail, fangs bared, pitchforks and torches waving, for the crime of foreseeing unfavorable events. Business Insider says Democrats Are Freaking Out About Nate Silver’s Latest Prediction. The National Journal leads “Democrats to Nate Silver: You’re Wrong“.

Guy Cecil, executive director of the Democratic Senatorial Campaign Committee, is not happy and insists Silver is wrong. This judgment is itself a prediction, for there is no way for Cecil to know Silver is wrong, but it’s a happy one because it tells Democrat supporters what they wish to hear. But then, even if the GOP does not re-take the Senate, it doesn’t make sense to say that Silver was wrong.

Cecil said, “In fact, in August of 2012 Silver forecast a 61 percent likelihood that Republicans would pick up enough seats to claim the majority,” but the Democrats held. Again Cecil said Silver was wrong.

But Silver wasn’t (and can’t be) wrong because there isn’t any way a (non-extreme) probability forecast can be wrong.

All probability forecasts sound like this: “Given my evidence, the probability of Q is P”. As long as P is less than 1 and greater than 0, there is uncertainty whether Q, the proposition of interest, is true. For Silver, Q = “The GOP retakes the Senate.” His evidence is proprietary and his P isn’t explicitly stated (but note that it can be calculated from the table he gives here; 20 bonus points for the reader who does it).

To be wrong, Silver’s forecast has to say P = 1 and the Democrats must retain control. There is no other way to err. If Silver’s P = 0.99 (and it isn’t), and the Dems keep regulating our lives, then Silver would still not be wrong.

There is a sense, though, that Silver’s prediction, in the light of the cold reality of the Dems holding power, might be seen as less than useful (we are imagining a future in which the GOP loses). This sense highlights the very real difference between a prediction and a decision. We’ve seen what a prediction is. A decision takes a prediction and acts on it. Decisions can be wrong. Non-extreme probability predictions cannot be.

One decision might be to bet that the GOP takes it. If the Democrats win, you lose your bet, and you lose because your decision is wrong. The prediction remains a probability, a true statement of the evidence used to create it.

Not everybody will use Silver’s prediction to bet. Why? Because some people don’t like to bet, or others like to but don’t see much pay off, or because a prediction which is just the other side of a coin flip doesn’t instill enough courage to gamble. But others might love to have a go and will plunk down lots, whether in terms of real money or in reputation points (a pundit might say “The GOP is gonna take it all!”).

Thus a prediction which is useful for one person can be of no use to another. Decisions made on predictions are so varied that there’s no way to know who, if anybody, they might be useful for. (Though there are ways to look at collections of predictions and surmise what might happen if these predictions are used for future decisions of a known sort.)

It’s clear, though, that Cecil doesn’t feel Silver’s latest clairvoyance is useful for him. If people act on Silver’s prediction such that they cease donating to Democrat candidates, thinking these candidates will lose, those candidates deprived of money will be more likely to lose. So Cecil must do what he can do to plant doubts about Silver’s prediction—and about Silver himself—even though Silver is scarcely making a bold guess.

Luckily for Cecil, the Media is ready to shoot the messenger for him.

———————————————————————–

1Many statisticians of lesser repute, such as Yours Truly, have done much worse.

10 Comments

  1. Briggs

    March 25, 2014 at 1:34 pm

    Boy, I must have done a good job with this one. Not even one criticism!

  2. I get cramps in my arm when I do that.

    I guess a 70% chance is slight.

  3. Briggs

    March 25, 2014 at 2:54 pm

    DAV,

    70% is it? Hmm. I didn’t check, but that seems high. Might be right, though.

    The GOP has 30 seats going in and there are 36 races (all listed in that table). They have to win 21 of these; any 21. What is the chance of winning at least 21 of these? That’s the same as the probability that the GOP takes the Senate.

    That’s just the probability of winning the first 21 and losing the last 13 plus the probability of losing the first and winning the next 21 and losing the last 12 plus etc., etc. Sum all the probabilities in scenarios where the GOP wins 21 or more.

    There are choose(34,21) = 927983760 ways of winning 21 seats, choose(34,22) = 548354040 ways of winning 22, and so on up to 1 way of winning at 34. That’s a lot of ways!

    A crude, cheap, cheesy approximation is to suppose the chance of winning any seat is the average of all those probabilities in his table. I make the average 0.578. That gives the chance of a GOP Senate as 0.39. Which isn’t too high.

    In order to push the probability of a GOP win over 50%, the average of that table has to be greater than 0.6.

    But this is only an approximation, as I said. To do it write the actual probabilities have to be calculated. And I’m too lazy to do it.

  4. Briggs,

    Yeah the 70% chance is way too high.

    Turns out the top 21 chances for a GOP start at the 50% mark. As a quick and dirty, assume all above that mark will be wins leaving one seat at 50% gives a 50% chance of winning that seat.

    To win all of the top 21 leaners (counting the 50% in the GOP favor) and lose the rest, which would be the highest product of all the terms, yields 5%.

    If we assume those at or below 5% and those at or above 95% are 0% and 100% respectively then there are 16 swing seats and they need half of those (there are 13 at of above 95%) which is a smaller problem.

    If we assume a win in the top 8 of the swings we get a product of winning the top 21 seats at 6%.

    These look like the scenarios with the best chances and they don’t look good.

  5. I acknowledge that as a non-consumer of most US media, I don’t know what you’re describing as Silver being showered with honour, praise, etc. regarding the Obama prediction, so I don’t really know what you’re comparing it to. Maybe that’s why I really don’t see how media reporting that ‘Democrats are turning on Nate Silver’ is the same thing as the media themselves turning on Nate Silver.

  6. Briggs, DAV, why are you treating the individual seat outcomes as independent?

  7. If they aren’t independent, there is nothing in the table to support it one way or the other. In general, it seems improbable that people in say Rhode Island and Texas base their voting on how each other have voted.

  8. Briggs

    March 26, 2014 at 11:24 am

    Johnathan D,

    DAV’s right. The only information we have is that table. Given just that, we can calculate the probability of a GOP victory. Like all (logical) probability, this is conditional on the information expressly supplied.

    Besides, “independent” is a word I don’t like. It implies knowledge of causality, which we don’t have.

  9. Briggs,

  10. Briggs, I take it you mean “implies knowledge of causality” in the connotational sense, rather than the logically implies sense. What is your argument that the information expressly supplied (the table) justifies a calculation that assumes the probability of a particular outcome is the product of the probabilities in individual states. Why not say the table is not enough in itself to calculate the probability of Rep control?

    Apart from that, we know that Silver is using more than is presented in the table. In particular, we know his assumptions are inconsistent with the property usually called independence. So why should anyone be interested in the probability given the table and ‘indepedence’ (regardless of whether ‘indepedence’ needs to be assumed or follows from lack of extra knowledge)?

    DAV, surely you’re not suggesting that Rhode Islanders and Texans being influenced by each other’s votes is the only possible explanation for a lack of independence?

Leave a Reply

Your email address will not be published.

*

© 2016 William M. Briggs

Theme by Anders NorenUp ↑