Skip to content

Category: Statistics

The general theory, methods, and philosophy of the Science of Guessing What Is.

January 9, 2008 | No comments

Now only 32.8% of people believe the polls: Did New Hampshire let us down?

Last week we looked at how the polls did in Iowa. How badly did the polls in the New Hampshire race do in yesterday’s Democrat primary? Here are the results:

Candidate Zogby WMUR Actual Error
Obama 42 39 37 +5
Edwards 17 16 17 0
Clinton 29 30 39 -10
Others 5 7 5 0
Undecided 7 8 0 +7
Huckabee 10 13 11 -1
Romney 27 26 32 -5
McCain 36 31 37 -1
Others 20 21 18 +2
Undecided 7 9 0 +7

We still tracked Zogby, but had to switch to the local CNN/WMUR/University of New Hampshire poll for the second comparison.

Last week, for Democrats, the total absolute summed error was 30 points. This week, it is only 22 points, and two candidates, Edwards and “Others”, were predicted perfectly, yet this week’s polls are widely thought to be poorer. Why is this?

The polls last week, while they did not do as well as this week’s in predicting the actual vote percentages, got the order of the top three candidates correct. For this week, the first two candidates were switched, but just barely; the margin of victory was only 2 points.

For Republicans, last week, the total error was 14 points. This week: 16 points! Both last and this week, however, the order of the top three was correctly predicted.

What matters most to people about poll accuracy is the candidate order; the actual predictions of vote percentages don’t seem too important, which is odd. Candidate order is supremely important in a two-person race, of course, where the only thing that counts is who is on top. But when there are multiple choices in a primary and multiple primaries, a good poll must also do well with the percentages—which the polls seem able to do. So don’t lose faith in the polls just yet.

All the polls were taken on the weekend before the primary, of course, and do not, and can not, account for what happens between the polls and the vote. For example, this past week saw Hillary’s breakdown, her interaction with the “Iron my shirt” guys, and Bill Clinton’s stage phone-whisper “I wuv you”, which all happened after the last poll was taken.

This only means that the plus or minus error you often hear reported (“This poll is accurate to within +/- 3 points”) is imperfect, and should in general be larger than given. These plus or minus points are actually theoretical results based on obscure math and many data assumptions, which rarely hold. So it’s far better to use an error rate based on the actual observed error of polls instead of the theoretical intervals.

The mean absolute (Zogby) poll error for the last two weeks, for both parties, was 4 points. This gives some hints that the theoretical error rates are too low and should be, say, doubled. I’ll post more on this after the next polls in Michigan and South Carolina come in.

January 8, 2008 | 2 Comments

Don Imus sidekick makes a point with a failed joke

Just heard Don Imus, who was attempting to explain why he wasn’t being sarcastic when he said “At least [George Bush] did something right” because we have not been attacked since 2001. One of his sidekicks quipped, with genuine sarcasm, that, “The Japanese haven’t attacked us since 1941, so he’s really doing something right.”

Of course, the last time the Japanese attacked us was in 1945, not 1941, but let’s not quibble about a few years. The real question is why the Japanese haven’t attacked us, or anybody else, since that time.

The answer is, it should be unnecessary to say, though I suppose it must be said, that it is because they attacked us in 1941.

This is a historical instance, one of a great multitude, of war working, of a successful democracy being installed by a Western power in another country. Don’t forget, too, that that war was not ended by “dialog” but by direct action.

It is therefore astonishing that so many now say that a “military solution in Iraq is impossible.”

January 7, 2008 | No comments

A safe, but misleading, prediction about global warming

Reuters’s, on 3 January, had this headline, “2008 to be in top 10 warmest years say forecasters.”

A quote:

2008 will be slightly cooler than recent years globally but will still be among the top 10 warmest years on record since 1850 and should not be seen as a sign global warming was on the wane, British forecasters said.

Where to start? First, the anemic forecast. Suppose the global mean temperature is, as predicted by several models, increasing, though this increase is subject to fluctuation from year to year, such that for one or two years the temperature might actually decrease, but in the long run, the temperature will still increase. Call this scenario the increasing temperature climate. It is important to emphasize that the increasing climate schema is consistent with both a significant man-made component and with the change being due to external causes. Now suppose, as has also been proposed, that the global mean temperature instead followed this cartoon, depicting a cyclically-changing climate:

Global mean temperature cartoon

This cartoon (and those based on other potential climate scenarios) is also consistent with either man-made climate changes or with the changes being due to external causes. It is just that, in this picture, the man-made component is harder to quantify. This is because of the trivial truth that man must influence the climate (see this), and that this influence will either be trivial or significant.

So it is a tautology to say that either man-made global warming is significant or it is not. One of those conditions must be true. It is also observed that the global mean temperature is correlated from year to year, so it is a fact that the temperature is somewhat constrained, in the sense that we will see little change from year to year, no matter whether mad-made global warming is significant or not, and regardless whether a general increasing or a cyclic climate holds.

Lastly, it is also the case that the global mean temperature has been increasing since the late 1990s until 2006. In 2007, the temperature decreased.

So the prediction that “2008 will be in the 10 ten warmest years” has an overwhelming probability of being true regardless whether man-made global warming is significant or not, and regardless whether an increasing or cyclic climate holds. That is, no matter what, this prediction is probably true, and it is useless as its intent was to give indirect evidence that the increasing climate scenario holds and that the man-made component of global warming is significant. It does neither such thing. Presenting this prediction as news is a clever debating tactic, but it is misleading, because the alternatives are not presented, even though the forecast is just as much evidence for them.

“‘The fact that 2008 is forecast to be cooler than any of the last seven years does not mean that global warming has gone away,’ said Phil Jones, director of climate research at UEA.” Jones is right, partially. But he forgot to say, “That fact the 2008 is forecast to be cooler than any of the last seven years also means that it might true that the climate is cyclic and that man has no real influence on it.”

This forecast is not additional evidence that an increasing climate or that man-made global warming holds.

| 1 Comment

Asinine comparison of holocaust deniers and honest scientists

Some guy named Joel Connelly at the Seattle Post-Intelligencer has written another one of those “We must do something now” global warming articles. What makes his piece distinct is that he compares those who express honest skepticism in global warming claims with those who deny the holocaust.

This sort of thing is exceedingly moronic, yes, but it is one more empirical observation that shows Godwin’s Law continues to hold. That law states:

As an online discussion grows longer, the probability of a comparison involving Nazis or Hitler approaches one.

And let’s not forget that exquisite corollary of Godwin’s Law, Benford’s Law of Controversy:

Passion is inversely proportional to the amount of information available.

Connelly’s piece follows Benford’s Law, too, because of the one thing that I find happening with increasing frequency. Here’s a quote:

Nobody is certain what will happen.

So far, however, change has proceeded at a faster pace than even pessimistic scientists predicted a few years ago.

Who could have forecast a 2-degree rise in Antarctica’s temperature in just 35 years? Nobody forecast the breaking off of giant Antarctic ice shelves. The rapid shrinking of the Arctic ice pack has surprised researchers…

Nobody is “certain what will happen”, except Connelly of course, because he then uses the empirical fact that scientists with their best models have failed to predict actual observations to argue that we should believe those scientists’ models and that “we must act quickly, within a decade.”

My friends, the opposite is true. If scientists are failing to predict actual observations, then we should have increased skepticism that what those scientists are saying will actually come to pass. At the very least, we should increase the error bounds, the “plus or minuses”, that accompany their predictions.

It also does not follow that because the model error is negative, i.e. that more warming took place than was predicted, that the situation is even worse than we thought, and that even faster warming rates will occur. This argument is a logical fallacy unless it is conjoined with the additional premiss that “warming must occur, and that if our predictions show that less warming will occur than actually does happen, then even greater warming will take place.” But if you assume that premiss, then your argument is circular, and therefore useless.

Failed predictions should not lead to increased passionate belief in those predictions.