Skip to content

Category: Statistics

The general theory, methods, and philosophy of the Science of Guessing What Is.

January 5, 2008 | 1 Comment

How much does winning Iowa and New Hampshire help?

On the earlier poll thread, a reader asked “What’s your opinion about the statistical summary of past results of Iowa caucuses and New Hampshire primaries?” pointing out data from Neatorama.com. The Neatorama blogger had tables like these below, but made separate tables for Iowa and New Hampshire and included information about who won the general election. I thought it would tell a better story by combining the primaries and by eliminating the general election data because the main question is who will be nominated.

The tables list who won each of the primaries and who was eventually nominated. So, does winning the first two primaries help? Certainly.

Democrats

Year Won Iowa Won NH Nominated
1976 “Uncommitted” Jimmy Carter Jimmy Carter
1980 Jimmy Carter Jimmy Carter Jimmy Carter
1984 Walter Mondale Gary Hart Walter Mondale
1988 Dick Gephart Michael Dukakis Michael Dukakis
1992 Tom Harkin Paul Tsongas Bill Clinton
1996 Bill Clinton* Bill Clinton* Bill Clinton*
2000 Al Gore Al Gore Al Gore
2004 John Kerry John Kerry John Kerry
2008 Barack Obama ? ?
*Ran Unopposed

Republicans

Year Won Iowa Won NH Nominated
1976 Gerald Ford Gerald Ford Gerald Ford
1980 George H.W. Bush Ronald Reagan Ronald Reagan
1984 Ronald Reagan* Ronald Reagan* Ronald Reagan*
1988 Bob Dole George H.W. Bush George H.W. Bush
1992 George H.W. Bush* George H.W. Bush* George H.W. Bush*
1996 Bob Dole Pat Buchanan Bob Dole
2000 George W. Bush John McCain George W. Bush
2004 George W. Bush* George W. Bush* George W. Bush*
2008 Mike Huckabee ? ?
*Ran Unopposed

Combining data from both parties, there were 4 candidates who won both Iowa and NH, and in each of those 4 cases those candidates went on to receive the nomination. Of course, it is not guaranteed that winning both will secure the nomination, but it does make it very likely. This ignores those candidates who ran unopposed, as, obviously, their elections were never in doubt.

Candidates who won at least one of Iowa or NH won 11 out of 12 times, or 92% of the time. Only one time did a candidate not win at least one primary but still went on to win the nomination; that was Bill Clinton in 1992.

This implies the obvious: that by this coming Tuesday, we’ll be nearly sure who the top two candidates are for each party, and if either Obama or Huckabee wins New Hampshire, it’ll be a safe bet that they’ll also win the nomination.

You could separate out the data for each party, but there is no great reason to do so statistically.? The results also are conditional on the past “political situation”, which is largely unquantifiable.? If we assume that today’s politics are not different than those from 1976 until present, then the results are useful.? But if they have somehow changed—e.g. Guliani’s strategy of ignoring Iowa and NH—then these results are far less helpful.

January 4, 2008 | 2 Comments

73.2% of Likely Voters Believe Poll Results

Pollster: Who will you vote for? Civilian: Oh, I don’t know; that guy, the tall one. Pollster: I’ll put you down for Obama.

The elections in Iowa are over and the results known. It’s the season of the polls! More time is spent analyzing, worrying over, and speculating about polls in the media than is spent on any other subject, such as what the candidates think about the Iraqi war. In one sense, this is understandable, because polls offer hard, quantitative data which can be “crunched” and “drilled down” into, and so on. A candidate’s opinion on the war (and on every other subject) is harder to think about, mainly because the candidates themselves tend to be as vague as they can get away with to avoid in-depth analysis.

Since so much time and effort is spent on polls, we would hope that they offer some value. So how good are these polls? Let’s look.

There are dozens of these polls done by different organizations. The leading polls, in the sense that they are quoted the most and have the biggest organizations behind them, are the Zogby and the Des Moines Register (in Iowa only, of course), so we will examine just these two, though the results are not too different for the other polls.

Here is a table of the polls by the actual results for the Iowa 2008 caucuses. I used the latest polls, taken in the day or days right before the election, not the entrance polls. These are the numbers, then, that you would use to make a guess which candidate will win, place, and show. Only the top three candidates from each party are shown. All poll data was gathered from Pollster.com. The error is the Zogby poll minus the Actual result.

Candidate Zogby Register Actual Error
Obama 31 32 37.6 -6.6
Edwards 27 24 29.7 -2.7
Clinton 24 25 29.5 -5.5
Others 12 10 3.2 +8.8
Undecided 6 9 0 +6
—————–      
Huckabee 31 32 34.4 -3.4
Romney 25 26 25.3 -0.3
Thompson 11 9 13.4 -2.4
Others 26 27 26.9 -0.9
Undecided 7 6 0 +7

The most striking thing is, regardless of party, the polls for the top three candidates under-predict the actual results. The “Others” candidates are sums of the results over all the other candidates. There are only “Undecideds” at the time of the polls and none at the time of the election when, of course, people have to actually select an actual candidate. The error is a combination of the uncertainty of what the “Undecideds” will eventually do plus error inherent in the poll itself (through biased sampling and so on).

The much larger error for “Others” candidates for Democrats is in part due to the different way the Democrat caucus is run. If, in an initial vote at a particular polling location, a candidate does not reach a minimum threshold (about 15%), then the votes for that candidate are taken away and reallocated to other candidates. So a person might have told the pollster that he was for Biden, and gone in and voted for Biden, only to have that vote taken away and given to, say, Clinton (of course, it may be he who then chooses Clinton as his second).

One thing we can tell from the Democrat caucus is that not all of the votes for the “Others” (and “Undecideds”) were re-distributed to the other candidates evenly. At the time of the vote, 12 – 3.2 = 8.8% of the “Others” were redistributed. So, too, were the 6% of the “Undecideds”. That makes 8.8 + 6 = 14.8% of the votes that were redistributed (this figure also includes the native poll error). Obama got 6.6, Edwards 2.7, and Clinton 5.5 (these are the errors). Or, stating it another way, Obama got 45% of the eligible redistributed votes, Edwards 18%, and Clinton 37%, numbers which give hints about how future elections might go once the field of candidates narrows: many more people eventually opted for Obama than the other candidates.

49% of the “Undecideds” opted for Huckabee, 4% for Romney, 34% for Thompson, and 13% went to “Others”. This again might show that there is much stronger support for Huckabee and Thompson than is generally believed. Right now, the latest Zogby New Hampshire polls have Huckabee at 10%, with Undecideds at 8%; these numbers were taken before the Iowa results. McCain and Romney are a little over 30% each. So my guess is that by the time the votes are in from New Hampshire, it’ll be fairly even between McCain, Romney, and Huckabee, the results being in that order.
Of course, some of the error is due to the polls themselves, and, using error results from polls in previous presidential elections, I predict that we will see this error actually increase as the number of candidates shrinks. New Hampshire is less than a week away, so we’ll soon see, as two of the Democrat candidates have already dropped from the race.

December 30, 2007 | 1 Comment

Hurricanes have not increased: misuse of running means

Most statistics purporting to show that there has been an increase in hurricanes do not use the best statistical methods. I want to highlight one particular method that is often misused, and which can lead one to falsely conclude that trends (increasing or decreasing) are present when they actually are not. Read my original post to learn more about this.

That technique is the running mean. As you can see in the rather dramatic graphic from Science Daily, a 9-year running mean has been plotted over the actual hurricane numbers (up to 2005 only) in the North Atlantic. It looks like, in later years, a dramatic upswing is taking place, doesn’t it? This type of plot has shown up in many scientific, peer-reviewed papers.

Science Daily hurricane running mean

Don’t be turned off by the equations! Something very surprising is coming at the end of this article and you will be rewarded if you read to the end.

What is a running mean? A p-year running mean converts a series of actual, observed numbers into a statistical estimate, or model, of what a supposed underlying trend of those numbers might actually be. Because it is a model, its use must first be justified. In math, a 5-year running mean looks like

Running mean equation

where the symbol y indicates the hurricane numbers, and the subscripts t, t-1 and so on indicate the time period: time t is now, time t-1 was last year (now minus one year) and so on. The superscript on the symbol y to the left of the equal sign indicates that this is the modified data value plotted, and is not the actual number. Even if you’re afraid of math, this equation should be fairly easy to understand: the current, modified, number is just the mean of the last 4 observations and the most current one.

Additionally, in the mathematics of time series models, an auto-regressive series of order 5 is written like this

Autoregressive formula

which shows how the current data point is predicted by a weighted sum of past values, and where the weights are the coefficients ?. Just let all the ? = 1/5 and you have a similar running mean structure like that above. The point is this: using a running mean implies an underlying statistical time series model. Which is OK, as long as the data support such a model.

Do they for hurricanes? No.

In order to justify using auto-regressive time series models, you start by looking at something called an auto-correlation plot, which is a plot of how each year’s number of hurricanes is correlated with the previous year’s number, and how this year’s number of hurricanes is correlated with the number of hurricane from two years ago and so on: the number of previous years is called the lag. If any of these correlation lags are significant, then you can use an auto-regressive time series model for this data. If none of these correlations are significant, then you cannot.

Here is a picture of the auto-correlation of hurricane number (number of storms s) for the North Atlantic using data from 1966 to 2006.


None of the correlations reach above the horizontal dashed lines, which means that none are significant, and so a simple running mean should not be used to represent North Atlantic hurricane numbers.

So far, so good, right? Now let’s look at some made up, fictional data. Take a look at the following pictures, which are all simulated hurricane numbers; one of them looks pretty close to what the real data looks like. The running-mean even shows a healthy upward trend, no doubt due to global warming. But what do these pictures really show?

Simulated data with 9-year running mean

To get this data (the R code to make it yourself is pasted bellow), I simulated hurricane numbers (Poisson with mean 10) for pretend years 1966 to 2005, four separate times. Each year’s number is absolutely independent of each other year: to emphasize, these are totally random numbers with no relationship through time. I also over-plotted a 9-year running mean (red line). Because all of these numbers are independent of one another, what we should see is a flat line (with a mean of 10). The reason we do not is because of natural variation.

I only had to run this simulation once, but pay attention to the lower-right hand numbers, I got something that looks like the actual North Atlantic hurricane numbers. The 9-year running mean is over-emphasizing, to the eye, a trend that is not there! Actually, this happens to two of the other simulated series. Only one shows what would expect: a (sort of) straight line.

Like I said, I am including the code I used to make these plots so that, if you are curious, you will see how exceptionally easy this is to do.

Good statistical models are hard to do. See some of these posts for more discussion and to find some papers to download.

R code to make the plots. You must first have installed the package gregmisc.

library(gregmisc)
par(mfrow=c(2,2))
for (i in 1:4){
x=rpois(40,10)
plot(1966:2005,x,type='l',xlab="",ylab="",axes=F)
axis(1)
lines(1966:2005,running(x, width=9, pad=TRUE, fun=mean),lwd=2,col="#CC7711")
}
December 28, 2007 | No comments

Were the cannonballs on or off the road first?

There’s something of a controversy whether photographer Roger Fenton placed cannon balls in a road and then took pictures of them. He also took a picture of the same road cleared of cannon balls. Apparently, there is a question whether the cannon balls were ON the road when he got there, or possibly they were OFF and he placed them there to get a more dramatic photo. This drama unfolds at Errol Morris’s New York Times blog.

Whether they were first ON or OFF (Morris uses the capitals letters, so I will, too), excited considerable interest, with hundreds of people commenting one way or the other, each commenter offering some evidence to support his position.

Some people used the number (Morris uses the ‘#’ symbol) and position of the balls, others argued sun shadows, some had some words about gravity, and so on. Morris compiled the evidence used by both sides, ON (cannon balls on first) and OFF (cannon balls placed there by Fenton), and he presented this summary picture (go to his blog to see the full-sized image):

Morris cannonball evidence pic

This is an awful graph: the order of evidence types is arbitrary, it would have been better to list them in order of importance; the use of color is overwhelming and difficult to follow; and, worst of all, the two graphs are on an absolute scale. 288 people supported ON, and 153 OFF, so counting the absolute numbers and comparing them, as this picture does, is not fair. Of course the ON side, with almost twice as many people, will have higher counts in most of the bins. What’s needed is a percentage comparison.

Continue reading “Were the cannonballs on or off the road first?”