Update: 2 March 2009. I am withdrawing this post, in the sense that I no longer think it is strictly accurate. See this post to see why. But I want to leave this post up for others to see how not to do statistics.

Background

Right after the close of the Republican and Democrat conventions (in 2008), I asked readers to participate in a study where they could guess who would win the presidential election. They could also indicate who they wanted to win, and if they ordinarily skewed Conservative or Liberal.

The idea was to search for traces of wishcasting, which is what happens when people let their desires influence their judgment about what will happen.

A wishcasting sales manager might suppose his will make a higher sales number next month than he probably will because he wants to save his job. A climate activist might forecast a higher probability of doom because he wants mankind to be responsible for various real and imagined ills. A sports fan will similarly put too much weight on a victory for his favorite team.

Or a voter might guess too high a probability for his candidate’s victory given that he desires him to win.

It is important to recognize wishcasting traits in yourself—or in your company—so that you can reduce or eliminate them and thus produce more accurate, and thus more valuable, forecasts. Recognizing wishcasting in your forecasts is the first step to making better predictions.

Results

Please remember that these guesses were made right after the conventions, well before the end-game of the campaign. At that point, the information on the candidates was roughly equal. Both media reports and personal conversations indicated that both candidates had a chance to win. That would obviously change later in the campaign, but a crude parity was in place after the conventions.

We received 624 legitimate responses (see the Data Caveats section for more details). 79% thought McCain would win, 21% Obama. Obama won 53% of the popular vote, McCain 46%. Given the information they had at the time, voters guessed too high for McCain and too low for Obama.

Not surprisingly, given the nature of this blog, 83% wanted McCain to win, 15% wanted Obama, and 2% chose not say. This was the same breakdown for philosophy: 83% reported Conservative, 15% Liberal, and 2% unknown.

80% of the participants were men. The median age was 51 (same for men and women), with a range between 16 and 89.

The tricky part of this analysis is that it is impossible to say exactly how much wishcasting was done. It might be true that each of the respondents did not let their desires influence their guess at all. Given what we know of human nature, this is probably false, but we cannot say with certainty. Everything below, therefore, is just an estimate.

We also cannot say about the level of wishcasting in any person: all estimates can only be for the “average” voter, that is, the type of voter who would take part in an internet study like this. If we had repeated guesses from one person, then we could hone in on his wishcasting component, but we only have one guess per person, so we can only say something about the group.

This table gives a first indication. It estimates the probability of who wins given who you wanted to win:

Of those who wanted McCain to win, 89% thought he would win, and only 11% thought Obama would. Likewise, of those who wanted Obama to win, 75% thought he would. Of the small number who didn’t express a desire, two-thirds thought McCain would win. That 89% and 75% are pretty big and indicate that some kind of wishcasting has taken place. Why?

If there were no wishcasting, and assuming the 600-some people’s actual votes did not influence the election unduly (presumably everybody who wanted a candidate to win voted for him), we would expect that who you wanted to win would have no bearing on who you thought would win. People should be able to separate their desire from their judgment.

Since overall 79% thought McCain would win, if no wishcasting took place, we would expect that 79% of those who wanted McCain to win to say he would. But 89% did. That 10-percentage point difference is the amount of wishcasting that took place among McCain supporters.

Since overall 21% thought Obama would win, then if no wishcasting took place, we would expect that 21% of those who wanted Obama to say he would. But 75% did. Thus, the 54-percentage point difference is the amount of wishcasting among his supporters.

For the people who didn’t say who they wanted, we would have expected a 50/50 split, but we saw a slight leaning towards McCain, 66/34. We shouldn’t read too much into this, as this was only for 9 people.

First finding

In other words, Obama supporters, as a group, wishcasted at a rate five times higher than McCain supporters. This result is not now surprising, given how the election played out, particularly in the media.

We cannot dismiss the element of doomcasting, either. This is when people guess what they don’t want to happen. Doomcasters have a counter-balancing effect on wishcasters, pulling the data back to where we would expect it had people not let their desires influence their guesses in a positive direction. Since we see a wishcasting effect, we cannot reliably estimate any doomcasting effect from the above table.

But we can say more. The next tables are just like the first, but broken up by those who identified themselves as Conservative or Liberal.

Estimated probability of who wins given who you want to win by philosophy:

For Conservatives, the amount of wishcasting for McCain was the same: a 10-percentage point bias. But for Obama supporters, the wishcasting fell 10-percentage points to only a 44-percentage point bias.

For Liberals, the amount of wishcasting for McCain was the same. But for Obama supporters, it increased a slight amount, to a 56-percentage point bias.

There were not enough people who did not specify a desired candidate to break down the numbers into philosophical groups.

Second finding

In other words, Liberal Obama supporters wishcasted at a rate higher than Conservative Obama supporters. The amount of McCain wishcasting remained the same regardless of philosophy, indicating more consistency. These results are also not surprising.

Third finding

If we break the data down by age into two groups (tables not shown): greater or lesser than the median 51 years, we find the McCain wishcasting bias remains the same, but for the older Obama supporters it goes up to a 65-percentage point bias. This was somewhat surprising given that we heard during the campaign that younger people who preferred Obama were more zealous. The younger group had just about the same percentage point bias as before (49 points).

Fourth finding

Among males, the McCain wishcasting bias stayed the same. But among females it increased just slightly to 13-percentage points. The amount of Obama wishcasting bias was the same for both males and females.

There was not enough data to reliably break the data down any further: for example, age by philosophy, sex by age, and so on.

Conclusion

Wishcasting almost surely took place in the McCain-Obama presidential election. This conclusion is conditional on the poll giving usable data (as to that, see the next section).

We must remember that these results are relevant for people who would come to a blog like this, during the specified time, and for elections like the one we had. At the least, we have said something about our small community; at the most, we have said something about average web browsers. We have probably not said much about the general population.

We should understand that any given Obama voter should not have necessarily subtracted 50-percentage points from his guess that Obama would win. That 50-percentage point bias was for the group and not necessarily for any individual. The implies that some people should have subtracted more, some less. The only way to say something about an individual is to collect more data on that individual so that we can estimate his typical bias.

Who wishcasted? Well, your author took part in the study: I thought McCain would win and also wanted him to. Wishcasting was probably there to some extent.

Other McCain supporters wishcasted, too, but not by very much on average. It didn’t seem to matter if they were young or old, male or female, or whether they identified themselves as Conservative or Liberal, the amount of McCain bias was about the same. Again, this was characteristic of readers of this blog, so we should be careful to say the same is true for all McCain supporters.

Those who wanted Obama to win really let that desire influence their guesses. Liberal Obama supporters wishcasted the most on average, a 56-percentage point bias; those who listed themselves as Conservative were more temperate, but not as tempered as McCain supporters. Sex didn’t make any difference to the results, but age did: older Obama supporters wishcasted more than younger ones did, a finding that goes against conventional wisdom. Once more, this result is for readers of this blog, or people like them.

Suppose you are, or were, an Obama supporter and you say, “So what if I say I wanted him to win. He did win, didn’t he? What does wishcasting have to do with anything? I made the right guess.” Yes, you did. This time.

The result—at the time of the conventions—was by no means a forgone conclusion. McCain might have won. You were making a guess about an uncertain future. You got it right this time, but you might not get it right next time. In making predictions of this type, if you are like the typical Obama supporter in this study, you are letting your desires influence your judgment too much, and over the course of many of guesses you will make more mistakes than the folks who do not wishcast. You’ll be losing either money, or prestige, or something.

There are probably plenty of formal studies, of which I am not aware, that show that the more controversial the matter the more people let their desires influence their predictions of the future. These findings do support the anecdotal perception that Obama supporters were more emotional than were McCain voters. It’s not that McCain supporters did not let their feelings influence them—again this data can say nothing about any individual, just the average behavior about a group of persons—but that influence was not sharp.

Obama supporters were, as everybody knows, more passionate (particularly read the next section). They let this passion sway them more. It worked out for them this time, but that passion can easily work against them the next time they make a prediction.

————————————————-
Data Caveats