Update: 2 March 2009. I am withdrawing this post, in the sense that I no longer think it is strictly accurate. See this post to see why. But I want to leave this post up for others to see how not to do statistics.
Background
Right after the close of the Republican and Democrat conventions (in 2008), I asked readers to participate in a study where they could guess who would win the presidential election. They could also indicate who they wanted to win, and if they ordinarily skewed Conservative or Liberal.
The idea was to search for traces of wishcasting, which is what happens when people let their desires influence their judgment about what will happen.
A wishcasting sales manager might suppose his will make a higher sales number next month than he probably will because he wants to save his job. A climate activist might forecast a higher probability of doom because he wants mankind to be responsible for various real and imagined ills. A sports fan will similarly put too much weight on a victory for his favorite team.
Or a voter might guess too high a probability for his candidate’s victory given that he desires him to win.
It is important to recognize wishcasting traits in yourself—or in your company—so that you can reduce or eliminate them and thus produce more accurate, and thus more valuable, forecasts. Recognizing wishcasting in your forecasts is the first step to making better predictions.
Results
Please remember that these guesses were made right after the conventions, well before the end-game of the campaign. At that point, the information on the candidates was roughly equal. Both media reports and personal conversations indicated that both candidates had a chance to win. That would obviously change later in the campaign, but a crude parity was in place after the conventions.
We received 624 legitimate responses (see the Data Caveats section for more details). 79% thought McCain would win, 21% Obama. Obama won 53% of the popular vote, McCain 46%. Given the information they had at the time, voters guessed too high for McCain and too low for Obama.
Not surprisingly, given the nature of this blog, 83% wanted McCain to win, 15% wanted Obama, and 2% chose not say. This was the same breakdown for philosophy: 83% reported Conservative, 15% Liberal, and 2% unknown.
80% of the participants were men. The median age was 51 (same for men and women), with a range between 16 and 89.
The tricky part of this analysis is that it is impossible to say exactly how much wishcasting was done. It might be true that each of the respondents did not let their desires influence their guess at all. Given what we know of human nature, this is probably false, but we cannot say with certainty. Everything below, therefore, is just an estimate.
We also cannot say about the level of wishcasting in any person: all estimates can only be for the “average” voter, that is, the type of voter who would take part in an internet study like this. If we had repeated guesses from one person, then we could hone in on his wishcasting component, but we only have one guess per person, so we can only say something about the group.
This table gives a first indication. It estimates the probability of who wins given who you wanted to win:
Who win? | |||
McCain | Obama | ||
Want win? | McCain | 89% | 11% |
Obama | 25% | 75% | |
None | 67% | 33% |
Of those who wanted McCain to win, 89% thought he would win, and only 11% thought Obama would. Likewise, of those who wanted Obama to win, 75% thought he would. Of the small number who didn’t express a desire, two-thirds thought McCain would win. That 89% and 75% are pretty big and indicate that some kind of wishcasting has taken place. Why?
If there were no wishcasting, and assuming the 600-some people’s actual votes did not influence the election unduly (presumably everybody who wanted a candidate to win voted for him), we would expect that who you wanted to win would have no bearing on who you thought would win. People should be able to separate their desire from their judgment.
Since overall 79% thought McCain would win, if no wishcasting took place, we would expect that 79% of those who wanted McCain to win to say he would. But 89% did. That 10-percentage point difference is the amount of wishcasting that took place among McCain supporters.
Since overall 21% thought Obama would win, then if no wishcasting took place, we would expect that 21% of those who wanted Obama to say he would. But 75% did. Thus, the 54-percentage point difference is the amount of wishcasting among his supporters.
For the people who didn’t say who they wanted, we would have expected a 50/50 split, but we saw a slight leaning towards McCain, 66/34. We shouldn’t read too much into this, as this was only for 9 people.
First finding
In other words, Obama supporters, as a group, wishcasted at a rate five times higher than McCain supporters. This result is not now surprising, given how the election played out, particularly in the media.
We cannot dismiss the element of doomcasting, either. This is when people guess what they don’t want to happen. Doomcasters have a counter-balancing effect on wishcasters, pulling the data back to where we would expect it had people not let their desires influence their guesses in a positive direction. Since we see a wishcasting effect, we cannot reliably estimate any doomcasting effect from the above table.
But we can say more. The next tables are just like the first, but broken up by those who identified themselves as Conservative or Liberal.
Estimated probability of who wins given who you want to win by philosophy:
Conservative | Liberal | |||||||
Who win? | Who win? | |||||||
McCain | Obama | McCain | Obama | |||||
Want win? | McCain | 89% | 11% | Want win? | McCain | 89% | 11% | |
Obama | 35% | 65% | Obama | 23% | 77% |
For Conservatives, the amount of wishcasting for McCain was the same: a 10-percentage point bias. But for Obama supporters, the wishcasting fell 10-percentage points to only a 44-percentage point bias.
For Liberals, the amount of wishcasting for McCain was the same. But for Obama supporters, it increased a slight amount, to a 56-percentage point bias.
There were not enough people who did not specify a desired candidate to break down the numbers into philosophical groups.
Second finding
In other words, Liberal Obama supporters wishcasted at a rate higher than Conservative Obama supporters. The amount of McCain wishcasting remained the same regardless of philosophy, indicating more consistency. These results are also not surprising.
Third finding
If we break the data down by age into two groups (tables not shown): greater or lesser than the median 51 years, we find the McCain wishcasting bias remains the same, but for the older Obama supporters it goes up to a 65-percentage point bias. This was somewhat surprising given that we heard during the campaign that younger people who preferred Obama were more zealous. The younger group had just about the same percentage point bias as before (49 points).
Fourth finding
Among males, the McCain wishcasting bias stayed the same. But among females it increased just slightly to 13-percentage points. The amount of Obama wishcasting bias was the same for both males and females.
There was not enough data to reliably break the data down any further: for example, age by philosophy, sex by age, and so on.
Conclusion
Wishcasting almost surely took place in the McCain-Obama presidential election. This conclusion is conditional on the poll giving usable data (as to that, see the next section).
We must remember that these results are relevant for people who would come to a blog like this, during the specified time, and for elections like the one we had. At the least, we have said something about our small community; at the most, we have said something about average web browsers. We have probably not said much about the general population.
We should understand that any given Obama voter should not have necessarily subtracted 50-percentage points from his guess that Obama would win. That 50-percentage point bias was for the group and not necessarily for any individual. The implies that some people should have subtracted more, some less. The only way to say something about an individual is to collect more data on that individual so that we can estimate his typical bias.
Who wishcasted? Well, your author took part in the study: I thought McCain would win and also wanted him to. Wishcasting was probably there to some extent.
Other McCain supporters wishcasted, too, but not by very much on average. It didn’t seem to matter if they were young or old, male or female, or whether they identified themselves as Conservative or Liberal, the amount of McCain bias was about the same. Again, this was characteristic of readers of this blog, so we should be careful to say the same is true for all McCain supporters.
Those who wanted Obama to win really let that desire influence their guesses. Liberal Obama supporters wishcasted the most on average, a 56-percentage point bias; those who listed themselves as Conservative were more temperate, but not as tempered as McCain supporters. Sex didn’t make any difference to the results, but age did: older Obama supporters wishcasted more than younger ones did, a finding that goes against conventional wisdom. Once more, this result is for readers of this blog, or people like them.
Suppose you are, or were, an Obama supporter and you say, “So what if I say I wanted him to win. He did win, didn’t he? What does wishcasting have to do with anything? I made the right guess.” Yes, you did. This time.
The result—at the time of the conventions—was by no means a forgone conclusion. McCain might have won. You were making a guess about an uncertain future. You got it right this time, but you might not get it right next time. In making predictions of this type, if you are like the typical Obama supporter in this study, you are letting your desires influence your judgment too much, and over the course of many of guesses you will make more mistakes than the folks who do not wishcast. You’ll be losing either money, or prestige, or something.
There are probably plenty of formal studies, of which I am not aware, that show that the more controversial the matter the more people let their desires influence their predictions of the future. These findings do support the anecdotal perception that Obama supporters were more emotional than were McCain voters. It’s not that McCain supporters did not let their feelings influence them—again this data can say nothing about any individual, just the average behavior about a group of persons—but that influence was not sharp.
Obama supporters were, as everybody knows, more passionate (particularly read the next section). They let this passion sway them more. It worked out for them this time, but that passion can easily work against them the next time they make a prediction.
————————————————-
Data Caveats
All volunteer-based surveys run the risk of being as valuable as that which comes out the backside of a horse. You never know what you are going to get or who will try and trip you up. So it was with this survey.
In the hopes of getting clean results, I did announce, “Please do not try and stuff the ballot box by voting more than once. I will not release any results until after the election is over. (I will also remove duplicate records.)” However, there is no way to tell if or how many people lied.
I had about 600 entries, most of which went McCain’s way—not surprising given the traffic of this blog. When—suddenly!—a throng of Obama supporters showed up. I was pleased until I saw in this new data some Your Mama fans, along with a good chunk of Spiderman and Batman voters.
What had happened was a faithful reader asked the far left-wing writer of Pharyngula to post the poll. Which would have been okay, but the author called me “right wing”, which sent his impressionable readers into a tizzy. You can go and look for yourself, but they said things like:
Posted by: VegeBrain | September 9, 2008 10:34 AM
I decided to be just as honest as our current commander in thief and
voted in this poll in order to add fuel to the worst fears of
conservatives. So I voted like this:Who will win the election? Obama
Who do you want to win? Obama
Select your majority political opinion: Conservative
Select your sex: Female
What is your age (nearest year)? 62It is my honest hope that the conservatives stay awake at night
worrying about retirement age women in their ranks that want Obama to
win the election.If anyone here can think up an even better combination of choices to
make the Conservatives even more paranoid, please do so.
Posted by: Chris | September 9, 2008 10:12 AM
Well, I participated in the poll, and it is now INVALID, because I am
not American!BWAHAHAHAHAHA!
I love skewing nonsense polls!
Much was worse, despite my posting there and asking them to remain civil because this was an honest study, the spoiling of which would do no good because the results were not going to be released until after the election. My pleas were ignored or were abused.
I also goofed when I used an easily-spoofed HTML form where anybody could deviate from the planned responses. So these people, extraordinarily upset that people were not supporting their candidate, went nuts and injected a few thousand bogus entries into my database. There were many instances of the exact same entries from the same IPs dozens of times all in row, for example. Plus negative ages, enormous ages (one was 2147483647). Strange sexes, fictional political parties, and other junk.
Luckily, I knew the point at which the poll appeared on the radical site, and I was able to exclude all the data from that point on. This means, of course, that I had to throw out some undoubtedly legitimate data as well. I have learned my lesson about this kind of internet poll.
Great post.
For me, there are great applications in finance/finanical markets/
Geckko,
Amen, brother.
There might be something interesting in to find in that. Does liberalism, over time, actually make you irrationally optimistic that things will go your way (i.e., stupider)? And then irrationally depressed when they (as in ’00 and ’04) don’t? What about all those (mostly older, AFAIK) celebrities that said they were going to move out of the country in ’04 if the “unthinkable” happened? Anecdotally I have observed that the older liberal, the more irrational he (or she) will be… in the limit, utterly irrational (which is to say utterly illiberal, but that is another question). I mean I have an older relative who believed that Reagan was morally equivalent to Hitler, and yet who believes Clinton (most of whose policies differed little from Reagan’s) could practically walk on water. Do lifelong liberals end up with widely different neuroses than lifelong conservatives? Can we do twin studies??
People of similar political leaning cluster on a local level. This will greatly impact “who you think will win”. Take the members of my wife’s yoga group. They meet in midtown Memphis, an overwhelmingly “blue” area. But more than that, they also hang out with each other in midtown Memphis, bike past all the midtown Obama signs in people’s yards, all of their friends share a liberal political stance. Yes they all wanted Obama to win, but from their viewpoint how could they not also think he would win? People tend to think their peer group is representative of the populous. Day in and day out they would see and hear Obama, global warming, arts, coffee shops (not starbucks) and think everyone was on board with the democratic movement.
point 2:
“Since overall 79% thought McCain would win, if no wishcasting took place, we would expect that 79% of those who wanted McCain to win to say he would. But 89% did. That 10-percentage point difference is the amount of wishcasting that took place among McCain supporters.
Since overall 21% thought Obama would win, then if no wishcasting took place, we would expect that 21% of those who wanted Obama to say he would. But 75% did. Thus, the 54-percentage point difference is the amount of wishcasting among his supporters”
These numbers don’t really add up and here is why. You give a 54% bias to Obama wishcasting. By this calculation the HIGHEST possible amount of wishcasting McCain wanters could have had was 21% because it would have capped at 100%-79%.
Second, the main reason 79% overall thought McCain would win in your poll is because the majority of the people who took your survey wanted McCain to win, and 89% of those thought he would win. If you had an even representation of people who wanted Obama and McCain to win (50%-50%) the expected outcome would be the same… 89% of McCain Wanters think McCain will win and 75% of Obama Wanters think Obama will win. Both groups wishcasting 25-40%
If National (want) Polls are even, anything above 50% “who do you THINK will win” is wishcasting.
How to lie with statistics…..
here is another way to look at “Since overall 79% thought McCain would win, if no wishcasting took place, we would expect that 79% of those who wanted McCain to win to say he would.”
% who Want McCain = 79
% available to wishcast 100-79 = 21
wishcast increase 89-79 = 10
10/21 = ~48%
% who Want Obama = 21
% available to wishcast 100-21 = 79
wishcast increase 75-21 = 54
54/79 = ~68%
John,
Point 1 is the main point, yes. People should learn to look outside their insular worlds to judge better the future. Because they do not, they wishcast.
Point 2 isn’t quite right. Remember, the estimates of wishcasting bias are for groups and not necessarily for any individual. You’re right to point out, as I did, that these results are valid for the people that would come and participate in an study like I did. It could very well be true that the level of wishcasting is higher or lower across all citizens. There is no way to know whether higher or lower using the data we have.
Also remember that because the group-level is, say, 51%, this does not mean you can add or subtract that much from any poll or person’s guess.
How about providing a flat contingency table (gender x partyid x whowin x wantwin) with a grand total of 624 or 100%?
Fascinating! So many “conclusions” from so little data!
Jae,
Pretty good, eh? If I only had two more people, I could have doubled my findings!
JH,
You asked for it. Variable names should be obvious; bent = Philosophy.
table(whowin,wantwin,sex,bent,older)
, , sex = F, bent = Conservative, older = FALSE
wantwin
whowin McCain NULL Obama
McCain 27 0 1
Obama 4 0 1
, , sex = M, bent = Conservative, older = FALSE
wantwin
whowin McCain NULL Obama
McCain 157 5 5
Obama 18 1 7
, , sex = F, bent = Liberal, older = FALSE
wantwin
whowin McCain NULL Obama
McCain 2 0 6
Obama 0 0 10
, , sex = M, bent = Liberal, older = FALSE
wantwin
whowin McCain NULL Obama
McCain 9 0 8
Obama 1 0 29
, , sex = F, bent = NULL, older = FALSE
wantwin
whowin McCain NULL Obama
McCain 3 0 0
Obama 0 0 0
, , sex = M, bent = NULL, older = FALSE
wantwin
whowin McCain NULL Obama
McCain 3 0 0
Obama 0 2 0
, , sex = F, bent = Conservative, older = TRUE
wantwin
whowin McCain NULL Obama
McCain 57 0 0
Obama 4 0 0
, , sex = M, bent = Conservative, older = TRUE
wantwin
whowin McCain NULL Obama
McCain 198 0 1
Obama 27 0 5
, , sex = F, bent = Liberal, older = TRUE
wantwin
whowin McCain NULL Obama
McCain 0 0 0
Obama 0 0 9
, , sex = M, bent = Liberal, older = TRUE
wantwin
whowin McCain NULL Obama
McCain 5 0 3
Obama 1 0 10
, , sex = F, bent = NULL, older = TRUE
wantwin
whowin McCain NULL Obama
McCain 0 0 0
Obama 0 0 0
, , sex = M, bent = NULL, older = TRUE
wantwin
whowin McCain NULL Obama
McCain 3 1 0
Obama 0 0 1
This post is incredible. How can Briggs, who has statistics qualifications coming out of his ears, post such nonsense? Even Homer nods? Or is this just a spoof, which Briggs is using to test his readers?
I am sure that most people can see what is wrong but in case there is someone who does not I shall explain. Imagine if he had taken the poll on a site whose readers had a political outlook exactly opposite to the profile of readers of this site.
In this case Obama would have 83% of supporters and McCain only 15% of support with 2% undecided. For the sake of argument (and since Briggs is presumably making a general point about Obama supporters, not just those who visit this site) I shall assume their expectations of who would win had been in exactly the same proportion as the Obama and McCain supporters who responded to Briggs’s poll.
25% of the Obama supporters thought McCain would win – giving 20.8%
89% of the McCain supporters thought McCain would win – giving 13.4%
66% of the undecided thought McCain would win – giving 1.3%
75% of the Obama supporters thought Obama would win – giving 62.3%
11% of the McCain supporters thought Obama would win- giving 1.7%
33% of the undecided thought Obama would win – giving 0.7%
The combined results for the expectations of the readers of that Democratic leaning site would have been McCain 35.5% Obama 64.7%
Using Briggs’s Very Dubious Method
*Since overall 35.5% thought McCain would win, if no wishcasting took place, we would expect that 35.5% of those who wanted McCain to win to say he would. But 89% did. That 54.5-percentage point difference is the amount of wishcasting that took place among McCain supporters.
Since overall 64.7% thought Obama would win, then if no wishcasting took place, we would expect that 64.7% of those who wanted Obama to say he would. But 75% did. Thus, the 10.3-percentage point difference is the amount of wishcasting among his supporters.
In other words, McCain supporters, as a group, wishcasted at a rate five times higher than Obama supporters*.
This conclusion is no more correct than the one in Briggs’s “spoof?” post.
How can “who do you think is going to win” be answered by anything other than wishcasting or doomcasting unless you base your answer solely on known data including numbers of registered “likely” voters? If you base your answer on that information, the non biased answer should reflect the national polling averages at the time of the question. If you do not have such information available, any answer you give will be 100% doom/wishcasting.
P Hadley has it pegged. You designate the expected unbiased% of each group to the average of the survey takers without considering that the 79% McCain only exists because of the extreme representation of McCain supporters in that survey. The 79%/21% expected values have to be weight adjusted (normalized) to each group’s sample size.
Patrick,
Thanks for the post. First, as I repeatedly emphasized, the results are relevant for the type of McCain or Obama supporters who would come to this site and take part in a survey of this type, for the dates the survey ran, for an election of this type, and so on. I hope I have made clear that I was not necessarily talking about the general population or general McCain/Obama supporters. Nowhere, I hope, did I claim I was making a general point about either candidate’s support.
Now I’ll try to convince you that my statistics are accurate. First fix the votes at 79%/31% McCain/Obama. Now suppose, what did not happen, that everybody was indifferent between the two Candidates, and that their preference was independent of their guess who would win. (Let’s also ignore, for ease, the small number of undecided preferences).
Then we would see a table like this.
The percentages are the same as the forecasts as we’d expect. After all, the candidate preference is independent of the guess. Notice the these rows add to 100%
Now suppose that, as happened, 83% wanted McCain to win, and 17% wanted Obama (giving him the undecideds). But suppose again that these preferences were independent of the forecasts made. That is, the preference proportions remain the same, but they had no bearing on the guesses. Then we would get this table:
Which is, of course, exactly the same, and for the same reason (I’ll try and post a little R simulation code of this for you to check; that’s how I convinced myself). Here, the desire is independent of the guess, so the averages of the forecasts should be the same. Don’t forget that these tables are conditional ones. They show, given the desired candidate, the average of the guesses of who will win. Each row adds up to 100%. Now, the actual table was
I then compare this table with the expected one under independence to compute the wishcasting bias.
I realize that I did fail to point out that the wishcasting is not just in favor of a candidate, it is also in disfavor of the opposite candidate. So those who wanted McCain, as group, went too high, but they also went too low to Obama. And vice versa for Obama.
I believe what you are trying to do is to work with the unconditional table, where the probabilities of all cells add up 100%. What percentages were actually seen in each cell of the 2×2 table. Am I right? That’s what your example seems to show; the standard independence calculations.
In the actual data, the Obama/Obama cell was about 12%, with the “standard statistical independent” cell having about 4%—in which case you would claim that the Obama wishcasting would be about 8%, which is of course much lower than my guess. The original McCain/McCain cell was about 74% and the “independent” was about 66%; for a difference of also 8%. Maybe I’m wrong about what you were thinking, but is that close?
Then your thinking is not wrong, but you are answering a different question than the one I was answering. You were looking at an unconditional estimate of independence, valid for each cell. That is, as I think you are saying, the standard statistical answer.
But I was asking something conditionally relevant to a McCain or Obama supporter. The question is, given a person specified a desire, how much did it influence their guess? The proper way to look at this is conditionally, not unconditionally across all voters, but among a given candidates supporters.
I was trying to get a number which, if the people expressed their guesses in the form of a probability, they could use to adjust their guess. That is, suppose a McCain supporter said, “I think McCain will almost certainly win.” Then, on average, those people should say, “Taking into account my bias, I should say 100-10%=90%.” Again, however, I can’t really say anything about any individual—these are “on average” numbers only.
I think I did not make clear that the tables were not written in the way people who work with 2×2 tables expect to see them (in the form of unconditional probabilities). I tried to, but I should have been more explicit.
So, no, no spoof.
John,
Well, really. You can base in on other evidence: your knowledge of politics, your take of the world situation, your estimate of how the media will play out the race, and so on. You certainly can make a guess, and people do, all the time. You can also ignore or use the polls as you see fit, taking into account their biases and so forth.
As to your second comment, see my reply to Patrick.
Since overall 79% thought McCain would win, if no wishcasting took place, we would expect that 79% of those who wanted McCain to win to say he would. But 89% did. That 10-percentage point difference is the amount of wishcasting that took place among McCain supporters.
Since overall 21% thought Obama would win, then if no wishcasting took place, we would expect that 21% of those who wanted Obama to say he would. But 75% did. Thus, the 54-percentage point difference is the amount of wishcasting among his supporters.
There IS something fishy about that conclusion.
If people were truly giving an unbiased guess, we would expect a 50/50 split. Half would guess heads, half would guess tails. But 89% of McCain supporters guessed McCain. That’s a 39% deviation (bias). 75% of Obama supporters guessed Obama. That’s only a 25% deviation.
Another way to look at it: out of 10 McCain supporters we would expect five to unbiasedly guess he would win. Instead 9 made that guess. That means 4 of the 10 let their hopes cloud their judgment. Of the Obama supporters only 1 in 4 let their hopes affect their judgment.
Ergo liberals are not as dreamy and prone to wishcasting, in the sampled population.
One might conclude that if you are like the typical McCain supporter in this study, you are letting your desires influence your judgment too much, and over the course of many of guesses you will make more mistakes than the folks who do not wishcast. You’ll be losing either money, or prestige, or something.
In my judgment, those folks lost a whole lot more than that. We all did.
But then I was a McCain-supporting doomcaster, or should I say a realist, since I was correct in my judgments twice!!!!
You say Since we see a wishcasting effect, we cannot reliably estimate any doomcasting effect from the above table.
Of course, the opposite is equally true. What we can conclude, however, is that doomcasters are prescient if not highly intelligent, and hence we should give doomcasters all our money since they will know the wisest way to spend it.
I think the trouble is that Dr Briggs knows so much about complicated probability and statistics that he is unable on this occasion to see the obvious.
His responses to those who have questioned the conclusions have all ignored the basic problem. Namely that if you have a sample which splits heavily in preference towards one candidate then using his method the supporters of the less favoured candidate will always appear to have a much higher rate of wishcasting. Therefore the method is fundamentally flawed and no amount of further analysis can save it.
Ah, but I did save it, Patrick. And with vorpal good humor, as is the Statisticians’ Way.
Patrick H,
You can’t be surprised that if you invert the figures you invert the result! That’s obvious. The whole purpose of collecting data is that it is real data and not imaginary. Maybe somewhere in an identical universe there’s another Mr. Briggs, heaven help them, that did a study and counted the inverse numbers, but that’s hypothetical isn’t it?
The following quote,
“Since overall 79% thought McCain would win, if no wishcasting took place, we would expect that 79% of those who wanted McCain to win to say he would. But 89% did. That 10-percentage point difference is the amount of wishcasting that took place among McCain supporters.
Since overall 21% thought Obama would win, then if no wishcasting took place, we would expect that 21% of those who wanted Obama to say he would. But 75% did. Thus, the 54-percentage point difference is the amount of wishcasting among his supportersâ€
I found this very confusing, for some reason. Initially I couldn’t even work out what was confusing until the fifth read I spotted what I thought was a mistake. I then discovered that Briggs was right, as usual Damn and blast it.
Here’s where I went wrong, maybe where Patrick is confused, and what I was going to say before I realised,
“Surely this ought to say,
“Since overall 79% thought McCain would win, if no wishcasting took place, we would expect that 46% (McCain’s actual result) of those who wanted McCain to win to say he would. But 89% did. That 43-percentage point difference is the amount of wishcasting that took place among McCain supporters.
Since overall 21% thought Obama would win, then if no wishcasting took place, we would expect that 53% (Obama’s actual result) of those who wanted Obama to say he would. But 75% did. Thus, the 22-percentage point difference is the amount of wishcasting among his supportersâ€
Which changes the conclusion altogether. If that is a measure of wishcasting then McCain supporters were twice as likely to wishcast as Obama supporters.â€
Only one very small criticism, but because I have difficulty with shorthand, some of the sentences could have been written out in full, which would have made the post repetitive, not so pretty but ‘thickies’ like me would have understood first time round.
SM: said McCain would win. SO: said Obama would win.
WM: wanted McCain to win; WO: wanted Obama to win; WN: no response.
………………………….Want Win……………..
Say Win………..WM….WN…..WO…..RowTotal
SM………………..464………6……….24……….494
SO……………………55……..3……….72……….130
Column Total…519………9……..…96………624
Too lazy to write html codes.
I think this table might help with understanding the calculations of the percentages. They are fairly simple. Patrick H, I think you can do it as well as Briggs in this case. For example,
P(SM) = P(SM)=79% (494/624); P(SO)=21%(130/624)
P(SM|WM)=89% (464/519); P(SO|WO)=75% (72/96)
P(WM|SO)=42% (55/130); P(WO|SM)=5% (24/494)
Yes, given a reader wanted Obama to win (WO), there is a 75% chance that he said Obama would win (SO). It shows that WO and SO are not independent. Though I am not sure what it means to subtract 21% from 75%!
Note that of (given) those who said Obama would win, 42% wanted McCain to win. Of those who said McCain would win, only 5% wanted Obama to win. Assuming that “want win†and “party affiliation†are closely related, I would conclude that Republican readers are more likely to be delusional. Haha… I should run away as fast as I can, oh, i forgot, this is blogosphere, can’t run anywhere!
Briggs’s argument begins like this:
A. 79% expected McCain to win.
B. IF there were no wishcasting THEN you would expect 79% of those who wanted McCain to win to expect him to win.
So far so good. He has reported the results of his survey and made a logical deduction.
Where he goes wrong is his next step. The IF “X” THEN “Y” statement only has force when the “X” is True. If “X” is False then we have no information about whether or not “Y” is True or False, partly true, or anything at all. These IF…THEN arguments are used either when one knows that X is true and want make deductions from it, or when you want to test whether X is true by seeing whether contradictions follow from that assumption. What you can never do is use an IF “X” THEN “Y” when you know “X” is false and then draw on the information in “Y”.
For example I could argue:
IF 2 = 3 THEN 2 squared = 3 squared.
Therefore 4 = 9.
All I can get from this logical argument is that the false deduction (4=9) has shown that 2 NOT= 3, anything else deduced would be unreliable.
Because “There was no wishcasting” is False the B statement does not tell us how many of the McCain supporters were or were not wishcasting. Everything that follows using the 79% figure is therefore false. If there were no wishcasting then you could use the 79% figure to make futher deductions, because there was wishcasting then the 79% figure is completely irrelevant.
It is just as silly to use 79% figure that would be the expectation if there were no wishcasting and do calculations based on that even when there has been wishcasting, as to use 4 = 9 in later calculations when you know that 2 is not equal to 3.
I’m not even a statistician or a Bayesian. But I can tell the zoomie has hashed this thing out awfully.
1. He never refers to the tradesports markets and what those showed at the time.
2. You can’t indicate how much either group wishcast (or who wishcast more) based on the info given. Perhaps the objective best info bet was at 50-50 and both groups wishcast about equal. You can’t use the particular weighting that you had of respondants to tell who was wishcasting more. Imagine doing this same thing with Redskins and Giant fans. The amount that you weight for each (based on if you sample in DC or NY or how far in between) is irellevant to the degree of wishcasting of the groups themselves. You’re confusing the nature of your sampling with the nature of your groups.
3. You need to pillory yourself for awful wishcasting on McCain (including very close to the election).
>TCO,
what does the cheesy “zoomie?â€mean?
Patrick,
You are mistaken. if the symbol/shape “2†= “3†then “2 squared†= “3 squared†is true but it gives no indication of the actual value of “2†or “3†except to say that they are equal.
Just as, the statement “if x = y then x squared = y squared†is true.
This, however has nothing to do with the wishcasting problem; the industrial phrase is “non sequiturâ€.
At the time of the survey, 79% of participants were convinced with the information at their disposal, that McCain would win. As you agree, it is logical to assume that this should be reflected in a sample of the Obama, the McCain supporters, the Females, the Males, the Young and the old, if taken as separate groups. Of course this assumes that sex, age and political bent have no affect on a said groups’ ability to predict the outcome. i.e. the 79% is taken for the purpose of this argument as the standard opinion although it was noted that this sample is only representative of the type of people that †would take part in an internet study like thisâ€.
However, we know that different groups are probably more likely to be more inclined to wishcasting just as different groups will be more inclined to vote or behave a certain way. It does not surprise me that women wishcast more than men. I’m assuming it doesn’t surprise you guys either. Why is it painful for left wingers to accept that they are more idealistic creatures! This ought to be obvious. It is the idealism that leads to the wishfull thinking.
This information could be rendered even without the actual result as the 79% figure was contemporaneous to the time of the study. The final outcome seems to have no bearing on the amount of wishcasting that is attributed to the participants in this study.
So the actual outcome is not measured against the two sets of predictions from the two parties as I thought it would be to see which was better at predicting the outcome and then saying that any deviation could be attributed to wishcasting, doomsaying, bad information, etc.
So Patrick, Perhaps your problem with the 79% is because you know that the participants are not representative of general opinion across the entire population of voters. This is one of TCO’s problems.
If you only have so much data from a select (mostly pleasant) group, then you can only work with what you’ve got.
To attribute a fifty/fifty is to miss the fact that even at the time of the study there must have been some general overall consensus of likelihood of one or other candidate winning.
Briggs assumed that he set the study at a time when both candidates were evenly matched. Was this a safe assumption? Clearly not, because at the time, and just to be really repetative,
79% of the entire study population thought McCain Would win.
The sample is biased but the method is fine.
Joy, I am afraid your defence of Dr Briggs’s method does not work. In particular you are wrong because the 79% figure was based on the assumption of no wishcasting – not because of the particular sample that was taken. Briggs’s method will always be wrong irrespective of the sample.
Look again at his argument:
79% expected McCain to win.
IF there had been no wishcasting THEN 79% of Obama supporters would have expected McCain to win.
That, as far as it goes, is perfectly logical. However once we know that there was wishcasting then the 79% figure has no relevance. The 79% is simply the result of a false premise – namely that there was no wishcasting. It is as wrong to use the 79% in later calculations as it would be to use 4=9.
Briggs’s argument continues: Since only 25% of Obama supporters thought that McCain would win that means that 79%-25% = 54% of Obama supporters were wishcasting. But where does the 79% figure come from in that piece of arithmetic? It comes from the situation that would have existed if there had been no wishcasting. Since there was wishcasting then the 79% figure is totally irrelevant.
Imagine if Briggs had added two extra questions to his survey:
Q. Do you eat do-nuts?
Q. How much weight have you put on over the last 12 months?
He finds the average amount of weight increase was 6lb. He then says:
The average increase in weight was 6lb.
IF nobody in sample ate any do-nuts THEN the average amount of weight increase not caused by do-nut eating was 6lb.
Would it be a good idea to use this 6lb as some sort of control figure to calculate the amount of weight increase caused by do-nut eating? If 25% of the survey did not eat do-nuts and they had not put on any weight at all, but the other 75% had put on 8lb – would that be evidence to suggest that just 8lb – 6lb = 2lb of the weight increase was caused by eating do-nuts?
I am amazed that two days after this completely nonsensical post was made there are still some people who defend it, and that Dr Briggs has not had time to make a proper correction.
Patrick,
I wonder if Mr Briggs is regretting posting the damn thing!
Meanwhile, back with the 79% because I really think, although I now understand your point, that you are mistaken, here’s why:
624 was never a large enough sample even if it had been evenly split Obama:McCain. It is still a stretch to think that “wishcasting†or “wishful thinking†could be extracted from so few questions. I think fair apology was made for this in the post, clearly you do not. How can one measure known human behaviour let alone hidden thought processes. It is always going to be open to discussion.
What other method should have been used? 79% included good, bad predictions, whimsical ones, wishcasts, and all. If you could extract the wishcasters from this figure, you wouldn’t need to do the exercise in the first place because you’d have your answer! This is a circular argument. You must work with what you’ve got. You’re not suggesting that there was another way of gauging overall opinion? In a larger sample, like 1 million people scattered neatly across whatever area you chose to pick, you might have an easier time accepting that wishcasts from both sides would cancel out. To expect not to use the 79% because it is contaminated is to say that one shouldn’t have bothered in the first place. Which might be a fair point.
My own opinion, which will be very unpopular, is that collecting stats on human behaviour, is probably always a waste of time. The real world is not mathematical no matter that clever people want to describe it as such. It’s the male desire (and rarely the female one) to place everything in sequence, code, lines, boxes, and so on. Causes arguments like this one.
Applied mathematics must always be like this to some degree. I think we’ve squeezed as much as we can out of the lemon.
Joy, you ask what method should have been used.
The simple answer would be that what was needed were data about the expectation of who would win from a control group of people who were not committed to either candidate. If you could find how many people from a sample who did not care who won (but presumably were in other all respects similar to the Obama supporters interviewed) then the relative difference between this figure and the expectation of the Obama supporters would have indicated the level of wishcasting.
However I do not think that it is quite as simple as that because I have doubts about the whole concept of identifying and measuring wishcasting.
Many people believe that the mainstream media was biased to Obama, and was exaggerating his support. Assuming that this is true, it would have led some neutrals to believe that he was more likely to win than he was and therefore even the results from neutrals would have shown an element of wishcasting by proxy.
A neutral who lived in an Obama supporting house and was subject to a diet of propaganda in his favour is probably more likely to expect him to win than someone who only got all their information from a range of balanced opinons. Indeed to what extent is an Obama supporter who only talked to other supporters and read material in his favour actually wishcasting when they say that they think he will win? It does not really say anything about their internal thought processes, instead it reflects the inputs they have received.
In the same way a McCain supporter who listened all day to right wing Talk Radio and watched Fox News is likely to base his belief that McCain was going to win on what he had been told – is this really what is meant by wishcasting? In all probability an Obama supporter who was forced to listen to that stuff all day long would begin to doubt whether his candidate could win.
Supporters of a candidate tend to pay much more attention to material that is favour of him than to that which is opposed to him. When they are asked who is going to win they are bound to be influenced by that – so it will not be easy to distinguish between how much of the response is wishcasting and how much is reflecting the biased evidence the supporter has heard.
Patrick is more than correct, even incomplete at his incorrectness.
The main problem in Briggs method, (which even I, not a statistician, can clearly see) is that he conflates the two questions into the same one. When he asks, “who do you want to win”, he’s asking a desire. Because the object of desire is a democratic event, if everyone’s desires are all add up, they are exactly framing who will win.
This is obvious. But what Briggs forgot in his calculations is that his audience is by no means at all a good representation of voting electorate. Hence, what this audience desires to happen is completely independent of what really happens.
Hence, they cannot be judged in comparison, but only in Patrick’s way. The fact that 89% of McCain’s supporters thought that McCain would win is much more biased than the 75% of Obama’s. Face it, that’s the obvious truth to the matter.
Face it, you’re letting your judgement be clouded by your own wishes and desires. I sincerely hope you understand the criticism with a rational thought. And as last note, I also state that I lost some trust in your statistical fairness. Well, the world isn’t simple…
Patrick,
Wishcasting by proxy! I think there’s ointment for that.
Luis,
Harsh, as usual,
Your words,
But what Briggs forgot in his calculations is that his audience is by no means at all a good representation of voting electorate. Hence, what this audience desires to happen is completely independent of what really happens.
What part of this,
“The tricky part of this analysis is that it is impossible to say exactly how much wishcasting was done. It might be true that each of the respondents did not let their desires influence their guess at all. Given what we know of human nature, this is probably false, but we cannot say with certainty. Everything below, therefore, is just an estimate.“
Did you miss?
Did you miss the other part about this study only taking into account the types of people who would visit a website like this?
The sample was skewed from which the general prediction was gauged. Had there been an equal number of Obi and McCain supporters then the 79 may well have been lower. But due to the leftwing sabotage of the whole thing, many more Obi votes may have had to be thrown out. There’s irony!
So the statement about the results saying only something about the sort of people that would visit a website like this holds true. There was no claim made about the population as a whole.
It would also be true though to say that 89% of McCain supporters were wrong in their prediction.
75% of Obi fans were correct in their prediction.
That says nothing about whether their prediction was wholly wishful thinking or completely dispassionate (which lets face it is unlikely in an Obama supporter.)
So, the 10% over the top for McCain and 54% for Obama was how far each side deviated from the general view. This bias may have been due to numerous factors, but it is reasonable to assume that a preportion of both figures was due to wishful thinking.
Again, shame that the left wing monkeys from the site that begins with F (I can’t remember it’s name) didn’t vote honestly, this post and results may have looked a whole lot different with some honest Democrat participation.
Twas ever thus.
Patrick,
Sorry,was just playing, but a neutral, by definition, has no wish to cast. They are indifferent.
So a wishcast is a prediction clouded by rose tinted spectacles. A bad prediction may come from wrong information or from over emphasis on certain types or sources of information about the subject. Wishcasting is just one element of a bad prediction. Your comment about being brainwashed or close minded due to local influences would be simple bias or prejudice.
We are all prejudiced, none more so than those who claim they are neutral.
I took part in this thing even though I am 100% English. I know that I wishcasted, or derived my prediction from some intangible hope that Obama wouldn’t be voted in. I couldn’t picture the kid in a suit being given such a powerful throne, so therefore I didn’t predict it would happen.
I can’t help thinking still that he was voted in because people were carried along on the wave of novelty at his colour. The romance, the sense of making ‘history’; but if colour is not important, why should it be historic! The day the Americans vote in a one armed blind woman with four teeth and talk about her intellect instead of her appearance, now THAT would be historic.
Hey, Joy, I wan some of the liquor you have been consuming while writing your comments on this post. Goodness! Obama was voted in because… his colour?
“I want”
Did you miss?
No, I didn’t. What you did miss was my entire point. Briggs’ method seems to me to be excellent given a true representation of the electorate. But since he got a 79% McCain, it was obvious from the start that this wasn’t the case (there was no national poll whatsoever, in any given time, that gave McCain such a figure). This very fact doesn’t impose a mere “caveat” in the analysis, but a fundamental flaw. Yes, academically, he is right, but not quite.
Lets see why. Just ask this question. Why has any one given supporter has his “wishcast figure” inversely correlated with the ammount of people that voted his own way? The only way this makes sense is if the group is a good representation. If it isn’t, it’s inherently biased against Obama’s supporters and for McCain supporters.
This is obvious.
Why?
1. Because the people that voted in the site aren’t thinking that the election depended upon the kind of people that voted in the site :), but rather that it depends upon a much wider population that follows its own rules.
2. Because the people that are making rational thinking don’t even know the current Briggs’ poll nor do they think it matters :);
3. If they don’t know Brigg’s poll, how are Obama’s supporters wishcasting more than McCain’s?
4. It doesn’t matter because they are only thinking in the national polls, which indicated a more 50/50% chance between them.
5. Therefore, “wishcasting” should be compared to that metric. That is, if we are to define wishcasting not a skew from Brigg’s poll, but a skew from the rational thinking of anyone that is aware of national polls.
6. Given that new reference, it’s obvious that the Obama’s supporters wishcasted less. Not that I care, I didn’t even enter the competition as such ;), but come on, this is the correct measure.
PS:
Yes, that inverse racist comment about Obama having won because of his colour is too amazing. Come on Joy, you can’t believe that. It’s as ridiculous as saying that Bush has won the two last mandates because half of the population is beyond dumb.
JH,
One of my dearest friends is a Labour (socialist) voter, so don’t be offended. As for the voting for colour, I do believe this was a factor. Next time round he will not have the “first†advantage;
No, no alcohol assisted comments. Not even chocolate assisted. Which is more than I can say for this great man, who had all the vices he admired:
“Socialism is a philosophy of failure, the creed of ignorance, and the gospel of envy, its inherent virtue is the equal sharing of misery.â€
Don’t blame me, blame Churchill.
And just to make you smile,
Snobby lady to Churchill,
“you’re drunk!â€
“Yes, miss, and in the morning I will be sober, but you will still be ugly.â€
Where’s that carrot “£$%^_^
Luis, “Manyana” sorry, physio bed-time.
Luis, Briggs’s method is flawed because it uses a false premise and then acts as if it were true. The make up of the sample is completely irrelevant. I have pointed this out several time before as clearly as I could. Why cannot anyone else see that? What more can I do? I thought my example with the do-nuts would have been enough. I shall try once more.
Put the case that the sample contained 50% Obama supporters and 50% McCain, and the proportion of expectations was the same as in Briggs’s survey .
This would give 89% of the 50% of respondents who were McCain supporters expecting him to win, while 25% of the 50% of respondents who were Obama would think that McCain was going to win – making McCain the expected winner according to 57%. 11% of the McCain 50% plus 75% of the Obama 50% gives him 43% of the expectations.
Briggs’s method says:
43% of the survey thought that Obama was going to win.
IF there had been no wishcasting THEN we would expect that 43% of those who supported Obama would expect that Obama would win.
But the whole point is that there was wishcasting, therefore we have absolutely no reason whatever to think that the 43% figure has any bearing on the situation.
Briggs would then use this 43% even though it was derived from a false premise and say: In fact 75% of the Obama supporters thought that McCain would win – which means that 75% – 43% = 28% of the Obama supporters were wishcasting. But this figure is utterly bogus and false because the 43% figure would only be true if there had been no wishcasting. You cannot take a figure that you know to be false and use it in a later line of reasoning. The only feasible way to get an idea of how much wishcasting is going on is to compare the expectations of the supporters with the expectation of neutrals.
I really cannot understand how anyone with more that a basic understanding of logic and probability could make this mistake. Perhaps it is a bit like the Monty Hall Problem where some accepted maths geniuses and (allegedly) 1,000 mathemticians with PhDs were bamboozled by a very elementary probability puzzle.
The cool thing about this study is that you can tell someone was wishcasting. Since the expectations differed markedly by Mc versus Ob groups. But you can’t really tell which group was wishcasting or how much.
Matt
Enjoyed the post.
Here downunder we have a concept called “push polling”, which seems to appear only at election time Don’t know what it is called on the topside but essentially it appears to be a modified form of wish casting which modifys a groups voting intention and gets it out into the electorate to modify overall voting intention.
I am not aware of published statistics on the success of the technique but you may have stumbled upon the technique to measure success. Please note voting is compolsiry down here so the outcomes may be different.
You have at your fingertips a wonderful consultancy – keep at it. Do you require the services of an agent??
Harry,
Voting is compulsory? I had no idea.
Nope, no agent, and none likely forthcoming after this episode!