Skip to content

Category: Statistics

The general theory, methods, and philosophy of the Science of Guessing What Is.

September 2, 2008 | 11 Comments


Around the 4th of July, here in the States, there is a tendency for official weather forecasts to show a probability of precipitation that is lower than it should be. It rains more than the forecasters guess.

The same thing inverted happens around December 25th (the Federally Recognized Holiday That Shall Not Be Named): the forecasts tend to give too high a probability of precipitation. It snows less than the forecasters guess.

This phenomena is well recognized in meteorology where it has long gone by the name of wishcasting: it is also found in many other areas of life, which I’ll talk about below. Wishcasting describes the tendency of the forecaster to tilt his guess toward the outcome which he would like to see, or toward the outcome he knows his viewers would like to see.

Good weather forecasters, obviously, are aware of this tendency and do their best to lessen its influence. But even the best of them tend to get excited when a big storm is on its way, these being matters of great and evident importance, and sometimes issue forecasts which exaggerate the chance of severe weather. Still, the influence of wishcasting is small among professionals, mostly because of the routine evaluation of forecast performance and criticism of peers. People like to pick on weather forecasters, but among any professional group, I have not found any to be better or more reliable than the National Weather Service.

Before we go further, let me answer an objection which might have occurred to you. Why not exaggerate the probability of a storm causing damage since “it’s better to be safe than sorry”? To do this takes the decision out of the hands of person who will experience the storm and puts it into the hands of the forecaster. And that is the wrong thing to do: the forecaster does not know better than his audience what decisions are best. Every person in the path of a storm knows what losses he will face if a storm hits, and how much it will cost him to protect. If people are routinely given exaggerated forecasts, then they will pay the cost of protecting more than they should, and those costs are not insignificant (how much money is being lost by the shops of New Orleans from the protracted evacuation?). You cannot use the forecast as a tool to warn people of dangers which are unimportant to them. It will make them less likely to believe forecasters when real dangers arise. The lesson of Chicken Little is pertinent.

While the Weather Service forecasters do a great job, this is not so among reporters who routinely wildly overstate potential dangers, even when that danger has passed. Anybody who watched television coverage of hurricane Gustav could attest to this. We saw fearless reporter Geraldo Rivera standing in the streets of New Orleans holding a small aneomometer shouting, “There’s a 60 miles per hour, Bob! Wait! A 61!” He bravely leaned into the stiff breeze and held his ground to bring us this breaking news. Of course, anybody who has driven a car and stuck their hand out the window will know that a 60 MPH wind is hardly life threatening.

Well, reporters shading the truth, embroidering facts, neglecting pertinent information, and at times outright lying is by now of no surprise. People have learned to “divide by 10” any statement issued from a newsroom, so journalists cause less harm than they would if they were taken at face value.

Wishcasting is by no means restricted to weather predictions. I’ll ask you right now, who will be elected president: McCain or Obama? It is difficult to remove the prejudices you have for one candidate or the other and give a good guess. If you love McCain, you are likely to increase the chance of him winning. If you fear Obama’s promised tax increases, that might increase your guess of the chance of him winning if you are naturally pessimistic. To carefully sift through all the evidence and arrive at an unemotional prediction is extremely difficult.

Gamblers often wishcast. “Red hasn’t come up if seven spins, so it’s more likely to now.” Part of this reasoning is due to misunderstanding or not knowing the rules of probability that govern simple games, but part is also due to the desire for the outcome. Wishcasting is prevalent in environmental circles. So much so, that an “activist” who doesn’t embellish is a oddity. Brokers, financial planners, stock pickers, and similar professionals are no less prone to wishcasting.

Wishcasting is somewhat different than the experimenter effect, although there is some overlap. The experimenter effect is when a scientist (or group of them), consciously or not, set up a model to demonstrate the effect they were looking for. A common example is a drug trial. One group is given a new drug, the other an old one or a placebo. If the patients are evaluated by a physician who knows which patient got which drug, it is likely the effects of the new drug will be exaggerated. This phenomena is so well known that the government mandates blinding of medical trials. This is where the physician who evaluates the patients has no idea which treatment the patient has received.

Michael Crichton, physician and author, in testimony to congress, gave an example of this:

It’s 1991, I am flying home from Germany, sitting next to a man who is almost in tears, he is so upset. He’s a physician involved in an FDA study of a new drug. It’s a double-blind study involving four separate teams—one plans the study, another administers the drug to patients, a third assesses the effect on patients, and a fourth analyzes results. The teams do not know each other, and are prohibited from personal contact of any sort, on peril of contaminating the results. This man had been sitting in the Frankfurt airport, innocently chatting with another man, when they discovered to their mutual horror they are on two different teams studying the same drug. They were required to report their encounter to the FDA. And my companion was now waiting to see if the FDA would declare their multi-year, multi-million dollar study invalid because of this chance contact.

His point in this testimony was to show that researchers in global warming are nowhere near as careful as their colleagues in medicine:

[T]he protocols of climate science appear considerably more relaxed. In climate science, it’s permissible for raw data to be “touched,” or modified, by many hands. Gaps in temperature and proxy records are filled in. Suspect values are deleted because a scientist deems them erroneous. A researcher may elect to use parts of existing records, ignoring other parts. But the fact that the data has been modified in so many ways inevitably raises the question of whether the results of a given study are wholly or partially caused by the modifications themselves…

…[A]ny study where a single team plans the research, carries it out, supervises the analysis, and writes their own final report, carries a very high risk of undetected bias. That risk, for example, would automatically preclude the validity of the results of a similarly structured study that tested the efficacy of a drug.

Wishcasting meets the experimenter effect when the results from a non-blinded experiment are exaggerated to “raise awareness” of the potential horrors that await us if we do not heed the experimenters. Sometimes this exaggeration is done on purpose, as with the weather forecaster who feels his viewers would be “better safe than sorry”, and sometimes the overstatement is unconscious because the forecaster has not recognized his limitations. Scientists often feel they are special and able to avoid the frailties that plague the rest of us, but of course, they cannot; they are still human.

It is nearly impossible to disentangle experimenter effect from wishcasting in any situation, nor can we easily identify the constituent facts and their relevance used by a forecaster in producing his forecast. To do so essentially means producing a rival forecast and is a laborious process.

What we can do (this is my line of country) is to check how good the actual performance of a forecast is. If the forecast routinely fails, we can say something has gone wrong. Just what requires more work: was it bad data, mistaken theory, wishcasting, or something else? If the forecast routinely fails, we are rational to suspect it will fail in the future, and that the theories said to underly the forecast might be false. If the forecast fails, we are also right to question the motives of the forecaster, because it is these motives that influence the presence or amount of wishcasting.

These cautions do not just apply to weather or climate forecasts, but in all areas where routine predictions are made. Could you be making more money in your stock portfolio or office football pool, for example? Generally, wishcasting takes places when forecasting complex systems, like the weather, climate, or any area involving human behavior. It’s much less likely in simple situations, like how much this electron will move under a certain applied force, or what will happen when these two chemicals are mixed. But we’ll save complexity for another day.

September 1, 2008 | 8 Comments

Gustav to spoil Moore’s fun

Gustav is weakening and steering to the west of the Big Easy.

The Air Force regularly sends in its “hurricane hunter” aircraft, and the last observation of about a hour ago showed central winds “generously described” at 100 knots, and those probably falling. The National Hurricane Center reports that the AF plane didn’t even find an eye wall, though radar reports an open (therefore weaker one) over the south.

The forecasters also report “WATER VAPOR IMAGERY ALSO SUGGESTS A DRY INTRUSION AND A RESTRICTION OF THE UPPER-LEVEL OUTFLOW OVER THE SOUTHERN PORTION OF THE HURRICANE.” What that means in English is that the storm will very probably get weaker. (We—I used to be a National Weather Service forecaster—always had to type forecasts in all upper case. This is a throwback to teletypewriter days.)


Make no mistake, however. Gustav is still a big storm and those in his path should get out of his way. It will absolutely cause damage and cost a lot of people a lot of money.

But the wide-spread death and destruction so avidly hoped for by the radical left (see two posts ago) Michael Moore, Former Democratic National Committee Chairman Don Fowler, inter alia now seems unlikely.

What a sad day!

The weather also spoils the media’s fun. You could almost feel the anticipation and eagerness in the news rooms yesterday. Reporters irresponsibly echoed New Orleans’ mayor’s foolish statement calling Gustav the “storm of the century.” This kind of idiotic hyperbole, while typical, is easy to spot, so why does the media regularly give it so much play?

I just turned on the news and they showed a reporter, ridiculously overdressed in rain gear that would hold back a flood, giving his best effort. He shouted in the microphone, held his hat as the breeze and drizzle assaulted him. What a trooper.

August 31, 2008 | 10 Comments

On taking pleasure in the misery of others

Conan the Barbarian was asked “What is best in life.” Arnie’s response: “To crush your enemies, to see them driven before you, and to hear the lamentations of their women.”

Mike Royko, no conservative and man whose writing I admire deeply, talked of his love of John Wayne, the admiration then as now seen to be an embarrassment among the elite. Royko said that he was standing on the armrests of his chair and hollering encouragement when, in True Grit, one-eyed Wayne faced evil Ned Pepper and two of his henchmen across a glen and yelled “Fill your hand you son-of-a-bitch!”

Admiral William “Bull” Halsey said that his life “reached a climax” when the dejected Japanese foreign minister Mamoru Shigemitsu signed the treaty of surrender aboard the battleship Missouri, anchored in Tokyo Bay 1945.

All of these men, real or approximate, took pleasure in the defeat of their enemies, real or adopted. I was with you Mike. I still jump on the couch when I see Wayne’s shocked look at Pepper’s last taunt (“Pretty bold talk for a one-eyed fat man!”). I admit to feeling part of Halsey’s pride. I can understand the surge of joy when victory is gained over the bad guys. This, after all, has driven the entire career of the Governator.

So are these emotions any different than those espoused by anti-American movie-maker Michael Moore who recently said that because a major hurricane is soon to hit the American coast, there was “proof there is a God in Heaven”?

See, Moore doesn’t want the Republican convention to pass without interruption and he delights in the idea that external events might force a suspension or cancellation in the GOP’s gathering.

Like a lot of his ilk, though, Moore would have his victory at the expense of hundreds or thousands of innocents, something John Wayne or Halsey would never have countenanced. Moore must have figured out his comments were asinine because he later added he hoped “nobody gets hurt” when the hurricane hits. This is nonsense, of course, because it is only the deadly destructive power of the storm that causes it to be of note in the first place. Even if nobody is physically injured, the storm will still cost everybody in its path a packet. Only a fool or an idiot doesn’t realize this. Moore knew what he was wishing for. He knew what would happen.

Moore is a member of the “Victory at Any Cost” club of the left. Whatever has to be said or done in order to gain power is acceptable. Because once in power, these elites reason, all will be put right. Utopia is just around the corner! And nobody or nothing should be allowed to stand in their way.

August 30, 2008 | 52 Comments

The importance of Palin’s experience

Update. I’ve been browsing the blogs to find out about Palin. Most fascinating of all is that fact that everybody is comparing here to Obama and not to Biden. Like I said below, people seem confused about who is actually the presidential candidate. It’s either fear, or people are tacitly admitting Obama’s experience.

Put it this way. Palin and Obama are about the same age (two years difference). Palin has served two years as Governor, Obama two years as Senator. They each had smaller, even similar, jobs before coming to their current one. Obama was an activist and organizer and Palin a city council member and mayor.

Of course, Obama spent about three-quarters of his senatorial service running for president, not actually serving as senator, where the majority of his votes were “NV”, or “No Vote.”

Palin served all of her time as governor, governing. And by all accounts accomplished quite a lot.

By any measure, Palin is better qualified for President than is Obama.

Which is why it is strange the only complaint heard so far against Palin is her “lack of experience” when she has more than Obama (especially on energy and oil).

Don’t forget, too, that we are selecting between McCain and Obama, not Palin and Obama. But you wouldn’t guess that by the commentary so far.

We can ask the Obama camp, why so nervous? I smell fear (I have a large, sensitive nose). Their reaction to Palin is an over-reaction.

Palin might turn out to be less than hoped for, but it’s unlikely. We’ll see when she debates plagiarist Biden.

I was charmed by the way she named her youngest son and impressed by the way she courted Hilary supporters.