Remember when I said how you shouldn’t draw straight lines in time series and then speak of the line as if the line was the data itself? About how the starting point made a big difference in the slope of the line, and how not accounting for uncertainty in the starting date translates into over-certainty in the results?
If you can’t recall, refresh your memory: How To Cheat, Or Fool Yourself, With Time Series: Climate Example.
Well, not everybody read those warnings. As an example of somebody who didn’t do his homework, I give you Phil Plait, a fellow who prides himself on exposing bad astronomy and blogs at Discover magazine. Well, Phil, old boy, I am the Statistician to the Stars—get it? get it?1—and I’m here to set you right.
The Wall Street Journal on 27 January 2012 published a letter from sixteen scientists entitled, No Need to Panic About Global Warming, the punchline of which was:
Every candidate should support rational measures to protect and improve our environment, but it makes no sense at all to back expensive programs that divert resources from real needs and are based on alarming but untenable claims of “incontrovertible” evidence.
Plait in response to these seemingly ho-hum words took the approach apoplectic, and fretted that “denialists” were reaching lower. Reaching where he never said. He never did say what a “denialist” was, either; but we can guess it is defined as “Whoever disagrees with Phil Plait.”
The WSJ‘s crew said, “Perhaps the most inconvenient fact is the lack of global warming for well over 10 years now.” This allowed Plait to break out the italics and respond, “What the what?” I would’ve guessed that the scientists’ statement was fairly clear and even true. But Plait said, “That statement, to put it bluntly, is dead wrong.” Was it?
Plait then slipped in a picture, one which he thought was a devastating touché. He was so exercised by his effort that he broke out into triumphal clichés like “crushed to dust” and “scraping the bottom of the barrel.” You know what they say about astronomers. Anyway, here’s the picture:
See that red line? It’s drawn on a time series—wait! No it isn’t. Those dots are not what Plait thinks they are. They are not—they most certainly are not—global temperatures. Each dot instead is an estimate of global temperature: worse, most dots are also different kinds of estimates from each other. That is, the first dot was estimated using data X and method A, and the second dot was estimated using data Y and method B, and so forth. Well, maybe the first and second dot were the same, but older dots are different than the newer ones.
With me so far? All you have to remember is these dots are estimates, results from statistical models. The dots are not raw data. That means the dots are uncertain. At the least, Plait should have shown us some “error bars” around those dots; some kind of measure of uncertainty.
Now—here’s the real tricky part—we do not want the error bars from the estimates, but from the predictions. Remember, the models that gave these dots tried to predict what the global temperature was. When we do see error bars, researchers often make the mistake of showing us the uncertainty of the model parameters, about which we do not care, we cannot see, and are not verifiable. Since the models were supposed to predict temperature, show us the error of the predictions.
I’ve done this (on different but similar data) and I find that the parameter uncertainty is plus or minus a tenth of degree or less. But the prediction uncertainty is (in data like this) anywhere from 0.1 to 0.5 degrees, plus or minus. That is, prediction uncertainty is about five times larger.
I don’t know what the prediction uncertainty is for Plait’s picture. Neither does he. I’d be willing to bet it’s large enough so that we can’t tell with certainty greater than 90% whether temperatures in the 1940s were cooler than in the 2000s. And also such that, just as the WSJ‘s scientists claim, we can’t say with any certainty that the temperatures have been increasing this past decade.
In other words, the scientists were right and Plait was wrong. Or, as he might phrase it, he blatantly misinterpreted long term trends. Notice old Phil (his source, actually) starts, quite arbitrarily, with 1973, a point which is lower than the years preceding this date. If he would have read the post linked above, he would have known this is a common way that cheaters cheat. Not saying you cheated, Phil, old thing. But you didn’t do yourself any favors.
Somewhat amusingly, Plait ends his semi-random venting by telling us that Michael Mann has been “tweeting furiously” about this. Good grief! This isn’t helping his case. Mann’s understanding of statistics may be likened to an overly enthusiastic undergraduate who left the lecture early.
P.S. Hey, Phil. Since you brought it up: the total consideration I’ve received for my work in global warming from Big Oil (or anybody) is number so small that dividing by it is forbidden. How much do you get for your blog or other environmental work, including government funds?
P.P.S. I didn’t forget about that “warmest years on record” stuff. Those “warmest years” are still estimates and have to be compared to the old data, which itself must be accompanied by uncertainty measures. And anyway, it has been much hotter in the past than it is now. Jurassic anybody?
Update Thanks for all the comments, everybody! 100+ and no signs of slowing. I will read them in all, in time, but for now, since many of them repeat odd claims and misunderstanding of statistical methods, let me point you to the BEST project posts (here and here). BEST had parameter-based error bars, but not predictive ones. But some acknowledgment of uncertainty is better than none! Also look under the Start Here tab and pay attention to the smoothing time series posts, the homogenization of temperature series posts, and read this weeks’ All of Statistics series. You may also read, inter alia, the Probability Leakage post which describes the Bayesian predictive approach I am using. A lot of confusion and frank unfamiliarity for some of you.
Update to the Update See this brand-spanking new post that clarifies some of the statistics some of you couldn’t be troubled to look up.
Update See this cartoon which shows that the IPCC has been known to employ the technique of variable start dates.
Update It is imperative that all read this series, where I describe just how so many people make mistakes. Those below who have been shouting the loudest are most in need.