Many—as in lots and lots—of folks wrote in and asked me to review the McShane and Wyner paper. Thanks!
Gordie Howe—Mr Hockey to you—didn’t need his stick, his hockey stick, to plaster his opponents against the boards. Nor did he have to wave his blade, Tim-Dr. Hook-McCracken1 style, in order to fill the other team with fear. No, sir. Old Number 9 relied almost solely on his elbows to raise temperatures on the ice and score goals.
Statistically speaking, McShane and Wyner emulate Howe by applying a forearm check to the throat to Mann’s proxy reconstruction of temperature, cracking his hockey stick irreparably, leaving his models sprawling on the ice.
Like old school players, McShane and Wyner start with a little trash talking, albeit using sophisticated phrasing: “In fact, Li et al. (2007) is highly unusual in the climate literature in that its authors are primarily statisticians.” And they quote Boss Wegman—who once picked on me, publicly in print, for being a prof. at a med. school, but I hold him no grudge; just don’t let me get him out on the ice—”While the literature is large, there has been very little collaboration with university-level, professional statisticians.” The authors also show off their team, my pal Tilmann Gneiting, as well as Larry Brown and Dean Foster, all men of statistical brilliance.
But we can tell these taunts were included as a matter of form, thrown in because it is traditional. They don’t spend much time on them, and instead focus their efforts where it counts, exploiting Mann’s huge, gaping statistical five hole.
There’s little point in summarizing the statistical methods the pair use to pummel Mann: the paper is not especially difficult and can be read by anybody. It’s also so that the boys haven’t said much new2, but what they do say, they say well and plainly. It’s the sheer spectacle that’s worth attending to.
Hip check! “[A] random series that are independent of global temperature are as effective or more effective than the proxies at predicting global annual temperatures in the instrumental period. Again, the proxies are not statistically significant when compared to sophisticated null models”
High stick to the chops! “[I]t is possible that the proxies are in fact too weakly connected to global annual temperature to offer a substantially predictive (as well as reconstructive) model over the majority of the instrumental period.”
Scientific deke! “[T]he proxy record has some ability to predict the final thirty-year block, where temperatures have increased most significantly, better than chance would suggest.” Proxies and temperatures have less measurement error the closer to we are to now. This implies the relationship between proxies and temperature is not stationary, as is usually assumed. That means a model applied to data now won’t work for older data. And that means we should be even less certain.
Are their weaknesses in the boys’ approach? Sure: their base model might stink, which they acknowledge. The Lasso can overfit, usually by following spurious dipsy-doodles in the data too closely. But even if that’s so here, it’s inconsequential. Look to their Figure 16, their backcast from a full Bayesian model. What’s most lovely about this picture is that it (tries to) show the complete uncertainty of the predicted temperatures.
The jaggy red line is their prediction, over which they lay bands of uncertainty due to various factors. Just look at that envelope of possible temperatures!—the dull gray window. The straight yellow line is mine: notice how it slides right through the envelope, never poking out through it at any point. This suggests that a flat-line, i.e. unchanging, temperature fits just as well as the boys’ sophisticated model. At least, the unchanging “null” model cannot be rejected with any great certainty.
This is just as beautiful as a shorthanded goal. It means we cannot tell—using these data and this model—with any reasonable certainty if temperatures have changed plus or minus 1oC over the last thousand years.
McShane and Wyner don’t skate off the ice error free. They suggest, but only half-heartedly, that “the proxy signal can be enhanced by smoothing various time series before modeling.” Smoothing data before using it as input to a model is a capital no-no (see this, this, and this).
Finally, we have our Amen, the touch of Grace, the last element of a Gordie Howe hat trick3, and worthy of an octopus tossed onto the ice:
Climate scientists have greatly underestimated the uncertainty of proxy-based reconstructions and hence have been overconfident in their models.
Update Reader Harold Vance has answered the call of science—thanks Harold!—and provided us with a greyed out picture, shown here:
Can a hockey stick fit this? Sure. Can a straight line? Also sure. A line which also starts high in 1000 and continuously drops until now also fits. It’s getting colder! Like the authors said, we can tell +/- 5 degrees or better, but not so well with less than +/- 1 degree.
1Slapshot reference. This, along with (the original) Bad News Bears, comprise the two best sports movies ever.
3A goal, an assist, and a fight.