The McShane and Wyner Gordie Howe Treatment Of Mann

Thanks for the great article. The hockey comparisons are really funny. Now, I have to read the actual paper.

Even though I am not a hockey fan, I was privileged to see Gordy Howe (and his sons) play for the Houston Aeros in the 1970’s. I only went to one game, but even in his fifty’s, Gordy Howe was really amazing, and I almost never use the word “amazing” for anything.

Matt:
Can you comment on their Bayesian reconstruction? In particular, can you look at their conclusion that there is an 80% chance that the last decade was the warmest in a 1000 years?

Excellent wit and pacing.

“start with a little trash talking”

Would the “Team’s” suppression of journal articles via peer pressure and review be akin to the Hansen Bros. sucker punching an opposing player before the game even begins??

@PolyisTCOandbanned — Nice try for a freebie. From Briggs’ resume it seems that he is a statistician to the stars and is a gun for hire. Wanna be a star?

If respecting Green Gang inputs, methodology, and highly manipulated graphics results in prima facie statistical garbage-out, we infer with a 95%+ confidence-level that Briffa, Hansen, Jones, Mann, Trenberth et al. put garbage in.

Apparently Schmidt, Romm, and other Cargo Cultists of their ilk are hastening to censor all references to McShane and Wyner lest Chicken Little take offense. We used to think that Reality would nail climate hysterics’ coffins shut around 2016; this plus Satellitegate (sic) and various pending worldwide lawsuits make 2012 seem a better bet.

John Blake,

The day that a majority of voters understand that these guys never checked their instruments, don’t make data and code available for others, and have no interest in replicating anyone else’s work anyway, Global Warming will be dead and Al Gore will wish he were (from the embarassment).

Try it with someone you know who thinks there might be something to global warming. Explain that they never checked their thermometers and when someone finally did, about 9 of 10 flunked basic scientific standards. That’s usually all it takes for the person to realize that incompetence reigns.

“This suggests that a flat-line, i.e. unchanging, temperature fits just as well as the boys’ sophisticated model”

Is that really true? The flat-line spends more time on the outskirts of the estimated envelope of possible temperatures, suggesting that it is a less likely trajectory than the central result.

I got to see Gordie Howe play in the old timers game before the All-Star game a few years ago. Gordie’s secret was finally made clear – his stick telescopes out to about ten feet long – what a riot!

A few word from the spin master at climate progress. What a moron!

One savory quote is about the disappearance of the MWP which never existed????????

http://climateprogress.org/2010/08/16/hockey-stick-paper-mcshane-and-wyner-statisticians/#more-31767

I saw recently, that Gordie Howe never actually had a Gordie Howe Hat-trick, is there any truth to that, or did Micheal Mann concoct that?

Matt

Very funny post, though the hockey references required a bit of deciphering by this Brit.

Over at Rabett Run, Eli posts the following vindication.

My explanation at link for why the M&W reconstruction is erroneous, was a little too simple. It’s the Wabett who gets it completely right: the fundamental error is calibration only against a hemispheric average, when local data — the 5×5 degree grid cells of instrumental data as used by Mann et al. — provide a so much richer source of variability — i.e., signal — to calibrate against.

It is this poor signal/noise ratio that helps the calibration pick up spurious low vs. high latitude temp difference “signal”, which in the reconstruction interacts with the Earth axis tilt change effect.

What stands is the observation that doing the calibration against the instrumental PC1 (instead of hemispheric average) will give you back pretty exactly the genuine Mann stick(TM) even in spite of this.

Congrats Eli!

In response, one commenter writes

Actually this argument only goes so far. Yes, the local temperatures vary more, but their coefficients are not the ones you’re interested in. They get averaged when you look at the hemispherical temperatures. And remember that the whole computation is linear in the instrumental temps: you can just add a row representing hemispheric mean to the RegEM computation tableau. Then it shouldn’t make any difference whether you first average instrumental values and then reconstruct, or reconstruct per grid cell and then average. The outcome, and uncertainties, should be the same.

And another

I don’t think this local/global calibration will matter one whit after the whole process is completed, given the linearity of the both sides of the model, but it would be good to check that with the exact process used, etc.

In any case, one of the main points of the paper is that because of the weak signal in the proxies you can get almost any shape you want within the error bars depending on meta-assumptions. The reason the shouting is so loud about paleo is because it’s impossible to decide objectively about all these meta and modeling assumptions.

In summary, because of the weak signal, and the sensitivity to assumptions, it’s (mostly) not science anymore.

This is beyond me statistically but I wonder if you’d care to comment? Does Eli have anything approaching a point?

Mr. Briggs: I too spent some time at Leader Shouldice’s Squirrel Cage (LSSU nee LSSC). I too miss the auroras. I think it’s too early to declare the “hokey stick” (to borrow someone else’s term for it)broken and dead. It is a good first attempt to get the requisite skilled people to look at the methodologies used by the “Team”. It will be interesting to see what shakes out of the trees in the following days/weeks/months. Go Lakers!!!!!

Matt, I think your criticism at this point is incorrect:
“McShane and Wyner don’t skate off the ice error free. They suggest, but only half-heartedly, that “the proxy signal can be enhanced by smoothing various time series before modeling.”

Looking at page 26 it seems you left out the word “Perhaps” at the beginning of that sentence. My reading of the whole paragraph leads me to believe that they took yet another shot at Mann here by first hanging the “literature” on two of his papers and then going on to list the various problems with the method. The last sentence should have had you nodding (if I’ve properly understood).

“Furthermore, smoothing data exacerbates all of the statistical significance issues already present due to auto-correlation: two smoothed series will exhibit artificially high correlations and both standard errors and p-values require corrections (which are again only known under certain restrictive conditions).

bob

OK, I always like to give the comments a go, and especially those that are critical, but TCO keeps coming up as “total cost of ownership” in all the texting dictionaries I try. Any help out there?

thanks,

brian

I am not a statistician, and I can see your point that the temperature could conceivably be a flat line and still be within the error window, but it is inescapable that the window itself shows cooling to ca 1840, and warming since. I guess the key point is that there is no evidence of “unprecedented in the last 1000 years.

Further to my previous post, I thought it might be worth flagging up the following
comment by Chris Watkins at Climate Audit. In subsequent comments, Chris tones down his rhetoric but he does seem to be statistically knowledgable. Or perhaps as a non-statistician, I’m too easily impressed!

The new McShane and Wyner paper due to appear in Ann. Stats. is clearly going to be much discussed, so I thought I would get in with a few comments, after scanning it briefly.

Let me say first that it is great news that some stats journals are taking a look at climate reconstructions. Unfortunately the first half of this paper is very silly, and the second half is slightly more sensible, and the most plausible reconstruction they produce…..looks rather like the hockey-stick.

In the first half, they take 1200 temperature proxy series (treated as independent variables) and fit them to 119 temperature measurements (keeping overlapping holdout sequences of 30 yearly temperature measurements). Fitting 1200 coefficients to 119 data points is of course hopeless without further assumptions. Instead of doing some form of thoughtful data reduction, they employ the lasso to to the regression directly, with strong sparsity constraints.

They justify their choice of the lasso by saying:
“…the Lasso has been used successfully in a variety of p >> n contexts and because we repeated
the analyses in this section using modeling strategies other than the
Lasso and obtained the same general results.”
Both parts of this statement are wrong, and the first part is a MORONIC thing for statisticians to say. They give absolutely no reasons to suppose that the Lasso — a method that makes _very_strong_ implicit assumptions about the data — is in any way appropriate for this problem.

The Lasso _is_ appropriate in certain cases where you believe that only a small subset of your variables are relevant. To use it as a substitute for any data reduction with 1200 variables and 119 data points, when _all_ the temperature proxy series are presumed to be relevant to some degree, and all are thought to be noisy, is simply stupid.

Not surprisingly, they find they can’t predict anything at all using the Lasso. (It is a completely inappropriate technique for the problem.)

In the second half of the paper, they do something which is almost sensible, (but less sensible than what the climate modellers do). They take 93 proxy series that go back a thousand years, and do OLS regression on various numbers of principal components of these series. Regressing on just one PC gives more or less Mann’s curve (ironically this is probably the most defensible prediction from all the ones they try)– when they regress on 10, they back-cast historical upward trends. If they were being agnostic statisticians, then I suspect that from the cross-validations they show, the most conservative model they could choose would be a model predicting on one or a very few principal components.

Hey presto, they’ve recovered Mann’s hockey stick as a most plausible estimate. As Garfield would say, Big Hairy Do.

That’s the bulk of the paper. Some but not all of the points they make about over-tight confidence intervals in the previous literature seem valid.

In my opinion they do not introduce any useful new techniques into palaeclimate reconstruction: their main contribution is to show that using the Lasso with no prior dimension reduction is as useless an idea as any sensible person would expect it to be.

This paper shows, if proof were needed, that it is possible to get ill-considered papers into good peer reviewed journals, especially if they are on hot topics.

Keep in mind that McShane and Tyner do not address the preferential inclusion of those samples showing stick behavior and the systematic omission of large numbers of nearby samples that do not. That is, they dealt only with statistical problems, not the cherry picking.

Flangepp: It’s not true that Gordie Howe never had a “Gordie Howe Hat-trick”. But it is true that he never had many in his career, and several players since then have recorded far more than he did. That statistic, however, must be considered in context. The reason why Gordie Howe recorded so few Gordie Howe Hat-tricks is that in his long career only a few men were crazy enough to fight Gordie Howe. (Just look at the photo of Lou Fontinato’s face.) One must be careful when interpreting old hockey statistics, just as one must be careful when interpreting tree rings.

Noblesse Oblige says:
Keep in mind that McShane and Tyner do not address the preferential inclusion of those samples showing stick behavior and the systematic omission of large numbers of nearby samples that do not. That is, they dealt only with statistical problems, not the cherry picking.

I think they dealt with this in the comment “We assume that the data selection, collection, and processing performed by climate scientists meets the standards of their discipline.”

Your update is effing priceless.

TGSG says:
I think they dealt with this in the comment “We assume that the data selection, collection, and processing performed by climate scientists meets the standards of their discipline.”

Indeed. And one can read a firm tongue in cheek there, if one wants, though it is an appropriate stament for them to make since they are statisticians, not climate guys.

My point was really that the gray error range is really substantially larger because of the cherry picking. Matt may have impied this when he said “it (tries to) show the complete uncertainty of the predicted temperatures,” emphasis “tries to.”

As always, Briggs, outstanding.

As someone who estimates econometric equations for a living, let me put in my two cent’s worth as well:

1) the grayed-out chart is indeed where the real story is. This means the “real” numbers could be anywhere within that path, stated with 95% confidence that this is so. That does leave a 5% possibility of individual or multiple outliers, but let’s accept that this is the reality of the situation for now.

2) Under the assumption of a normal distribution, the red line of the original chart could suggest a cooling and then a warming. What is not, however, stated, is the reason that a normal distribution would be applicable here (if anything, given the claim of warming, you’d expect to see a skewed distribution that would drive up the temperatures…but this is not discussed anywhere).

3) The authors do look at distribution and find that it is problematic: see fig. 7 on p. 12 of the paper. That is, however an indication of the fundamental problem: this distribution is actually the test of proxies vs. actual temperatures (lag one autocorrelation coefficients between proxies and instrumental temperatures during the instrument period), meaning…what?

If these proxies were accurate, the distribution would be around the 0 point on the axis. They are not: they are centered around 0.5° C instead. Hence the proxies have an inherent bias upwards, and are rejected by the authors of being usable to reconstruct the series:

“To assess the strength of the relationship between the natural proxies and
temperature, we cross-validate the data. This is a standard approach, but
our situation is atypical since the temperature sequence is highly autocorrelated.
To mitigate this problem, we follow the approach of climate scientists
in our initial approach and fit the instrumental temperature record using only proxy covariates. Nevertheless, the errors and the proxies are temporally
correlated which implies that the usual method of selecting random
holdout sets will not provide an effective evaluation of our model.”

This simply means that the proxies cannot be used for regression analysis to determine temperature, as their error terms are not independent from that of the instrument temperatures. They can be used as a proxy for temperatures, but NOT as an independent variable to regress a temperature from multiple proxies.

4) From 1 and 2 we do not know the distribution of the mean from within the set of proxies, just the variance from a possible joint mean. From 3 we know that we can’t use a simple regression analysis to find that mean.

5) The idea that the series should be smoothed to improve them, especially for regression analysis…ye gods.

Here’s that paragraph in its entirety:

“Finally, perhaps the proxy signal can be enhanced by smoothing various
time series before modeling. Smoothing seems to be a standard approach
for the analysis of climate series and is accompanied by a large literature
(Mann, 2004, 2008). Still, from a statistical perspective, smoothing time series
raises additional questions and problems. At the most basic level, one
has to figure out which series should be smoothed: temperatures, proxies,
or both. Or, perhaps, only the forecasts should be smoothed in order to reduce
the forecast variance. A further problem with smoothing procedures
is that there are many methods and associated tuning parameters and there
are no clear data-independent and hypothesis-independent methods of selecting
amongst the various options. The instrumental temperature record
is also very well known so there is no way to do this in a ”blind” fashion.
Furthermore, smoothing data exacerbates all of the statistical significance
issues already present due to autocorrelation: two smoothed series will exhibit
artificially high correlations and both standard errors and p-values
require corrections (which are again only known under certain restrictive
conditions).”

Here is one of the key problems, correctly identified but ignored: that smoothing seems to be a standard approach for the analysis of climate series. It has always bothered me, since conclusions reached on that basis are determined more by the smoothing methodologies, rather than the data. That is fundamental!

Smoothing removes exactly the information needed to do things like determine distributions of the data and make all the fun tests available moot. It creates artificially high values for things like r^2 or DW, making the data look like it is vastly better than it is, and for me that is a sign of intellectual sloppiness and very, very poor methodology.

Now, there is one thing that has always bothered me and in my cursory review of the literature simply doesn’t appear: smoothing methodologies always appear to be some sort of multiple moving averages. Why is not something used that is heavily documented, widely accepted with a very large body of analysis and statistically sound, such as … seasonal adjustment? Census X-11, anyone? Even X-12 with its totally funky AR components?

It seems to me that this is really what the climatologists should be using: they want to have instrumental readings that have outliers removed in a statistically sound manner, that removes spurious and random variances, that identifies noise, seasonal factors affecting the development of a time series, and provides both a forecast of seasonal factors for a full year as well as a long-term trend component of the deconstructed original data, while all the time remaining within strict statistical parameters.

Just a thought. As I said, I haven’t been able to research this, but perhaps one of the others has. Seasonal adjustment should be the basis for climate research, methinks…

PS to the above:

Obviously, seasonal adjustment can only be used for monthly and quarterly frequencies: smoothing annual time series has to be done differently. But my point remains: there has to be a clear reason for smoothing annual series, rather than working with the original data and using statistical distribution analysis to determine variance from the mean. Failing to do so means that the data is not being properly analyzed.

So if the fact that climate scientists publish without the help of statisticians leads to problems (which I would agree with), then why would there not be similar problems if statisticians publish without the help of climate scientists?

Seems that a limited familiarity with climate science has led to some clear shortcomings in this paper:

http://klimazwiebel.blogspot.com/2010/08/mcshane-and-wyner-on-climate.html

Kris,

Many people seem to be pointing to the Zorita people as take down of M&W. I don’t see that all. Most of the criticism seems to be on tangential points and it contains a number of inaccuracies itself (as pointed out in the comments). Zorita even slights the paper by saying that its conclusions are already well known! Hardly a devastating criticism.

The final point by Zorita (“It is not clear to which models the authors are referring to … If they mean ‘climate models’, they are again dead wrong) is mystifying. The word “model” is used throughout the paper and always with the same meaning (reconstruction statistical models). This attempt by Zorita to set up a strawman is rather foolish and suggests he has not read the paper in detail.

Undoubtedly there will be problems with that post, and I agree that some of them seem tangential comments that do not affect the results. But in that post and a related one at Deep Climate (http://deepclimate.org/2010/08/19/mcshane-and-wyner-2010/) there seem to be quite a few weaknesses being pointed out in the statistical techniques that M&W used. I’m neither a statistician nor a climate scientist so I can’t judge the significance of these comments for the final conclusions of the paper. But I am a working scientist who reads and produces papers for a living, and I know that one paper rarely holds the final word on a topic. Rather, the truth emerges slowly from an ongoing debate between people who are knowledgeable but may hold different scientific opinions.

In any case, what you have said does not alter my main point of criticism: if it is problematic when climate scientists work without help of statisticians, then the reverse should also be avoided.