Another question from the statistics mailbag:
Dear Matt: I recently got into a discussion with a CAGW “believer” and of course the discussion turned to global average temperature (whatever that is) anomalies and that the predictions of climate catastrophe are based on computer model output. I then said, “If a computer model cannot predict regional changes, it cannot predict global changes. Averaging incorrect data does not give accurate data,” referring to the computer models. Was that a correct statement?
Although I once took statistics courses, about the only things I remember are median, mode, mean, and standard deviation so if you have time to respond to this e-mail, please do so in a ridiculously simple way that I might be able to understand.
Thanks. By the way, I like your new format.
Sort of Yes, Chuck. The part that’s tricky is your conditional: climate models necessarily do better at “higher” scales than “lower ones.” But your second part is right: averaging a Messerschmidt, no matter how large, still leaves you with a Messerschmidt, if I may abuse the punchline to the old joke.
First, a “climate” model is just a model of the atmosphere. What makes it “climate” is its scale and nothing more; what we call “climate” versus what we label “weather” is really a matter of custom. So imagine a model of climate of the Earth from the view of Alpha Centauri. From that vantage the Earth is indeed a pale blue dot and its “global mean” temperature can be modeled to high accuracy, as long as we don’t try for too many decimal places. We can even ignore seasonality at this distance. Heck, I’d even believe a forecast from James Hansen for “climate” as defined this way.
But now imagine the temperature and precipitation on a scale of a city block for each hour of the day and over the entire surface. This would be incredibly complex to model and verify. Even trying to write down the computing resources required produces a dull pain in the occipital lobe. To my knowledge nobody tries this for the globe as a whole, though it is done over very small areas for limited time frames. The hope that this scale of model would be accurate or useful as a climate model matches that of a Marxist who says to himself, “Next time it’ll be different.”
Here’s the tricky part. A climate model built for large-scale climate can do well, while another built for smaller-scale climate will fare more poorly, each verification considered at the scale intended of each model. We can, as you suggest, average the small-scale model so that the resultant output is on the same scale as its coarser brother.
Now it can happen that the averaged model judged on the coarser scale will outperform itself judged on its original scale. This could be simply because the model did well on the coarse scale but poorly on the fine scale. Of course, the averaged model may also perform poorly even on the large scale. There is no way to know in advance which will be the case (it all depends on the competence of the modelers and how well the models reproduce the physics).
But, all things equal, the variance (of the verification or the model itself) of the averaged model will be larger than the variance of the large-scale-from-birth model. That means we would have either more trust in the large-scale model, or in its verification statistics (even if those stats showed the model to be poor) or both.
The old tale of the Chinese Emperor’s Nose is somewhat relevant here. Nobody in China knew its length, but they desired to have the information. Why they wanted to know is a separate question we leave to competent psychologists. Anyway, many people each contributed a guess, each knowing that his answer was probably wrong. But they figured the average of all the wrong guesses would be better than any individual guess.
Wrong. Taking the mean of nonsense, as suggested above, only produces mean nonsense. So that if the small-scale model stunk at predicting small-scale climate, taking averages of itself (via ensemble forecasting, say) and then examining the average model on the same small (not large) scale will still leave you with a Messerschmidt.