But this time criticism of the latest we-are-dooomed scenario has extended beyond the usual cluster of sites. Even Der Spiegel picked up on it: Globale Erwärmung: Forscherkrieg um Klimastudie.
The relevant passage, which I used Google Translate to provide, is this:
More serious the allegation several critics, the data base of the study was chosen so that a certain result come out. Hsiang and his colleagues had “comparing apples with rollerblades”, wrote about William Briggs, statistics professor at the U.S. elite Cornell University, dripping with sarcasm in a blog post . “Data of past Tuesday will be just as meaningful as that of 8000 BC.” At the end are “many great graphics” and a result came out: “Hot, dry weather is bad for us.” Therefore, mocks Briggs, also draw everyone into the hot from the cold Michigan South Carolina – “to be where the action is.” “Complete nonsense,” was still the kindest description of the study.
No, no. My kindest description was to say that Hsiang and his co-authors put in a staggering amount of work. No faulting their diligence, he said sincerely.
But hard work doesn’t always imply good work (as regular readers of this blog will attest!).
Here is the succinct summary of Hsiang: they mixed physical measures of the atmosphere and land from widely disparate times and places without accounting for the uncertainty inherent in each or in their combination; they defined “violence” haphazardly, intent to let it be rape at one location and “leader removal” at another. They cobbled the whole together into a—let me charitable—creaky statistical model which was able to produce a wee p-value, defined as a p-value less than the magic number (0.05).
Additionally, every site and data source they used did not show increasing “violence” with increasing inclement weather (people do not experience a climate, but moment-to-moment weather). It was only after mixing the stew of the sources and kinds of “violence”, all said to be just plain “violence”, into their model were they able to “prove” what they hoped would be true.
Suppose they were interested in rape. What they should have done is to look at rape statistics from localities all across the globe, accompanied by those meteorological variables which they could show, or plausibly show, are directly responsible for rape. They’d have to answer questions like, Why isn’t rainier weather correlated with less rape? After all, rapists like to keep dry, too.
But this alone would not have been good enough to show a connection. After all, rape rates are measured with substantial error which depends on location. These uncertainties would have to be “carried forward” in their models. And then culture plays an enormous role in rape: these variables would have to be controlled for.
Of course, the weather/climate variables are also subject to large measurement uncertainties, which also depend on location and time. Time itself is highly problematic. Cultures change through time, immigration and emigration have to be understood, technology changes things (e.g. cameras may dissuade rape). And on and on.
And these are just the bare minimum to be able to say anything about just rape. What about “leader removal”? Same set of considerations, only now expanded for politics, that simple subject. And so forth for the other kinds of “violence.”
Then, finally, when all this is done, they’d have to look at the “opposites” of these variables. It could be that “leader removals” increase with temperature, but it might also be that people are happier, politically speaking. It was not joke about people moving from the North to the South, where it is much hotter: it’s happening with increasing frequency and people seemed pleased about it. They do not seem to be turning to violence at their new addresses.
Just because it is easy to dump a bunch of numbers into a computer does not mean that it should be done. The software is forced to give you an answer, but that does not necessarily mean the answer is what you hope it is. That this paper showed up in Science says lots about how easily influenced even our best and brightest (but see Belloc’s exception) can be by fad and bias.
Update More from the Der Spiegel article (at the end):
Storch also makes allegations of the “Science” Editorial: “The peer-review process did not work you should have critical evaluators, which was obviously not the case here..” A spokeswoman of the journal, however, can see nothing but good in the debate. “Science is a self-correcting process,” she wrote in an e-mail. Researchers published their results so that they confirmed could be refuted or corrected. “In this way science makes progress.”
You bet it is, baby. Consider this a massive self-correction.