Nov 20 2009
The University of East Anglia Climate Research Unit Hacking
Be sure to also see this story!
And this story a follow-up
See also see this story on proxies
I often write emails to pals of mine that are in shorthand, that take many things for granted, that begin with understood knowledge, and that if they were read out of context could be construed as damning.
It’s easy to produce indictments. It’s as simple as adding “in bed” to the end of Chinese fortune cookies. “You will have great success in the future” suddenly takes on an entirely new meaning for somebody intent on discovering an x-rated conspiracy among fortune cookie writers.
So, caution, friends. Be careful.
For those who don’t know, the University of East Anglia Climate Research Unit was hacked and a series of emails and documents related to climate change were released on several FTP servers; now it’s on the torrent, and so it’s far too late to prevent their spread.
The hacking was undoubtedly illegal, as may be the reproduction of this private correspondence publicly. So again, caution, friends. Don’t get cocky.
The reason many might be daring is that the emails were written by the Big Cheeses in global warming: all the important names are there. They are discussing many topics, from computer code to how to handle skeptics and the press. But most curious are the long discussions of “Where is the warming?”
A lot of people in those emails are deeply concerned about the lack of observed warming (it has actually been getting just a little cooler). Some lay the blame at the feet of the certain components of the models, others try to dismiss the observed cooling, still more advocate statistical manipulation to mask the cooling so the public and its leaders don’t get confused.
It’s far to early to give a concise summary of this scandal. So, caution, friends. Don’t rush to judgment. It’s too easy in situations like this for statements to come back and bite you.
I have seen the files—not all of them, there are too many—and my early take doesn’t change the view I have already formed: climate models have no skill beyond about one year. The models predict warming, but the warming isn’t there, therefore the models are wrong. Why they are wrong is an interesting question, and worth investigating. Many of the emails responsibly take this tack. And they should.
I have not seen open acknowledgement that the premise that forms the models is false. That is, that it is possible, even with the observed small increase in atmospheric CO2, that that gas has at best a marginal effect. As far as I can tell by my early reading, all the folks in those emails truly believe their models (it’s the observations they don’t love).
There is no conspiracy, as far as I can tell. A conspiracy would obtain if the participants knew their stated beliefs were false, yet the still espoused them with the goal of winning either money, or power, or control, or whatever. My early, and admittedly incomplete, judgment is that all of these people really are convinced that catastrophic warming is on the way and that it will be caused by mankind. Further, they believe it fervently.
So, caution, friends. Try not to use the word “conspiracy” too readily. It is an extremely strong word—and it is beside the point. Or should be.
If, as a skeptic, you try to club the email originators over the head with conspiracy, they have the easiest defense: they believe. And people will see that they believe, that there is no conspiracy, and you will look like a brute and an ass. Meanwhile, what should really be at question—will it get hotter, colder, or can we know with sufficient precision—will be left unanswered.
Again, I see conviction in these emails, and strained attempts to tame and fix their creations, the models, so that the models’ outputs fall in line with what they believe. This sort of “experimenter’s bias” is fair game, and should be noted. It is the main story, I think.
But attempts to point out bias should be formed with compassion and not passion. This is not the time to settle scores, but to gain allies. We are dealing with a group of highly intelligent people and they can be convinced of mistakes where they exist.
So, caution, friends. Try to remain calm.
Be sure to also see this story!
And this story a follow-up
See also see this story on proxies
74 responses so far
Tony,
The records are sealed.
“The hacking was undoubtedly illegal, as may be the reproduction of this private correspondence publicly. So again, caution, friends. Don’t get cocky.”
The release of the Pentagon Papers was illegal too. And yet, the Left hails it to this day as a seminal moment in US history. Now, all of a sudden, the Left has a problem with illegally leaked information about misconduct on the part of powerful people. Not to mention the fact that opposition to the government was the ultimate expression of patriotism up until the election of Mr. Obama. Now it’s racist, to oppose the government. The Left needs to get it’s act together and figure out what is most important: politics or the truth.
I tend to believe that these “scientists” fudged on the data given that the main argument for AGW has been “we’ve ALL decided that man is to blame here so anyone who says anything different we will denounce as a moron, idiot, racist, homophobe, etc.” Like the release of the IPCC report; they released the results summary first so it could be used by pro-AGW politicians and activists as proof that man is to blame for global warming. The report itself, however, with all of the data and methods of how the data was obtained, wasn’t released for review until months later. Conclusions first, reviewable data dead last. And, let anyone point out this scientific ethical lapse and you are a “denier” on par with those who deny the Holocaust. That is the level of honesty in this debate on the part of AGW supporters. There is very little actual science in it. So, its not a large leap of logic to believe that such people would actually doctor up their data to fit their preconceived conclusions.
I’ve no doubt they are ‘true believers’ in global warming and it may not have been a conspiracy to lie about it or manufacture results.
But it sure looks like they conspired to downplay the uncertainty they knew was contained in their work and to marginalize any scientist with any sort of dissenting viewpoint.
And the overall impression is that they were driven by ideology and politics rather than science.
Calm? Remain calm?????
What a party pooper!!! Let us instead dance around the fire while twisting the knife.
Note to Krugman [http://wmbriggs.com/blog/?p=727]: who pray tell are apparently guilty of unforgivable treasonous betrayal of the planet now?
William,
I agree that a certain amount of caution is in order. Sometimes it was difficult for me to understand what the authors of the Hadley CRU emails were trying to say. However the part about hiding the decline–with respect to the dendroclimatolgy divergence problem–did not look good. I do feel comfortable in using the word Climategate. But since forensics is not my bag, I do not view the CRU hack/leak as a Gotcha moment–yet.
If anyone feels let down by all of the cautious talk here, I guest-posted a conversation-stopper to counter the claim of ‘unprecedented’ global warming at the end of the 20th Century at Anthony’s blog last week.
http://tinyurl.com/y9jl8c6
Sorry, no statistics in that article. However even without Climategate, my Seat-of-the-Pants Dendroclimatology trumps all of the premature hockeystick graphs generated by past tree-ring studies.
I’m a bit surprised that readers of this blog would take so much pleasure in the ad hominem opportunities this leak raises. If your goal is denouncement and defamation then I guess there is some ammunition in the leak, but if your goal is truth then the leak doesn’t help much. Just because you think these people’s procedures unscientific, or their lack of openness with data distasteful, it doesn’t mean they’re wrong about CO2 for example, and as WMB said, that should still be the big issue here.
Even with respect to the politics, I don’t see any reason why policies or public opinion would change as a result of this leak. Again, it’s truth rather than method that matters there; method is only important because it can improve people’s confidence in your theory. I’m also not sure why U.S. law is relevant, nor what a Congressional Inquiry would achieve.
For those of us who don’t particularly want to see people’s careers and reputations destroyed, maybe the best that can come out of this is more openness with regard to data and procedures, less censorship of off-message reports, more openness to criticism and hopefully a better scientific process as a result, which should result in more confidence in the truth of whatever theories stand up to more intensive cross-examination, and perhaps a change of theories altogether if that turns out to be warranted. And maybe less political meddling from supposedly scientific bodies, though I wouldn’t hold my breath.
The trouble is, I would have said exactly the same thing a week ago – as others have noted, you only have to pay a bit of attention to RealClimate and ClimateAudit to get some idea that the process is not as open as it should be. So I don’t think this leak has changed my mind – I just hope it might debunk the argument that “secrecy is OK and we can trust these guys” – which seems to come up in one form or another whenever freedom of information is mentioned.
For a topic like this, secrecy is not OK – I don’t mean that private emails should be published, but that scientific papers should be more thoroughly reviewed by hostile reviewers (the word “peer” seems to mean “friend” these days), and the published paper should contain everything needed to duplicate the result, including (links to) source data and working source code, with full details of where the data came from. They also really need to be honest and public about their levels of certainty in their work.
Beyond problems of method, I suppose the other interesting confirmation is that the face presented to politicians and the media is not telling the truth, or at the very least is overconfident about the theories’ provenness. For example, it is refreshing to see acknowledgement from a source of CRU’s calibre that the current climate trend was unpredicted and is hard to explain with existing models. It certainly contrasts with what the media was told a few years ago, and shows that scientific organizations do way too much PR.
“The hacking was undoubtedly illegal, as may be the reproduction of this private correspondence publicly. ”
Not exactly private. The emails were sent to and from email servers belonging to the university and/or government, depending on how the CRU funding is arranged. As far as I can see, none of the emails contain any personal / private information, aside from saying someone was sick and not available to respond to an email. This is hardly an invasion of privacy in normal day to day life.
While I am against hacking activity, the emails certainly indicate a culture of secrecy and selective response to FOI and other data requests. There is also a suspicion of collusion to stack peer review panels and editorial staff of scientific journals. Perhaps this is what motivated the whistleblower / hacker.
What surprises me most is that technically savvy scientists would not realize that their emails are persistent, even after they delete them from their own account. Email servers retain data for several years through company policy / government decree.
Many of the apologists for CRU in the blog world point to the relative paucity of data, spread over several years as an indication that the data supporting a “conspiracy theory” is slight. A counter to this argument is that scientists are just people and only occasionally forgot to maintain proper restraint in their emails. The term conspiracy in itself is designed to belittle its proponents. I prefer cultural maladjustment, wherein those operating within the culture see outsiders as threats, along with a good helping of confirmation bias. Trenberth in his “travesty” email shows there may have been attempts at questioning their models, but it looks like they still went with the flow. This would not be of major concern if this occurred in some minor institution where calm heads might address the problem/disfunction, but the CRU has a central and important role with respect to the IPCC, and has been seen as a clearing house for informing the panel of the science.
What needs to happen is an open enquiry into the hacking incident, because all we have at this stage is the allegation of a hack, and the appearance of confirmed genuine emails sourced from the CRU. The police and the university ethics committee should be compiling very long lists of questions for those involved. The citizens of the UK have right to question how a large amount of their taxes are being spent, and should be pressuring their MP’s to find the answers.
The “scientist” who expressed glee over the death of a skeptical blogger should of course apologise immediately to the family of the departed.
This is one of the calmest, most rational views yet, a pleasure to read. Still, although I do not think there is evidence of a conspiracy, there *is* pretty good evidence of collusion, which in science is just about as bad. With that, they have given up objectivity and they are no longer conducting honest or verifiable science. *All* of their work now needs to be re-worked from the start.
1. You don’t know that it wasn’t a whistleblower.
2. The conspiracy is to block dissent, at the least. Whether there’s other conspiracy remains to be seen.
It’s interesting to see a large number of statements along the “they may be crooks, but global warming is still real and important issue” lines.
This is not about GW. This is about catastrophic lack of scientific integrity and catastrophic abundance of zealotry in the “embedded” scientific community. This issue is more important than global warming: without credible science we are doomed in many ways, far before we get drowned.
And credible science is dissapearing faster than the polar ice. With friendly peers, corporate and government funding, academy cabals, soon the only scientists to trust will be the clandestine ones, that you meet in a dark corner and pay in cash.
Mr. Briggs,
While I understand your “hesitancy” to make any harsh statements, I also find it amazing that you’ve gone out of your way to make certain statements which are not only misleading, but factually false.
First of all, there are at least 3 different issues involved in the allegations of “conspiracy” (not just 1 issue, as you have falsely attempted to claim).
I agree with you that these scientists are likely “true believers” in man-made global warming, and thus I agree that they are not attempting to propagate a hoax which they don’t believe is true (with regard to the theory of man made global warming).
However, there are several other conspiracies which these scientists have knowingly participated in — (unless the emails were fabricated, which doesn’t seem likely since the authors have already validated the genuineness of several of the most damning emails to news outlets).
For example, these scientists knowingly conspired together (via email) to HIDE A DECLINING TEMPERATURE TREND from the public by using “obfuscation” and “omissions” of important scientific data. A conspiracy only needs to include 2 people to qualify as a “conspiracy”, it doesn’t need to include the whole scientific community and it doesn’t need to be “illegal” per se.
How do we arrive at the conclusion (without passion or emotion) that they participated in a nefarious conspiracy?
We know this is true because they talked about using clever “tricks” (clever types of “data manipulation”) which had the EXPLICIT and STATED GOAL of obfuscating a DECLINING TEMPERATURE TREND in scientific reports.
People from the University of East Anglia (CRU) have attempted to downplay the usage of the term “trick” (saying it was taken out of context), by suggesting that it merely indicates “a clever way of doing something” in science. (LOL)
However, any true news reporter (with even a modest IQ) would quickly point out (to their readers) that the term “clever” and “nefarious” are not mutually exclusive. Likewise, a “clever” idea can also be “illegal” or “unethical” too.
In this particular case the INTENT of the email author (and thus the CONTEXT of his message) was shown to be a desire to HIDE A DECLINING TEMPERATURE TREND by using CLEVER DATA MANIPULATION. This is not in dispute, since it’s written in plain English.
Also, in another email his STATED INTENT was to circumvent the Freedom of Information Act by (illegally) asking other people to delete certain emails which they didn’t want the public finding out about via the FOIA. This was stated in plain English, and it cannot be refuted in any logical manner.
I totally agree with the CRU that the author was attempting to be “clever” (when using the term “trick”), just as a bank employee would be “clever” if they found a way to steal money without anybody ever finding out.
Additionally, there was a 3rd issue of conspiracy involved in those emails…
The 3rd conspiracy involves attempting to “prevent” and/or “discourage” PEER REVIEWED JOURNALS from publishing scientific reports which these nefarious scientists didn’t agree with.
These unethical scientists openly discussed ways of using nefarious and underhanded methods (political pressure) on journal editors to encourage them to stop publishing reports for PEER REVIEW.
This is certainly not illegal, but it’s quite unethical for one of the world’s leading scientists to use backroom political pressure to “suppress data” from PEER REVIEW simply because he disagrees with it.
The fact that this was discussed via EMAIL (between 2 people) does make it a conspiracy to suppress scientific data from peer review (using nefarious tactics), regardless of whether it’s legal or not.
…and please keep in mind that these are the world’s TOP SCIENTISTS for climate change who are engaging in these nefarious activities, these are not just “run of the mill” teachers at some local college.
These are the leading climate authorities which the IPCC uses to base some of its findings on.
There are huge dollar amounts which are being influenced by the actions of these rancid scientists with their rotten ethics.
Oh, one final point…
Mr. Briggs, I challenge you to imagine a scenario where George Bush’s Administration (his science advisor) had sent out emails trying to conspire together with climate scientists to encourage them to suppress and/or obfuscate a “declining temperature trend”.
Just imagine if such emails existed and were HACKED AND PUBLISHED on the Internet.
The IPCC (and every other climate group) would publish that story on Page #1 less than five minutes after the story broke, and nobody would assume that George Bush’s team was engaging in “innocent scientific discourse” which was merely “taken out of context”. LOL.
Oh, and what if Sarah Palin was caught trying to delete climate data from public computers which was subject to FOIA disclosure, while admitting this fact (in emails) to other people?
Somehow I don’t think the climate groups would be as “neutral” and “understanding” IF Sarah Palin or George Bush were the people involved, LOL.
Likewise, I don’t think you’re taking these emails too seriously — since you seem to really be downplaying these emails.
Perhaps you have a personal bias. Maybe you view these people as “general colleagues” (fellow academics) who deserve a break from the “intense scrutiny” which you’d normally apply to other people?
If so, shame on you Mr. Briggs.
Matt:
I have a slightly refined view of this whole situation now that I learned about the timing of the final rejection of Steve McIntyre’s appeal of a UEA ruling on a FOIA request on November 12th!!
Anyone who has worked in a highly competitive environment will not be surprised by many of these emails. The notion that they somehow either prove or disprove a scientific conspiracy as opposed to a very cohesive coalition of like-minded individuals is an over-reach. Many academics are probably all too familiar with the politics and personal animosities associated with article publication in prestigious scientific journals. Dr. Roger Pielke Snr has already written on this point and I suspect that he will be taking further action as a result of the revelations in these emails.
So what is the issue? At core, it is the behavior of key scientists in response to requests to release data. The emails show a pattern of deliberate efforts to undermine the existing legal process for freeing information. The emails are unambiguous as to the efforts of Dr. Jones on this count. Given the title of the file – FOIA, its content and the timing of its release, in close proximity to a rejection of an appeal by Steve McIntyre for releasing data and other information – the odds are that this was not the action of a hacker but of a whistleblower.
If so, Dr Jones and the UEA administration have dug themselves a very deep hole. If the person who released this file was privy to the FOIA discussions and objected to the stonewalling in writing or made contemporaneous notes, then any efforts to pursue them may result in even more damage to the credibility of Dr. Jones et al. It is a genuine Catch-22.
So, folks should try to keep an eye on the pea here. The assertions by advocates for CAGW that skeptics believe that these emails somehow demonstrate a Michael Crichton-like conspiracy is a smokescreen that hides the simpler and more fundamental issue. There is and never has been a real reason for not disclosing the data and the code. The rather juvenile, silly and short-sighted effort to stonewall McIntyre and others has produced the real scandal. The notion that McIntyre and many others are part of some vast conspiracy to delay action on CO2 emissions, besides being neurotic, vastly underestimates the sheer puzzle value of climate issues to those of us used to doing large scale data analysis in other fields. Love him or hate him, nobody has any grounds for doubting McIntyre’s (and a growing number of other “amateurs”) abilities to analyze complex data sets and uncover large and small data and analysis errors.
To my mind it is remarkable that Gavin’s comment at Real Climate, Andrew Revkin’s piece, the pieces in the BBC, the Guardian, NYT, and most of the others I have seen fail to highlight the FOIA issue. I suppose the other stuff has greater voyeuristic value, but it really does miss the whole point of someone posting a file of emails and data title FOIA.
The CRU seem to have learned a lot from MPs, so instead of dealing with the historic fraud in the evidence they are squealing about who leaked the news. There seems to be a trend for Universities producing fraudulent information to get finance, witness the London Metropolitan. Nobody will ever believe a word this Unit says again. I’m sorry for the students who will now always have a question mark over their degrees, given the apparent deception that seem to have been rife in this place.
A. Watts, at his WUWT blog, has a plausible theory that the CRU website was NOT hacked; he infers the possibility that the content of the disclosure was a compilation for Steve “McInteyer’s” (spelling?) FOIA request, and, that this information was zipped into a file that was inadvertently put in a place that was publicly accessible. His speculation, based on general behavioral patterns coupled with the pattern of the content disclosed is at:
http://wattsupwiththat.com/2009/11/23/the-crutape-letters%c2%ae-an-alternate-explanation/#more-13003
Sorry folks, but the only clear conspiracy here was the conspiracy to suppress dissent. They still refer to us as denialists and they will never stop. However, I have decided to stop referring to them as warmists. Now I will simply call them liars.
Yes, there is a conspiracy to silence dissent, first by trying to prevent the publishing of reports which did not fit with the conspiracists’ viewpoint, and then to shut out or shut down journals which dared after all to print opposing viewpoints. This is not science, this is dirty politics.
Craig
Maybe the truth will now come out about climate change. The worst floods for a thousand years – so there has been more.
The warmest temperature since 19 so & so – again it has been hotter.
Science becomes what the mighty dollar tells us, those who are making the money from cliamte change must perpetuate the myth to justify their existence. lets all get to Copenhagen – but not by car, train, plane – just walk, naked, because all the doomsayers create more emissions in their constant world travels, producing of documentaries and endless rants than the rest of us who accept that the world changes, it lives and you are trying to stop our Mother Nature doing her thing.
Its a bit like our old Russia – you can say what you like but if we (the changers) disagree, we will humiliate and disgrace you.
Freedom has long since gone.
Why on earth didn’t they do a sensitivity analysis, via perturbation, on their results? There seems to be a lot of dividing a snowball by a moonbeam and expressing the result to six significant figures. Pergaps they havn’t heard of ill-conditioned models?
Richard Saumarez,
See yesterday’s story: the primer on the code and variability of the predictions.
To view the actual University of East Anglia’s Climate Research documents which were discovered on the internet showing various attempts to fool and manipulate the public regarding climate change search for the folder FOI2009.zip it is available on many file sharing sites or view at
http://forkbomb.org/~ml/cmail/
http://www.eastangliaemails.com/index.php
Amusing
http://www.youtube.com/watch?v=nEiLgbBGKVk&feature=player_embedded
All sorts of public policies are based on science from, health policy, vaccinations; mercury, fluoride, BGH, genetic modified foods, food irradiation, nuclear power … and these decisions require public trust. No one can know the cumulative affect censorship and deception has had. Scientists who disagreed were marginalized and insulted! This issue perhaps more than others has been spread by teachers and journalists to students and the public. This discussion changed national plans, impacted presidential election, and man’s sense of personal responsibilities to the world. Brilliant business people reviewed the opinions of these scientists and made plans based on misrepresentation. Researchers have hidden the facts that the world was possibly cooling for perhaps the last ten years while CO2 levels rose! How can science find truths if they bully and intimidate dissenters and hide facts? How can the public or businessmen back important public policy decisions? This sort of behavior unfortunately may be all too common. Among senior scientists in every field, this is a black eye. It is too soon too know how this will fall out but it is a reminder of the importance of integrity (not having separate public and private representations of research.)
I also am an applied statistician, and the first thing I would like to see from whatever climate models these climate scientists used is the residual analysis (analysis of the differences between the observed and modeled temperatures) and also tests for significance of the independent variables (especially so-called greenhouse gases). If the model, such as a time series with carbon dioxide and methane as regressor variables, is valid, the residuals will follow a normal distribution (“bell curve”). Of course, since they admitted to throwing away the raw data, I doubt that we will ever see this.
Kimberley Strassel meanwhile published an accusation in the Wall Street Journal, while naming a specific EPA supervisor (and therefore probably legally actionable if false and reckless), to the effect that the EPA suppressed research that does not support Team Obama’s cap and trade agenda. Suppression of findings that do not support a commercial or political agenda meets generally accepted definitions of academic dishonesty and scientific misconduct.
Senator Kirsten Gillibrand of New York wrote a letter to the Wall Street Journal that says openly that the purpose of cap and trade is to enrich Goldman Sachs, J.P. Morgan Chase (both of which bundled campaign contributions to elect her and Obama), the Green Exchange, and the New York Stock Exchange. All of them stand to profit from trading or speculation in carbon credits, today’s equivalent of medieval indulgences for sins.
It should now be clear that anthropogenic global warming is junk science, a 21st century Cardiff Giant or Piltdown Man, whose purpose is to enrich special interests at the expense of working Americans who will have to pay even more for energy and whose jobs will be destroyed by higher energy costs. The fact that some of the countries that complain the most loudly about global warming are burning down rain forests (carbon sinks) to clear land for agriculture shows that their concerns extend solely to lip service.
If the global warming alarmists do not like what I just said, they are free to prove me wrong. Let’s see the time series model, including the regressor variables and the raw data. Minitab is pretty good at handling time series models, and I recall using a much older version in night school for my time series class. Let’s put the numbers in and see what comes out in terms of tests for significance and residuals. This assumes, of course, that the CRU didn’t lose the necessary raw data.
In fact, I may even ask the CRU for the model and raw data so I can attempt to reproduce their results, which any engineer or scientist should be able to do if their results are valid.
The emails at East Anglia were not hacked….this is just “spin” which attempts to give a measure of cover to the true criminals here- the scientists involved in massive fraud. The computer technicians at East Anglia have release analysis reports…..the zip file containing the emails was compiled ON a computer AT the university, and was done only moments before the file was released onto the internet……furthermore, the file was released from that computer to MULTIPLE servers……both of these demonstrate that the emails were leaked by an insider blowing the whistle on the fraud…….LEAKED, NOT HACKED.
If the global warming alarmists do not like what I just said, they are free to prove me wrong. Let’s see the time series model, including the regressor variables and the raw data. Minitab is pretty good at handling time series models, and I recall using a much older version in night school for my time series class. Let’s put the numbers in and see what comes out in terms of tests for significance and residuals. This assumes, of course, that the CRU didn’t lose the necessary raw data.