Andrew Montford, who runs Bishop Hill blog, had a post titled “IPCC statistics ruled illegal” in which he said, “Bayesian statistics, the approach favoured by the IPCC in its assessments of the world’s climate, has been ruled illegal by the Appeal Court in London.”
Montford formed this opinion from a piece by David Spiegelhalter, who is a named person in statistics, and who writes at the Understanding Uncertainty blog. Spiegelhalter’s “Court of Appeal bans Bayesian probability (and Sherlock Holmes)” concluded the English Court of Appeal “denied that probability can be used as an expression of uncertainty for events that have either happened or not.” Spiegelhalter worried for his students.
I’m not, as Rumpole would say, though readers might dispute, a “Queer Customer”, i.e. a Queen’s Counsel, a sort of superior barrister, but my reading of the case is that, while one judge did make a mistake, Bayesian philosophy has not been ruled illegal.
The Court of Appeal’s decision tells us an electrician named Nulty was working on electrical equipment at a recycling center and, without finishing, went to a canteen for a smoke break. He returned to find the center on fire. Where there’s smoke, there’s fire, and where there’s fire, there’s lawsuits.
The judge of the original case decided that the equipment, left in an imperfect state, was not likely to have caused the fire, but that a discarded cigarette butt was more likely to have done. He also ruled that the cigarette butt belonged to Nulty, and thus Nulty was the cause of the fire. (Nulty was dead by the time the case came to court.)
Where there’s lawsuits, there’s appeals, and where there’s appeals that involve Bayesian philosophy, there’s blog posts.
The Court of Appeals’s decision was to reject the appeal and to conclude the original judge had done a proper job. The original judge used (thank the Lord) non-quantitative Bayesian reasoning, perhaps imperfectly, but Bayes nonetheless. Since it was Bayes, and since the Court of Appeals upheld the trial judge’s decision, Bayes is in no danger.
All parties agreed that the cigarette, electrical equipment, or perhaps a vague something else (intruder, random arsonist, etc.) caused the fire, though the mysterious intruder theory was eventually abandoned. How they arrived at this is not of direct interest to us. The original judge said that, considered by itself, it was unlikely the cigarette caused the fire. And considered by itself, it was unlikely the equipment did. But given a choice between butt and equipment, the butt was much more likely.
That’s Bayesian reasoning all right. (Bayes is forming probabilities assuming certain fixed evidence: when the evidence changes, the probabilities change.)
If our evidence is that we have a 1,000-sided object with just one side labeled “fire” and an 10,000-sided object with just one side labeled “fire”, and if one side on both must show when tossed, then the Bayesian probability the 1,000-sided object shows “fire” is 1/1000, which can be called unlikely. It is also unlikely the 10,000-sided object would show “fire”; indeed, more unlikely.
But if I now add evidence that one and only one of these objects showed “fire”, but I do not tell you which, then you are right to guess it was the 1,000-sided object; it is 10 times likelier.
The appellate judge quoted a case in which Sherlock Holmes appeared: “when you have eliminated the impossible, whatever remains, however improbable must be the truth?” This is Bayesian, too, but in a more complicated, subtle way. Too subtle to do all in one post, so we shall save it for another day.
What rankled Spiegelhalter was when the appellate judge said:
The chances of something happening in the future may be expressed in terms of percentage. Epidemiological evidence may enable doctors to say that on average smokers increase their risk of lung cancer by X%. But you cannot properly say that there is a 25 per cent chance that something has happened: Hotson v East Berkshire Health Authority  AC 750. Either it has or it has not.
This is wrong (in part). A doctor, citing certain evidence, can say a patient has an X% chance of cancer. But we can form probabilities for any proposition, even those concerning events which happened in the past. For example, ten minutes ago (longer by the time you read this) I flipped a coin. Do you know which side landed? You do not; you are uncertain; you express this uncertainty with probability. Saying “it was a head or it wasn’t” is a tautology, the probability of which is 1, or 100%. Adding tautologies to evidence never changes any probability conclusion.
It’s obvious this judge made his blunder because of his (entirely reasonable) suspicion of quantification; he speaks of the real “danger of pseudo-mathematics”. The discussion of the “balance of probability” test referenced in the appeal came too close to quantification for him, hence his perfunctory statement. However it is clear he prefers (Bayesian) probability done without numbers, which is plain in his opinion, in which he restated that the trial judge’s (Bayesian) reasoning was spot on. (Two brother judges concurred.)
Update “Bayes” was nowhere mentioned in the opinion which, if you’re looking for cheering news, is well written, concise, jargon free, and serious. It’s a pleasure to see these judges’ honest struggle to discover the truth.
Thanks to Gareth, Mike Gee, Roger Cohen, K.A. Rodgers, and the others who suggested this post.