Stephen Senn very kindly answered a post I wrote on p-values (Unsignificant Statistics: Or Die P-Value, Die Die Die) by sending me his “You May Believe You Are a Bayesian But You Are Probably Wrong” (in Rationality, Markets and Morals).
Since I will be teaching at Cornell these two weeks, and the topics are the same, I will use part of this time to answer his paper in depth.
It would be best to start here Subjective 
Probability
Senn went wrong before he even began, with his title: “You May Believe You Are a Bayesian But You Are Probably Wrong.” If you are only “probably wrong” about your belief then you also might be right. And if you were certainly wrong, then we would have a proof which says so. A proof is a string of deductions, i.e. a valid and sound argument, which begins with obviously true premises (agreed to by all) and ends at a proposition we must believe—even if we don’t want to.
Senn does not have, nor does he claim to have, a proof which shows being a Bayesian is certainly wrong. It is only his best guess that this philosophy is wrong. Probably wrong. So here we are, already at probability. What could Senn mean by his probabilistic statement “probably wrong”? (Besides the pun, I mean.) It can’t be any kind of frequentist statement, as in “I’ve collected a ‘random sample’ of Bayesian philosophies, itself embedded in an infinite sequence of such philosophies, and the mean of this sample (considering errors in theory equal to zero) tends towards zero.” That makes no kind of sense, as I’m sure Senn would agree, but it would have to if probability was frequentism.
Bayesian philosophy, at best, comes in a finite number of flavors. It could be that some of these are false (I agree subjectivism, as it is usually understood, is), but in no way can we imagine any individual theory as being embedded in an infinite sequence of theories, which is required for frequentist theory to hold. No: either we can prove each theory true or false, or our evidence is not (yet?) sufficient, and thus we are only probably sure each theory is true or false. This sounds like a Bayesian statement, no? (If so, do we fail because of self-reference? Well, no, because we can build this theory from simpler propositions.)
It could be that Senn took a subjective Bayesian tack when he formed his title, or perhaps he took a logical probability, or objective, Bayesian one. (Incidentally, I’ll call this latter theory LPB for short.) Or he could have meant some as yet unknown (or at least unidentified) theory. Whatever it was, it couldn’t have been frequentism, as shown.
His leading candidate is eclecticism (Senn is not frequentist), which is one of two things. One is no belief at all. It means “I’ll do whatever I want whenever it seems good to me.” There is no theory here to disprove, nor prove. To say “I’m an eclectic” this way means “I don’t want to argue for anything, just against things.” Since we go nowhere engaging with this “theory”, we pass on to number two. This is to say, “I’ll take a little of that, some of this, and some of the other.” Here we have several sub-theories. As such, this kind of eclecticism is actually a whole theory (the compilation of sub-theories) which might be true or false. Thus Senn might have used Bayes for his title and he might use frequentism for (say) dice tosses.
Senn recalls that Fisher himself was “skeptical” of attempts to unify probability. Hacking, another Big Cheese, in line with other well-aged curds, is of the same opinion. Why should we have a theory? Why not many? The obvious answer to this is that there is that which is true and that which is false and we should seek the truth. If it turns out a theory of probability works for all kinds of uncertainty, we’re stuck with it. If it must be that several theories are true, then we must accept them all. But it’s wrong to use desire or suspicion as proof there are many and not one theory.
Senn himself proved that frequentism is out (and forever) as a complete theory of probability because it cannot handle propositions like his “probably wrong.” But this isn’t proof that Bayes everywhere right; not yet. Senn’s later examples might be sufficient to show all versions of Bayes are wrong, in which case some other theory must be true.
But we’ll have to see next time, because we’re already out of space, and because next topic isn’t simple.
You write: “Senn himself proved that frequentism is out (and forever) as a complete theory of probability because it cannot handle propositions like his ‘probably wrong.'” Since when was frequentism (or Bayesianism, for that matter) *trying* to be a “complete theory of probability”? It’s an *application* of probability. Sure, application influences theory, even in the multi-step approach between applied statistics to theoretical statistics to math; and that’s well and good; but there’s value in keeping a clear firewall between the symbol-pushing that is probability, and the interpretation of those symbols that is statistics.
Jeremy,
Since the concept was first introduced, that’s when. Not all say it is a complete theory, but many do (Senn does not). These people we now know are wrong.
Saying frequentism is an “application of probability” doesn’t make sense, except in a limited way we’ll see in Part III. There is no such thing as an “application of probability” anyway, since probability just is the quantification of uncertainty.
No: we have an idea of what probability is, a “theory” if you like, and we’re trying to figure out if this idea is one or multi-part. I say one, Senn says multi-part.
But I’m with you as far as “symbol-pushing” goes. Using symbols can ease understanding, but only after the basics are understood. If used before, they lead to misunderstanding.
Analysis and examples of probability started as frequentism, I concede. But the modern notion of mathematical probability — measuring sets in a way that happens to be convenient for many applications — doesn’t invoke uncertainty at all, and couldn’t care less whether it’s representing frequentist or Bayesian models of inference. Frequentism, Bayesianism, and the other -isms of statistics are additional baggage on top. Probability hardly seems worthwhile without one or the other, but it is distinct from each.
Pingback: Why is the Pope so Popular? - BIG PULPIT
The actual meaning Senn’s title can quite easily be interpreted as frequentist, whether or not it’s any good as a complete theory.
I was not arguing for frequentism. I was not even arguing against Bayesianism (in the sense of a theory as to how a perfect being should remain perfect). I have always maintained that the DeFinetti theory is extremely impressive. It would be nice to meet someone who uses it. I was arguing against ‘Bayesianism’ as a theory of how to become good. As regards eclecticism, I have never met a ‘Bayesian’ who at one point or another did not engage is arm-waving. Sure, there was some formal stuff going on but there was always a point at which some informal stuff came to the rescue of the formal.
Stephen,
Right! And thanks much for answering! You did not argue for frequentism, but for eclecticism. I’ll modify the text to make sure this is clear. For now, apologies for any misinterpretation.
As for hand waving, I think it’s done to generate a breeze to cool the necks of listeners who grow hot under their collars.
There’s more than one shade of Bayesian, so as for formal vs. informal, stick around.
Pingback: This Week in Reaction | The Reactivity Place