Frequenstism Is Entirely Ad Hoc; And So Are Priors. Use The Third Way Instead

@A_NEVATHIR @Noahpinion @mattstat Sticking to logical probability doesn't tell us about the world, where do initial assignments come from?.

— ?Deborah Mayo? (@learnfromerror) September 1, 2015

Short note today, because today, and most days for the next couple of weeks, are book days. I am nearly finished with the damned thing. I’m now handling drudgery like the bibliography and rooting out typos (yes). And also tying the thing together so that all my terms and examples are consistent.

It’s nearly there. I think the editor I contacted, who was initially enthusiastic, must have looked me up on line and discovered my, um, academic non-conformities, because I haven’t had any email answered from her for over eight months. So once I’m done done, I’ll go searching for another. But I’ll also send a draft copy of the book to selected colleagues for comment.

In the book, incidentally, there is no global warming, no ethics, no (let us call them) social questions. It’s pure philosophy of probability and statistics, it’s all applied epistemology.

As you can see from the tweet above (if you can’t see it, click here), I was dragged into a discussion about p-values. Mayo is a respected philosopher of probability, but she takes the frequentist line and objects to Bayesian procedures because, she claims, priors are ad hoc.

She’s right: they are. But then, with even greater force, so are frequentist models, which raise ad hocness to an art form—but one resembling the graffiti at the backs of grocery stores. Seems to me, if you can swallow regressions—used for everything—you can buy flat priors for the parameters of that regression.

And you should. Swallow them, I mean. Because, as Mayo knows, a regression with flat priors gives the same numerical results as frequentism. The same.

Thus the battle between frequentists and Bayesians is a fight over a territory we logical probabilists have long abandoned. This is why I advocate (the title is goofy) The Third Way Of Probability & Statistics: Beyond Testing and Estimation To Importance, Relevance, and Skill.

This is logical probability, where models can be deduced, where the origin of parameters is made clear, and where parameters don’t even exist, unless one heads out the limit. The Third Way recognizes that we will never be able to eliminate all ad hocness, so the attention is turned from an arbitrary model’s innards to its actual performance.

This has the added benefit of being natural and easy to explain to users of our models. We speak in plain English. And since our models are exposed to the world, they can be verified by anybody. Plus, some of our models are deduced, and therefore impeccable.

The focus in the Third Way is on understanding cause, and understanding probability isn’t cause. Now that sounds mysterious because cause is deeply misunderstood in probability and statistics. The old way—hypothesis testing and Bayes factors—confuse decision and probability and thus mix up cause.

Best part of the Third Way is the dramatic reduction of over-certainty, which is now at pandemic levels. Hence the “replication crisis”, among other calamities. The Third Way, i.e. logical observable-and-not-parameter-based probability,

Notice I said “reduction” and not “elimination.” No program can do that. The old methods pretend they can, though. They claim to have discovered truth or the “optimal” action (hence the confusion of decision and probability implicit in hypothesis testing etc.), but logical probability forces due consideration of all uncertainties.

Anyway, there you are. You can read my Arxiv papers for a small taste of the book, but only a very small taste.

Incidentally, I’m available to speak on these topics. Even, possibly, for no cost.