Le livre, he is done! I yesterday sent a proposal to a philosophy editor at Cambridge. Not enough equations or pictures in it to pique the interest of the statistics editor, I guess.
Thanks to those who volunteered to copy edit. I may still call on your services, especially if I can’t find a publisher who isn’t interested in it or me (I have eccentricities) and want to self publish.
Here is the Preface (the Author’s Forward has all the acknowledgements and thanks). I’ll also in time put up pieces of the proposal so that you can see details of each chapter. (The Preface doesn’t reveal much.)
P.S. I have been neglecting my email in order to finish, so if I haven’t answered yours (which is likely, since I have about 200 to answer), this is why.
Update Shot down at Cambridge. On to the next! I just sent query to Oxford. And Springer. And MIT press. And Wiley.
Dear Professor Briggs
Thanks for sending me the details relating to your proposed project ‘The Philosophy of Uncertainty’. The topic is an interesting one, but I’m afraid I don’t think that the style and approach of this book would be suited to the Cambridge list. I’m sure you will find that other publishers think differently, though, so I hope you will try the project out on them. I hope you are successful in finding a home for it.
Best wishes —
Fellow users of probability, statistics, and computer “learning” algorithms, physics and social science modelers, big data wranglers, philosophers of science, epistemologists; other respected citizens. We’re doing it wrong.
Not completely wrong; not everywhere; not all the time; but far more often, far more pervasively, and in far more areas than you’d imagine.
What are we doing wrong? Probability, statistics, causality, modeling, deciding, communicating, uncertainty. Everything to do with evidence.
Your natural reaction will be—this is a prediction based on plentiful observations and simple premises—“Harumph.” I can’t and shouldn’t put a numerical measure to my guess, though. That would lead to over-certainty, which I will prove to you is already at pandemic levels. Nor should I attempt to quantify your harumphiness, an act which would surely contribute to scientism.
Now you may well say “Harumph”, but consider: there are people who think statistical models prove causality or the truth of “hypotheses”, that no probability can be known with certainty until the sound of the last trump, that probabilities can be read from mood rings, that induction is a “problem”, that randomness is magic, that parameters exist, that p-values validate theories, that computers learn, that models are realer than observations, that model fit is more important than model performance.
And that is only a sampling of the oddities which beset our field. How did we go awry? Perhaps because our training as “data scientists” (the current buzzword) lacks a proper foundation, a firm philosophical grounding. Our books, especially our introductions, are loaded with a legion of implicit metaphysical presumptions, many of which are false. The student from the start is plunged into formula and data and never looks back; he is encouraged not to ask too many questions but instead to calculate, calculate, calculate. As a result, he never quite knows where he is or where he’s going, but he knows he’s in a hurry.
The philosophical concepts which are necessarily present aren’t discussed well or openly. This is only somewhat rectified once, and if, the student progresses to the highest levels, but by that time his interest has been turned either to mathematics or to solving problems using the tools with which he is familiar, tools which appear “good enough” because everybody else is using them. And when the data scientist (a horrid term) finally and inevitably weighs in on, say, “What models really are”, he lacks the proper vocabulary. Points are missed. Falsity is embraced.
So here is a philosophical introduction to uncertainty and the practice of probability, statistics, and modeling of all kinds. The approach is Aristotelian. Truth exists, we can know it, but not always. Uncertainty is in our minds, not in objects, and only sometimes can we measure it, and there are good and bad ways of doing it.
There is not much sparkling new in this presentation except in the way the material is stitched together. The emphasis on necessary versus local or conditional truth and the wealth of insights that brings will be unfamiliar to most. A weakness is that because we have to touch on a large number of topics, many cannot be treated authoritatively or completely. But then the bulk of that work has been done in other places. And a little knowledge on these most important subjects is better than none, the usual condition. Our guiding light is St Thomas, ora pro nobis, who said, “The smallest knowledge that may be obtained of the highest things is more desirable than the most certain knowledge obtained of lesser.” It is therefore enough that we form a fair impression of each topic and move onward. The exceptions are in understanding exactly what probability is and, as importantly, what it is not and in comprehending just what models are and how to tell the good from the bad.
This isn’t a recipe book. Except for simple but common examples, this book does not contain the usual lists of algorithms. It’s not that I didn’t want them, it’s more that many proper ones don’t yet exist, or aren’t well understood; and anyway, they can be a distraction. This book is, however, a guide on how to create such recipes and lists, as well as a way to shoehorn (when possible) older methods into the present framework when new algorithms haven’t yet been created. This book is thus ideal for students and researchers looking for problems upon which to work. The mathematical requirements are modest: this is not a math book. But then probability is not a mathematical subject, though parts of it are amenable to calculation.
Some will want to know what to call this unfamiliar new theory. Well, it isn’t a theory. It is The Way Things Are. The approach taken is surely not frequentist, a method which compounds error upon error, but it is also not Bayesian, not in the usual sense of that term, though it is often close in spirit to objective Bayesianism. There is no subjectivism here. The material here is closely aligned to Keynes’s, Stove’s, and Jaynes’s logical probability. Many elements from the work of these and similar gentlemen are found here, but there are also subtle and important differences. If a name must be given, Probability As Argument is as good as any, though I prefer simply Probability.
If we’re doing it wrong, what’s right? Models should be used to make probabilistic predictions of observable entities. These predictions can, in turn, be used to make decisions. If the predictions fail, the models fail and should be abandoned. Eliminate all forms of hypothesis tests, which only serve to confirm biases. Do not speak of parameters.
Here is the book in brief. All truth is conditional on or with respect to something. There are thus necessary or universal and conditional or local truths. Truth resides in the mind, and not in objects except in the sense that they exist (or not). Truth is not relative in the modern sense of that word. Probability aims at truth. We come to know many truths via induction, which is widely misunderstood and is not a “problem”, indeed it provides the surest form of knowledge. Logic is the study of the relationship between propositions, and so is probability. All probability, like all truth, is therefore conditional.
Most probability is not quantifiable, but some is. Probability is not subjective, and limiting relative frequency is of no use to man or beast. Chance and randomness are not mystical causes; they are only other words for ignorance. Science is of the empirical. Models—whether quantum mechanical, medical, or sociological—are either causal or explanative. Causal models provide certainty, and explanative models uncertainty. Probabilistic models are thus not causal (though they may have causal elements).
Bayes is not what you think. Hypothesis testing should immediately and forever be tossed onto the scrap heap of intellectual history and certainly never taught to the vulnerable. Probability is not decision. The parameter-centric, even parameter-obsessed, way of thinking about models must also be abandoned; its use has lead to widespread, enormous over-certainty and caused more than one soul to be lost to scientism. Its replacement? Models which are and must be checked against reality. The best way to check against reality is conditional on the decisions to which models are put. The most common, widespread errors that come in failing to understand not treating probability logically are shown, including the common mistakes made in regression, risk measures, the over-reliance on questionnaires, and so on.
The language used in this book will not be familiar to regular users of probability and statistics. But that is rather the point. It ought to be.
How working statisticians and probabilists should read this book. Start with Chapter on Probability Models, then read the two successive Chapters on Statistical & Physical Models and Modelling Strategy & Mistakes. After this, start at the beginning for the proofs of the assumptions made in those Chapters.
Everybody else, and in particular students, should start at the beginning.