Preface to Uncertainty: The Soul of Modeling, Probability & Statistics

The book is now at a 15% discount at Amazon (I apologize for the price). Buy today. And buy again tomorrow! This will be linked permanently on the book’s official page. Springer has once again changed prices to match Amazon. Amazon now has a Kindle version. As of this post, there have been almost 900 downloads at Springer.

Fellow users of probability, statistics, and computer “learning” algorithms, physics and social science modelers, big data wranglers, philosophers of science, epistemologists; other respected citizens. We’re doing it wrong.

Not completely wrong; not everywhere; not all the time; but far more often, far more pervasively, and in far more areas than you’d imagine.

What are we doing wrong? Probability, statistics, causality, modeling, deciding, communicating, uncertainty. Everything to do with evidence.

Your natural reaction will be—this is a prediction based on plentiful observations and simple premises—“Harumph.” I can’t and shouldn’t put a numerical measure to my guess, though. That would lead to over-certainty, which I will prove to you is already at pandemic levels. Nor should I attempt to quantify your harumphiness, an act which would surely contribute to scientism, which is when pseudo-numerical values assigned to mental states is taken as scientific.

Now you may well say “Harumph”, but consider: there are people who think statistical models prove causality or the truth of “hypotheses”, that no probability can be known with certainty until the sound of the last trump, that probabilities can be read from mood rings, that induction is a “problem”, that randomness is magic, that chance is real, that parameters exist, that p-values validate or invalidate theories, that computers learn, that models are realer than observations, that model fit is more important than model performance.

And that is only a sampling of the oddities which beset our field. How did we go awry? Perhaps because our training as “data scientists” (the current buzzword) lacks a proper foundation, a firm philosophical grounding. Our books, especially our introductions, are loaded with a legion of implicit metaphysical presumptions, many of which are false or which contradict one another. The student from the start is plunged into formula and data and never looks back; he is encouraged not to ask too many questions but instead to calculate, calculate, calculate. As a result, he never quite knows where he is or where he’s going, but he knows he’s in a hurry.

The philosophical concepts which are necessarily present aren’t discussed well or openly. This is only somewhat rectified once, and if, the student progresses to the highest levels, but by that time his interest has been turned either to mathematics or to solving problems using the tools with which he is familiar, tools which appear “good enough” because everybody else is using them. And when the data scientist (a horrid term) finally and inevitably weighs in on, say, “What models really are”, he lacks the proper vocabulary. Points are missed. Falsity is embraced.

So here is a philosophical introduction to uncertainty and the practice of probability, statistics, and modeling of all kinds. The approach is Aristotelian. Truth exists, we can know it, but not always. Uncertainty is in our minds, not in objects, and only sometimes can we measure it, and there are good and bad ways of doing it.

There is not much sparkling new in this presentation except in the way the material is stitched together. The emphasis on necessary versus local or conditional truth and the wealth of insights that brings will be unfamiliar to most. A weakness is that because we have to touch on a large number of topics, many cannot be treated authoritatively or completely. But then the bulk of that work has been done in other places. And a little knowledge on these most important subjects is better than none, the usual condition. Our guiding light is Thomas Aquinas, ora pro nobis, who said, “The smallest knowledge that may be obtained of the highest things is more desirable than the most certain knowledge obtained of lesser.” It is therefore enough that we form a fair impression of each topic and move onward. The exceptions are in understanding exactly what probability is and, as importantly, what it is not and in comprehending just what models are and how to tell the good from the bad.

This isn’t a recipe book. Except for simple but common examples, this book does not contain the usual lists of algorithms. It’s not that I didn’t want them, it’s more that many proper ones don’t yet exist, or aren’t well understood; and anyway, they can be a distraction. This book is, however, a guide on how to create such recipes and lists, as well as a way to shoehorn (when possible) older methods into the present framework when new algorithms haven’t yet been created. This book is thus ideal for students and researchers looking for problems upon which to work. The mathematical requirements are modest: this is not a math book. But then probability is not a mathematical subject, though parts of it are amenable to calculation.

Some will want to know what to call this unfamiliar new theory. Well, it isn’t a theory. It is The Way Things Are. The approach taken is surely not frequentist, a method which compounds error upon error, but it is also not Bayesian, not in the usual sense of that term, though it is often close in spirit to objective Bayesianism. There is no subjectivism here. The material here is closely aligned to Keynes’s, Stove’s, and Jaynes’s logical probability. Many elements from the work of these and similar gentlemen are found here, but there are also subtle and important differences. If a name must be given, Probability As Argument is as good as any, though I prefer simply Probability.

If we’re doing it wrong, what’s right? Return to understanding cause. Models should be used to make probabilistic predictions of observable entities. These predictions can, in turn, be used to make decisions. If the predictions fail, the models fail and should be abandoned. Eliminate all forms of hypothesis testing, Bayesian or frequentist, and forever banish p-values, which only serve to confirm biases. Do not speak of parameters; talk of reality, of observables. This alone will go miles towards eliminating the so-called replication crisis.

Here is the book in brief. All truth is conditional on or with respect to something. There are thus necessary or universal and conditional or local truths. Truth resides in the mind, and not in objects except in the sense that objects exist or not. Truth is not relative in the modern sense of that word. Probability aims at truth. We come to know many truths via induction, which is widely misunderstood and is not a “problem”; indeed, it provides the surest form of knowledge. Logic is the study of the relationship between propositions, and so is probability. All probability, like all truth, is therefore are know because of the conditions assumed.

Most probability is not quantifiable, but some is. Probability is not subjective, and limiting relative frequency is of no use to man or beast. Chance and randomness are not mystical entities or causes; they are only other words for ignorance. Science is of the empirical. Models—whether quantum mechanical, medical, or sociological—are either causal or explanative. Causal models, which are in reality as rare as perfect games (baseball, of course), provide certainty, and explanative models state uncertainty. Probabilistic models are thus not causal (though they may have causal elements).

Bayes is not what you think. Hypothesis testing should immediately and forever be tossed onto the scrap heap of intellectual history and certainly never taught to the vulnerable. Probability is not decision. The parameter-centric, even parameter-obsessed, way of thinking about models must also be abandoned; its use has led to widespread, enormous over-certainty and caused more than one soul to be lost to scientism. Its replacement? If somebody asks, “How does changing X change my uncertainty in Y” tell them that and nothing else. Models, which provide the basis of these statements, are and must be checked against reality. The best way to check against reality is conditional on the decisions to which models are put. The most common, widespread errors that come in failing to not treating probability logically are shown, including the common mistakes made in regression, risk measures, the over-reliance on questionnaires, and so on.

The language used in this book will not be familiar to regular users of probability and statistics. But that is rather the point. It ought to be. Along the way we’ll solve thing like induction, Gettier “problems”, Grue, the Doomsday Argument, so-called paradoxes in probability assignment, the reproducibility crisis, and much more.

How working statisticians and probabilists should read this book. Start with Chapter on Chance and Randomness, then read the four successive Chapters on Causality, Probability Models, Statistical & Physical Models and Modelling Strategy & Mistakes. After this, return to the beginning for the proofs of the assumptions made in those Chapters.

Everybody else, and in particular students, should start at the beginning.

Manhattan Island, William M. Briggs, December 2015