Book review

Preface to Uncertainty: The Soul of Modeling, Probability & Statistics

unc2

The book is now at a 15% discount at Amazon (I apologize for the price). Buy today. And buy again tomorrow! This will be linked permanently on the book’s official page. Springer has once again changed prices to match Amazon. Amazon now has a Kindle version. As of this post, there have been almost 900 downloads at Springer.

Fellow users of probability, statistics, and computer “learning” algorithms, physics and social science modelers, big data wranglers, philosophers of science, epistemologists; other respected citizens. We’re doing it wrong.

Not completely wrong; not everywhere; not all the time; but far more often, far more pervasively, and in far more areas than you’d imagine.

What are we doing wrong? Probability, statistics, causality, modeling, deciding, communicating, uncertainty. Everything to do with evidence.

Your natural reaction will be—this is a prediction based on plentiful observations and simple premises—“Harumph.” I can’t and shouldn’t put a numerical measure to my guess, though. That would lead to over-certainty, which I will prove to you is already at pandemic levels. Nor should I attempt to quantify your harumphiness, an act which would surely contribute to scientism, which is when pseudo-numerical values assigned to mental states is taken as scientific.

Now you may well say “Harumph”, but consider: there are people who think statistical models prove causality or the truth of “hypotheses”, that no probability can be known with certainty until the sound of the last trump, that probabilities can be read from mood rings, that induction is a “problem”, that randomness is magic, that chance is real, that parameters exist, that p-values validate or invalidate theories, that computers learn, that models are realer than observations, that model fit is more important than model performance.

And that is only a sampling of the oddities which beset our field. How did we go awry? Perhaps because our training as “data scientists” (the current buzzword) lacks a proper foundation, a firm philosophical grounding. Our books, especially our introductions, are loaded with a legion of implicit metaphysical presumptions, many of which are false or which contradict one another. The student from the start is plunged into formula and data and never looks back; he is encouraged not to ask too many questions but instead to calculate, calculate, calculate. As a result, he never quite knows where he is or where he’s going, but he knows he’s in a hurry.

The philosophical concepts which are necessarily present aren’t discussed well or openly. This is only somewhat rectified once, and if, the student progresses to the highest levels, but by that time his interest has been turned either to mathematics or to solving problems using the tools with which he is familiar, tools which appear “good enough” because everybody else is using them. And when the data scientist (a horrid term) finally and inevitably weighs in on, say, “What models really are”, he lacks the proper vocabulary. Points are missed. Falsity is embraced.

So here is a philosophical introduction to uncertainty and the practice of probability, statistics, and modeling of all kinds. The approach is Aristotelian. Truth exists, we can know it, but not always. Uncertainty is in our minds, not in objects, and only sometimes can we measure it, and there are good and bad ways of doing it.

There is not much sparkling new in this presentation except in the way the material is stitched together. The emphasis on necessary versus local or conditional truth and the wealth of insights that brings will be unfamiliar to most. A weakness is that because we have to touch on a large number of topics, many cannot be treated authoritatively or completely. But then the bulk of that work has been done in other places. And a little knowledge on these most important subjects is better than none, the usual condition. Our guiding light is Thomas Aquinas, ora pro nobis, who said, “The smallest knowledge that may be obtained of the highest things is more desirable than the most certain knowledge obtained of lesser.” It is therefore enough that we form a fair impression of each topic and move onward. The exceptions are in understanding exactly what probability is and, as importantly, what it is not and in comprehending just what models are and how to tell the good from the bad.

This isn’t a recipe book. Except for simple but common examples, this book does not contain the usual lists of algorithms. It’s not that I didn’t want them, it’s more that many proper ones don’t yet exist, or aren’t well understood; and anyway, they can be a distraction. This book is, however, a guide on how to create such recipes and lists, as well as a way to shoehorn (when possible) older methods into the present framework when new algorithms haven’t yet been created. This book is thus ideal for students and researchers looking for problems upon which to work. The mathematical requirements are modest: this is not a math book. But then probability is not a mathematical subject, though parts of it are amenable to calculation.

Some will want to know what to call this unfamiliar new theory. Well, it isn’t a theory. It is The Way Things Are. The approach taken is surely not frequentist, a method which compounds error upon error, but it is also not Bayesian, not in the usual sense of that term, though it is often close in spirit to objective Bayesianism. There is no subjectivism here. The material here is closely aligned to Keynes’s, Stove’s, and Jaynes’s logical probability. Many elements from the work of these and similar gentlemen are found here, but there are also subtle and important differences. If a name must be given, Probability As Argument is as good as any, though I prefer simply Probability.

If we’re doing it wrong, what’s right? Return to understanding cause. Models should be used to make probabilistic predictions of observable entities. These predictions can, in turn, be used to make decisions. If the predictions fail, the models fail and should be abandoned. Eliminate all forms of hypothesis testing, Bayesian or frequentist, and forever banish p-values, which only serve to confirm biases. Do not speak of parameters; talk of reality, of observables. This alone will go miles towards eliminating the so-called replication crisis.

Here is the book in brief. All truth is conditional on or with respect to something. There are thus necessary or universal and conditional or local truths. Truth resides in the mind, and not in objects except in the sense that objects exist or not. Truth is not relative in the modern sense of that word. Probability aims at truth. We come to know many truths via induction, which is widely misunderstood and is not a “problem”; indeed, it provides the surest form of knowledge. Logic is the study of the relationship between propositions, and so is probability. All probability, like all truth, is therefore are know because of the conditions assumed.

Most probability is not quantifiable, but some is. Probability is not subjective, and limiting relative frequency is of no use to man or beast. Chance and randomness are not mystical entities or causes; they are only other words for ignorance. Science is of the empirical. Models—whether quantum mechanical, medical, or sociological—are either causal or explanative. Causal models, which are in reality as rare as perfect games (baseball, of course), provide certainty, and explanative models state uncertainty. Probabilistic models are thus not causal (though they may have causal elements).

Bayes is not what you think. Hypothesis testing should immediately and forever be tossed onto the scrap heap of intellectual history and certainly never taught to the vulnerable. Probability is not decision. The parameter-centric, even parameter-obsessed, way of thinking about models must also be abandoned; its use has led to widespread, enormous over-certainty and caused more than one soul to be lost to scientism. Its replacement? If somebody asks, “How does changing X change my uncertainty in Y” tell them that and nothing else. Models, which provide the basis of these statements, are and must be checked against reality. The best way to check against reality is conditional on the decisions to which models are put. The most common, widespread errors that come in failing to not treating probability logically are shown, including the common mistakes made in regression, risk measures, the over-reliance on questionnaires, and so on.

The language used in this book will not be familiar to regular users of probability and statistics. But that is rather the point. It ought to be. Along the way we’ll solve thing like induction, Gettier “problems”, Grue, the Doomsday Argument, so-called paradoxes in probability assignment, the reproducibility crisis, and much more.

How working statisticians and probabilists should read this book. Start with Chapter on Chance and Randomness, then read the four successive Chapters on Causality, Probability Models, Statistical & Physical Models and Modelling Strategy & Mistakes. After this, return to the beginning for the proofs of the assumptions made in those Chapters.

Everybody else, and in particular students, should start at the beginning.

Manhattan Island, William M. Briggs, December 2015

Categories: Book review, Statistics

11 replies »

  1. I’m assuming you keep mentioning discounts for? Those of us who bought the book bought it because we wanted to read it and the price was what it was. Discounts are for people who “might” want to read it? Guess I’m seeing too much typical “marketing” here, something I had hoped would not come to a subject such as this one. There, I’ve vented and I’m better now. 🙂

  2. Sheri,

    I have zero idea how the prices work. Amazon seems to be steadily dropping the price. Springer had the high price up until recently, then today I noticed this new, lower price. No clue why the change.

    I earlier asked my editor for a lower price, but he said price was set by corporate and he had no control.

    So there you are. Still too expensive.

  3. “Still too expensive.” And yet many of us purchased it. My point is those who are interested in the book paid what it cost at the time. Discounts or no discounts.

  4. Prices work by subjective valuation. I ask myself, is what I am getting worth more to me, right now, than what I have to supply in exchange to get it. A seller asks his price. The buyer says yes or no. If bidding or negotiation is in play, this will be repeated until both sides find the terms agreeable, if they can. If they can’t, there’s no sale. NB this is predicated on there being no force or fraud on the part of either or both actors.

    If there are buyers at the original set price maybe more will buy as the ‘sunk’ costs get covered and the seller is getting his internal rate of return on his investment. Sales people will tell you that you always ask for a higher price than the one that is ultimately acceptable, for if you don’t, you’ll never get your price.

    In other words, don’t sweat it.

  5. All,

    I am of course deeply grateful to anybody who buys the book at any price. Thank you.

  6. Not trying to be nit-picky, but I didn’t get this:

    “All probability, like all truth, is therefore are know because of the conditions assumed.”

    I think your enemies are still at work. 🙂

  7. Phil R,

    Thank God that one was fixed in the final draft. This typo was placed by my enemies in my local code. They still haven’t given me back the final LaTeX.

  8. I meant to comment on yesterday’s post but didn’t have the opportunity. I’ll stick a boot in here that might be vaguely relevant to both.

    There is no certainty (conviction) that can match the “certainty” of an ideological prejudice. I contend that a probability is held “valid” or indicative if the function can be made to seem to support the ideology. I refer to scientism’s #1 dogma… that is that everything is in the state of “becoming”.

    Lip service is paid to the notion of “reality” where it does not conflict with the ideology to give the impression that the ideology must conform to reality. This is salesmanship, not the logic of science or the science of logic.

    Hypothesis testing is an evil that must be abandoned unless the hypothesis is a model that can be “verified” by comparing it to other models and the ideology.

    The mere fact that scientism’s dogma of “becoming” cannot be reconciled with observations of Natura Laws or with the logical certainties of philosophy is deemed irrelevant… who needs testable hypotheses if you can come up with a model that says what you want to hear?

    Like I quoted some drongo in a previous post; who needs proof if you’ve got “credible theories”?

  9. I pre-ordered, paid full price and just the discussion on parameters on pp. 143-151 is worth the price of admission. That discussion actually caused me to realize or “re-realize” that after all the contortions frequentists go through to justify wee-P-Values, the overarching weakness is that p values usually pertain to hypotheses about…..parameters. So even if they could patch them up, the result would be what someone once called a “Type 4” error: providing a clever solution to the wrong problem…..

    Congratulations to Briggs on a great job. It will take me a long while to fully read, digest and come back with questions and issues, and I already knew De Finnetti.

  10. So give some actual real examples of your amazing method in practice already, not just ever more philosophy or talking down of frequentist p-values and Bayesian stuff. That would be productive.

    Severe testing and error philosophy by Deborah Mayo et al. seems better as a more developed way of moving forward IMO.

  11. Dill,

    You must have missed the book page, conveniently set into the main menu at the top of the page, where your excellent suggestion has in fact been taken up.

    And p-values should forever be banned in science. As proved in the book.

Leave a Reply

Your email address will not be published. Required fields are marked *