Jane Orient is the lead doc at Association of American Physicians and Surgeons, publishers of the journal.
Uncertainty: The Soul of Modeling, Probability & Statistics, by William Briggs, hardcover, 258 pp, $59.75, ISBN 978-3-319-39759-9, Springer International Publishing Switzerland, 2016.
This book has the potential to turn the world of evidence-based medicine upside down. It boldly asserts that with regard to everything having to do with evidence, we’re doing it all wrong: probability, statistics, causality, modeling, deciding, communicating—everything. The flavor is probably best conveyed by the title of one of my favorite sections: “Die, p-Value, Die, Die, Die.”
Nobody ever remembers the definition of a p-value, William Briggs points out. “Everybody translates it to the probability, or its complement, of the hypothesis at hand.” He shows that the arguments commonly used to justify p-values are fallacies. It is far past time for the “ineradicable Cult of Point-Oh-Five” to go, he states. He does not see confidence intervals as the alternative, noting that “nobody ever gets these curious creations correct.”
Briggs is neither a frequentist nor a Bayesian. Rather, he recommends a third way of modeling: using the model to predict something. “The true and only test of model goodness is how well that model predicts data, never before seen or used in any way. That means traditional tricks like cross validation, boot strapping, hind- or back-casting and the like all ‘cheat’ and re-use what is already known as if it were unknown; they repackage the old as new.”
Yes, this book is about probability and statistics, and there is some mathematics in it, but fundamentally it is a book of philosophy. If you follow his blog, wmbriggs.com, you will recognize that he is a devotee of Thomas Aquinas.
The book discusses science and scientism, and belief and knowledge. Chapter Two is about logic, with delightful examples from Charles Dodgson (Lewis Carroll). Briggs states that an entire branch of statistics, hypothesis testing, is built around the worst fallacy, the “We-Have-To-Do-Something Fallacy.”
Some of the book’s key insights are: Probability is always conditional. Chance never causes anything. Randomness is not a thing. Random, to us and to science, means unknown cause.
One fallacy that Briggs chooses for special mention, because it is so common and so harmful, is the epidemiologist fallacy. He prefers his neologism to the more well-known “ecological fallacy” because without this fallacy, “most epidemiologists, especially those employed by the government, would be out of a job.” It is also richer than the ecological fallacy because it occurs whenever an epidemiologist says “X causes Y” but never measures X. Causality is inferred from “wee p-values.” One especially egregious example is the assertion that small particulates in the air (PM 2.5s) cause excess mortality.
Quantifying the unquantifiable, which is the basis of so much sociological research, creates a “devastation to sound argument…[that] cannot be quantified.” Briggs deconstructs famous examples and the “instruments” they use.
We suffer from a crisis of over-certainty, Briggs writes. He believes we need a science that is “not quite so dictatorial and inflexible, one that is calmer and in less of a hurry, one that is far less sure of itself, one that has a proper appreciation of how much it doesn’t know.”
Statistical significance should be “treated like the ebola virus,” he writes, “i.e. it should be placed in a tightly guarded compound where any danger can be contained and where only individuals highly trained in avoiding intellectual contamination can view it.”
Briggs will not be well loved by those who write “evidence-based” papers replete with parameters, regressions, and p-values. Those who study Briggs will no longer be overawed by such papers, however prestigious the journal that publishes them. But he validates the importance of first-hand observation, insight, and intuition. To my mind, he shows that the need for the art of medicine is proven by the science.
Despite its heavy subject matter, the book is full of humor and a delight to read and re-read.
Jane M. Orient, M.D.