Prominent Academic Asks: When And How Did Statistics Lose Its Way? 

The question is not mine. It belongs to Philip Stark, who asked it of Deborah Mayo at a Berkeley lecture. Stark plainly asked five good questions:

1. When and how did Statistics lose its way and become (largely) a mechanical way to bless results rather than a serious attempt to avoid fooling ourselves and others?
2. To what extent have statisticians been complicit in the corruption of Statistics?
3. Are there any clear turning points where things got noticeably worse?
4. Is this a problem of statistics instruction ((a) teaching methodology rather than teaching how to answer scientific questions, (b) deemphasizing assumptions, (c) encouraging mechanical calculations and ignoring the interpretation of those calculations), (d) of disciplinary myopia (to publish in the literature of particular disciplines, you are required to use inappropriate methods), (e) of moral hazard (statisticians are often funded on scientific projects and have a strong incentive to do whatever it takes to bless “discoveries”), or something else?
5. What can academic statisticians do to help get the train back on the tracks? Can you point to good examples?

I have the answers.

1. Statistics went sour a century ago, at its beginning. The seeds of it becoming a “mechanical way to bless” were then planted, sowed in large part by the bombastic efforts of one brilliant man, RA Fisher. It was he who invented the “P value.”

Which became magic, religion. A wee P blesses a study. A large P causes weeping and gnashing of teeth.

But it is absurd. It is ridiculous. It answers no questions anybody wants to know. None. Not one. Everything you think a P does, it does not.

What is the mighty magical P? The probability of seeing data you didn’t see, assuming something nobody believes.

One example only, since I’ll be doing this to death in the Class: you want the probability your new drug beats the old one. You could just calculate this, using probability. But no. Let’s calculate a P instead! You assume the drugs aren’t any different, then you calculate the probability of the value of an ad hoc statistic (based on a parameter inside a model) you didn’t see. If that’s wee, you claim the new drug is better. A fallacy. Every time a fallacy. Every time.

But a beloved fallacy, because why? Because Ps do the thinking for you. You are relieved of the burden of thought, and instead seek a score, which is all you need to claim your correlation turned into causation.

2. Statisticians are the direct cause of the crisis they brought upon us all. It was they who gifted us with the replication crisis, which will with us until (a) P values are dumped, and (b) parameter-centric analysis is expunged.

Parameters are the tunable dials inside models, if you will. They are so mathematically delightful, statisticians talk only of them, and forget–they forget—why they were doing an analysis in the first place.

One example, and the same example. What is the probability your new drug is better than the old? You could just calculate it. But no. Let me instead amaze you with the estimate of a parameter inside a model, and its confidence interval, a parameter which I will mistakenly call the real thing of interest, and imply that reality is less important than the parameter. It isn’t.

3. Things got worse with the introduction of statistics software, which anybody could use, and which everybody did. The mad hunt for the wee P was on.

4. The problem indeed is of instruction. It is the failure to emphasize philosophy to prioritize blind use of mathematics. Probability is no more a branch of mathematics than physics is. Both make heavy use of math, and should, but the true interest in both of these subjects is what is going on, not math. An equation doesn’t mean anything: the idea behind the equation does.

Physics uses equations in service of ontology, about the way the world works. Probability uses equations in service of epistemology, about our knowledge of the world. The two, epistemology and ontology, are not the same. Both need each other.

Do not teach casual students the math. Teach them the ideas. Do not teach serious students the math until they damn sure understand the ideas.

(I would never be allowed to teach my Class in a statistics department, for instance, without agreeing also to teach introductory classes which insist what I say is false are true.)

5. Academics could help by trashing the old bad ideas. But it won’t happen. It almost never does. What does happen is one funereal at a time. New subjects, like “AI” and “machine learning”, rise up and take the place of decrepit, flawed and moribund subjects.

There are many excesses and lacunae in AI, which are nothing but probability models after all, married to wonderful increases in storage and processing. These excesses and gaps will need correction, too, but the subject is young enough that many are still ignoring them.

Subscribe or donate to support this site and its wholly independent host using credit card click here. Or use the paid subscription at Substack. Cash App: \$WilliamMBriggs. For Zelle, use my email: matt@wmbriggs.com, and please include yours so I know who to thank.