You or I might perhaps be excused if we sometimes toyed with solipsism, especially when we reflect on the utter failure of our writings to produce the smallest effect in the alleged external world. —David Stove, “Epistemology and the Ishmael Effect.”
Statistics is broken. When it works, it usually does so in spite of itself. When it doesn’t, which is increasingly often, it inflates egos, promulgates scientism, idolizes quantification, supports ideologies, and encourages magical thinking.
I’m not going to prove any of that today (you’re welcome to read old posts for corroboration), but assume it. This is just a Friday rant.
I weep over the difficulty of explaining things. I can’t make what is obvious to me plain to others. Flaubert was right: “Human speech is like a cracked kettle on which we tap crude rhythms for bears to dance to, while we long to make music that will melt the stars.”
So most of the fault is mine. But not all of it.
Last week I had as a header this blurb: In Nate Silver’s book The Signal and the Noise: Why So Many Predictions Fail he says (p. 68) “Recently, however, some well-respected statisticians have begun to argue that frequentist statistics should no longer be taught to undergraduates.” That footnote recommended this paper.
Easy to say. Impossible to do. You cannot, in any university I know, teach unapproved material. There are exceptions for “PhD-level” courses and the like, where the air is thin and the seats never filled, but for undergraduates you must adhere to the party line. The excuse for this is circular: students must be taught what’s approved because what’s approved is what students must be taught.
The scheme does work, however, for material which resembles cookbook recipes. Rigid syllabuses are best for welding, accountancy, physics, and sharpshooting courses. That’s why the Army uses them. But they fail miserably in what used to be called the humanities, which I say includes probability; at least its philosophical side. Humanitarians see themselves as scientists these days. Only way to get funding, I guess. Skip it.
I don’t mean to swap Bayes with frequentism, at least not in the way most people think of Bayes. Problem is everybody learns Bayes after learning frequentism, which is like a malarial infection that can’t be shaken. Frequentists love to create hypotheses? So do Bayesians. Frequentists form an unnatural and creepy fascination with parameters? So too Bayesians. Frequentists point to the occult powers of “randomization”? Bayesians nervously follow suit. Effect is that there’s very little practical difference between the two methods. (Though you wouldn’t know it listening to them bickering.)
There is no cure for malaria. Best maneuver is to avoid areas where infections are prevalent. That unfortunately means learning probability and statistics outside those departments. There’s some hope they can be learnt from certain physicists, but a weak one. The lure of quantification is strong there, and the probability is incidental.
One can always wander to the website of some eccentric—a refugee from academia—but that isn’t systematic enough for lasting consequence.
I don’t have a solution. And what am I doing wasting my time wallowing? I have to finish my book.