Our friend Christos Argyropoulos (â€@ChristosArgyrop) to a popular medical site in which Stephen Reznick asks “Keep statistics simple for primary care doctors.”
He can’t read the journals, because why? Because “Medical school was a four year program. The statistics course was a brief three week interlude in the midst of a tsunami of new educational material presented in a new language…While internship and residency included a regular journal club, there was little attention paid to analyzing a paper critically from a statistical mathematical viewpoint.”
Reznick has been practicing for some time and admits his “statistical analysis skills have grown rusty…When the Medical Knowledge Self Assessment syllabus arrives every other year, the statistics booklet is probably one of the last we look at because not only does it involve re-learning material but you must first reâ€“learn a vocabulary you do not use day to day or week to week.”
What he’d like is for journals to “Let authors and reviewers say what they mean at an understandable level.”
Now I’ve taught and explained statistics to residents, docs fresh out of med school, for a long time. And few to none of them remember the statistics they were taught either. Why should they? Trying to squeeze a chi-square test among all those muscles and blood vessels they must memorize isn’t easy, and not so rewarding either.
Medical students learn why the ankle bone is connected to the ulniuous, or whatever the hell it is, and what happens when this or that artery is choked off. Useful stuff—and all to do with causality. They never learn why the chi-square does what it does. It is presented as mystery, a formula or incantation to invoke when the data take such-and-such a form. Worse, the chi-square and all other tests have nothing to do with causality.
A physician reading a journal article about some new procedure asks himself questions like, “What is the chance this would work for patients like mine?”, or “If I give my patients this drug, what are the chances he gets better?”, or “How does the cure for this disease work?” All good, practical, commonsense queries.
But classical statistics isn’t designed to answer commonsense questions. In place of clarity, we have the “null” and “alternate” hypotheses, which in the end are nothing but measures of model fit (to the data at hand and none other). Wee p-values are strewn around papers like fairy dust. What causes what cannot be discovered, but readers are invited to believe what the author believes caused the data.
I’ve beat this drum a hundred times, but what statistical models should do is to predict what will happen, given or conditioned on the data which came before and the premises which led to the particular model used. Then, since we have a prediction, we wait for confirmatory, never-observed-before data. If the model was good, we will have skillful predictions. If not, we start over.
“But, Briggs, that way sounds like it will take longer.”
True, it will. Think of it like the engineering approach to statistics. We don’t rely on theory and subjectively chosen models to build bridges or aircraft, right? We project and test. Why should we trust our health to models which have never been been put through the fire?
One benefit would be a shoring up of the uncertainty of side effects, especially the long-term side effects, of new drugs. Have you seen the list of what can go wrong when you eat one of these modern marvels? Is it only us civilians who cringe when hearing “suicide” is a definite risk of an anti-depressant? Dude. Ask your doctor if the risk of killing yourself is right for you.
What the patient wants to know is something like, “If I eat this pill, what are the chances I’ll stroke out?” The answer “Don’t worry” is insufficient. Or should be. How many medicines are released only to be recalled because a particular side effect turned out more harmful than anticipated?
“Wouldn’t your scheme be difficult to implement?”
It’s a little known but open secret that every statistical model in use logically implies a prediction of new data. All we have to do is use the models we have in that way. This would allow us to spend less time talking about model fit and more about the consequences of particular things.
“What are the chances people will switch to this method?”