We’re finally getting it, as evinced by the responses to the article “Netherlands Temperature Controversy: Or, Yet Again, How Not To Do Time Series.”
Let’s return to the Screaming Willies. Quoting myself (more or less):
You’re a doctor (your mother is proud) and have invented a new pill, profitizol, said to cure the screaming willies. You give this pill to 100 volunteer sufferers, and to another 100 you give an identically looking placebo.
Here are the facts, doc: 72 folks in the profitizol group got better, whereas only 58 in the placebo group did.
Now here is what I swear is not a trick question. If you can answer it, you’ll have grasped the true essence of statistical modeling. In what group were there a greater proportion of recoverers?
This is the same question that was asked [before], but with respect to…temperature values. Once we decided what was meant by a “trend”—itself no easy task—the question was: Was there a trend?
May I have a drum roll, please! The answer to today’s question is—isn’t the tension unbearable?—more people in the profitizol group got better.
Probability models aren’t needed: the result is unambiguously 100% certain sure.
As before, I asked, what caused the difference in rates? I don’t know and neither do you. It might have been the differences due to profitizol or it might be due to many other things about which we have no evidence. All we measured was who took what substance and who got better.
What caused the temperature to do what it did? I don’t know that either. Strike that. I do know that it wasn’t time. Time is not a cause. Fitting any standard time series model is thus admitting that we don’t know what the cause was or causes were. This is another reason only to use these models in a predictive manner: because we don’t know the causes. And because we don’t know the causes, it does not follow that the lone sole only cause was, say, strictly linear forcing. Or some weird force that just happened to match what some smoother (running means, say) produced.
Probability isn’t needed to say what happened. We can look and see that for ourselves. Probability is only needed to say what might yet happen (or rather, to say things about that which we haven’t yet observed, even though the observations took place in the past).
Probability does not say why something happened.
I pray that you will memorize that statement. If everybody who used probability models recited that statement while standing at attention before writing a paper, the world would be spared much grief.
In our case, is there any evidence profitizol was the cause of some of the “extra” cures? Well, sure. The difference itself is that evidence. But there’s no proof. What is there proof of?
That it cannot be that profitizol “works” in the sense that everybody who gets it is cured. The proof is the observation that not everybody who got the drug was cured. There is thus similar proof that the placebo doesn’t “work” either. We also know for sure that some thing or things caused each person who got better to get better, and other causes that made people who were sick to stay sick. Different causes.
Another thing we know with certainty: that “chance” didn’t cause the observed difference. Chance like time is not a cause. That is why we do not need probability models to say what happened! Nothing is ever “due” to chance!
This is why hypothesis testing must go, must be purged, must be repulsed, must be shunned, must be abandoned, must be left behind like an 18-year-old purges her commonsense when she matriculates at Smith.
Amusingly for this set of data a test of proportions gives a p-value of 0.054, so a researcher who used that test would write the baseless headline, “No Link Between Profitizol And The Screaming Willies!” But if the researcher had used logistic regression, the p-value would have been 0.039, which would have seen the baseless headline “Profitizol Linked To Screaming Willies Cure!”
Both researchers would falsely think in terms of cause, and both would be sure that cause was or wasn’t present. Like I said, time for hypothesis testing to die the death it deserves. Bring out the guillotine.
Since this is the week of Thanksgiving, that’s enough for now.