I was at the Doctors for Disaster Preparedness conference in Ontario (LA) California and gave my paper The Crisis Of Evidence: Why Probability And Statistics Cannot Discover Cause, which we also discussed here. It went well; actually, better than I had hoped.
I didn’t have my computer with me. Bliss. But I am now rather behind.
Anyway, our friend Roy Spencer once did an analysis which showed (something like) the number of annual UFO reports correlated with the global average temperature anomaly. Passed all the statistical tests he could think of. Wee p-values etc. Conclusion? UFOs cause global warming—and didn’t Michael Crichton say the same thing?
Now we all laugh at this example. Why? There is nothing wrong with it. Spencer followed all the classical rules. It must be the case that we are forced to say, at least, UFOs are “linked to” global warming. If we cannot admit that, then we must toss out the classical statistics.
And that’s what we should do. Out the door it goes. Never use hypothesis tests or Bayes factors again. They can say nothing about cause. That’s nothing as in no thing.
We laugh at Roy’s example (one of thousands) because we know that UFOs can’t possibly cause changes in temperature. And because we know what can’t be the cause, any statistical procedure that says UFOs are a cause must be the logical equivalent of a congressman making a promise.
Of course, no scientist would ever run the tests “proving” UFOs cause global warming. Because scientists are, and should be, interested in cause and not wee p-values or big Bayes factors. Indeed, any professional (sober) statistician the scientist consulted would also refuse to do a test. Why? Because he would also know UFOs can’t be a cause.
But if you asked a statistician to provide a justification for his move consonant with statistical philosophy, she would not be able to give it. She’ll mumble something about “correlation isn’t causation”, but that is nothing more than a restatement of the problem. Why isn’t correlation causation.
“Well, correlation isn’t causation,” she’ll say, “because something else because the proposed mechanism might be the true cause.”
Okay, but since all we have are p-values or Bayes factors or the like, and these are often or always used to claim cause, how do we know in this case UFOs aren’t a cause?
“That’s because any good statistician looks outside the data as well as the uses the analysis.”
But if you’re looking outside the data, which is surely a good thing to do, why use the analysis at all? In this case, it can only lie to you.
“That’s because sometimes the proposed mechanism is a cause. And in those cases we want to use a statistical test.”
Hold on. In the cases we know the proposed mechanism is a cause, then also we don’t need to use statistical tests to tell us so, because, of course, we already know the mechanism is a cause.
“No, because it still might be the case that the results are due to chance. Hypothesis tests and Bayes factors can tell us that.”
Wait, wait, wait. We have descended into madness. Chance isn’t a force. It isn’t physical. It is a matter of epistemology and not ontology. It is a measure of our ignorance and not cause. Nothing in the world can be “due to”, i.e. cause by, chance. Saying the results are “due to” chance is to say either something wildly false or misleading, or it merely admits we don’t know what caused our observations. But then if that’s true, it must be that when we say the results are not “due to” chance, we are saying we have proved, or proved with some high probability, what the cause of the observations was. And that we have already seen is false.
So why do we use statistics in any but two senses? (1) Reporting: just saying what happened with no overlaying of any model or test, or (2) Predicting: saying what the chances are for new observations given we assume we have knowledge of the cause.
“You know, you’re right. I shall no longer teach p-values to my students.”
I had nice conversation with George Gilder about the usefulness of falsifiability. Regular readers will know I think it has only limited abilities. Probability models are often not falsifiable, for instance.
Gilder uses it in an economic sense. Very briefly, if wealth is essentially (used in the “mostly” or “more or less” sense) knowledge, to increase knowledge is to increase wealth. Now if we do not allow businesses to “falsify”, i.e. to fail, and we insist on propping them up, we force ourselves to remain ignorant of the (free market) solutions that could have fixed it. In other words, we force stagnation.
That appears to me to be absolutely true.
I am late, writing this on the fly, so only a note about Edward Calabrese who spoke on hormesis. Look it up. We’ll talk more about that later.