Video: The Crisis Of Evidence, Or, Why Probability & Statistics Cannot Discover Cause

“Nobody wants to think, they just want to look at a pretty picture and be reassured that X will solve all their problems.”

And the fact that X wont solve any of their problems is irrelevant.

Thank you for making the talk available.

In the abstract, should be:
Classical statistical procedures, in both their frequentist and Bayesian implementations, falsely imply they can speak about cause.
(no “are”)
Also:
This over-certainty is made much worse when parametetric and not predictive methods are used.
should be “parametric”

Slides are useful for showing pictures. Keep the words to a minimum. I think too many presenters use them as note cards because they don’t have the skill to talk with their audience and need a crutch.

I enjoyed the talk Briggs, thanks. Every medical paper I have ever read always say “associated with” or a similar term. None of them say because they found p value less than 0.05 that they are certain that x causes y. They only say that it is possible, perhaps even likely in light of other research and plain reasoning. In medicine decisions have to be made because lives are at stake. So the evidence using p-values is not perfect, but it is likely to be better than accepting the opinion of experts (I think). Example: imagine you observe a group of people. Some of them get shot in the face with an assault rifle, and some get soap bubbles blown onto their faces. You are likely to notice a difference in the number of deaths between the bubble and assault rifle group, with whee p values. This does not PROVE that shooting someone in the face with an assault rifle is responsible for the deaths, but, the p values taken in context of physiological reasoning and common sense does strongly suggest that shooting someone in the face causes death!

Francois, isn’t the null hypothesis in your experiment “shooting people in the face will cause no harm”? Aren’t you supposed to actually *believe* the null?

Francsois,

The p-value, at best, gives you the likelihood that the difference between the groups in the data you’ve collected is not a figment of your imagination. It tells you nothing else.

Hi Nate, why would have to believe the null? Dav, “likelihood that the difference between the groups in the data you’ve collected is not a figment of your imagination”. That sounds pretty good to me! I can go with that.

but noting that a difference exists between two descriptive statistics tells you nothing about wther there is some causal nexus between the two.
You dont need to do any kind of statistical analysis to determine that getting shot in the face causes harm
We already know the physical mechanism responsible for your wee p value…
why not tackle an issue like gender and incarceration rates…
Can your analysis tell us anything about why they disparity exists?
really.
What about hem lines and GDP growth…care to posit a causal link between the two that makes some sense to us?

You might find a association between hemlines and GDP growth…. But because there is no sensible way to explain, and since it is rather silly, one can dismiss it for now. Just because you cannot prove causation with associations, does not mean it is clever to ignore associations either. Sometimes we find associations that do not have a preceding basic scientific explanation. The association prompts us to look at possible reasons for the association, i.e. we investigate causal mechanisms with basic science etc. Cigarette smoking is a good example. Associations were found between smoking and death. Nobody really understood the pathophysiological reasons for the association, but it seemed plausible, and was investigated further. Do you believe that smoking causes cancer David? I agree that associations can be abused, as in Briggs PM 2.5 example. But it would be silly to ignore associations.

” but noting that a difference exists between two descriptive statistics tells you nothing about whether there is some causal nexus between the two.” I know this, and said as much: “None of them say because they found p value less than 0.05 that they are certain that x causes y. They only say that it is possible, perhaps even likely in light of other research and plain reasoning.”

They only say that it is possible, perhaps even likely in light of other research and plain reasoning.”

But it’s that likely, often without any basis other than an association, that’s the problem. In a surprising departure from political correctness**, Scientific American, published It’s Time to End the War on Salt

All too often, a paper jumps to a conclusion either directly or by implication in press releases. Or even worse, there is often failure to correct public misinterpretations or the paper. Follow up studies rarely test if said “cause” is genuine but instead do “let’s see if the association holds” testing. So, the only certainty is that the association holds. There is a proposed mechanism for salt’s, er, badness but to date, no one has really tested if it’s true.

An interesting quote from the SciAm article:

For instance, a study published in February 2010 in the New England Journal of Medicine estimated that cutting salt intake by about 35 percent would save at least 44,000 American lives per year. But such estimates are not evidence, either; they are conjecture.
…
Alderman and his colleague Hillel Cohen propose that the government sponsor a large, controlled clinical trial to see what happens to people who follow low-salt diets over time.

One can only wonder how those lives-saved estimates were derived if “cause” was only possible.

Salt isn’t the only example. There are plenty more. It’s stuff like this that Briggs is railing against. There is far too much leaping from p-value hypothesis testing to causes of X.

** then blew it in the See Also section. Aw well.