Correlation
I was at the Doctors for Disaster Preparedness conference in Ontario (LA) California and gave my paper The Crisis Of Evidence: Why Probability And Statistics Cannot Discover Cause, which we also discussed here. It went well; actually, better than I had hoped.
I didn’t have my computer with me. Bliss. But I am now rather behind.
Anyway, our friend Roy Spencer once did an analysis which showed (something like) the number of annual UFO reports correlated with the global average temperature anomaly. Passed all the statistical tests he could think of. Wee p-values etc. Conclusion? UFOs cause global warming—and didn’t Michael Crichton say the same thing?
Now we all laugh at this example. Why? There is nothing wrong with it. Spencer followed all the classical rules. It must be the case that we are forced to say, at least, UFOs are “linked to” global warming. If we cannot admit that, then we must toss out the classical statistics.
And that’s what we should do. Out the door it goes. Never use hypothesis tests or Bayes factors again. They can say nothing about cause. That’s nothing as in no thing.
We laugh at Roy’s example (one of thousands) because we know that UFOs can’t possibly cause changes in temperature. And because we know what can’t be the cause, any statistical procedure that says UFOs are a cause must be the logical equivalent of a congressman making a promise.
Of course, no scientist would ever run the tests “proving” UFOs cause global warming. Because scientists are, and should be, interested in cause and not wee p-values or big Bayes factors. Indeed, any professional (sober) statistician the scientist consulted would also refuse to do a test. Why? Because he would also know UFOs can’t be a cause.
But if you asked a statistician to provide a justification for his move consonant with statistical philosophy, she would not be able to give it. She’ll mumble something about “correlation isn’t causation”, but that is nothing more than a restatement of the problem. Why isn’t correlation causation.
“Well, correlation isn’t causation,” she’ll say, “because something else because the proposed mechanism might be the true cause.”
Okay, but since all we have are p-values or Bayes factors or the like, and these are often or always used to claim cause, how do we know in this case UFOs aren’t a cause?
“That’s because any good statistician looks outside the data as well as the uses the analysis.”
But if you’re looking outside the data, which is surely a good thing to do, why use the analysis at all? In this case, it can only lie to you.
“That’s because sometimes the proposed mechanism is a cause. And in those cases we want to use a statistical test.”
Hold on. In the cases we know the proposed mechanism is a cause, then also we don’t need to use statistical tests to tell us so, because, of course, we already know the mechanism is a cause.
“No, because it still might be the case that the results are due to chance. Hypothesis tests and Bayes factors can tell us that.”
Wait, wait, wait. We have descended into madness. Chance isn’t a force. It isn’t physical. It is a matter of epistemology and not ontology. It is a measure of our ignorance and not cause. Nothing in the world can be “due to”, i.e. cause by, chance. Saying the results are “due to” chance is to say either something wildly false or misleading, or it merely admits we don’t know what caused our observations. But then if that’s true, it must be that when we say the results are not “due to” chance, we are saying we have proved, or proved with some high probability, what the cause of the observations was. And that we have already seen is false.
So why do we use statistics in any but two senses? (1) Reporting: just saying what happened with no overlaying of any model or test, or (2) Predicting: saying what the chances are for new observations given we assume we have knowledge of the cause.
“You know, you’re right. I shall no longer teach p-values to my students.”
Falsifiability
I had nice conversation with George Gilder about the usefulness of falsifiability. Regular readers will know I think it has only limited abilities. Probability models are often not falsifiable, for instance.
Gilder uses it in an economic sense. Very briefly, if wealth is essentially (used in the “mostly” or “more or less” sense) knowledge, to increase knowledge is to increase wealth. Now if we do not allow businesses to “falsify”, i.e. to fail, and we insist on propping them up, we force ourselves to remain ignorant of the (free market) solutions that could have fixed it. In other words, we force stagnation.
That appears to me to be absolutely true.
Other
I am late, writing this on the fly, so only a note about Edward Calabrese who spoke on hormesis. Look it up. We’ll talk more about that later.
Aliens don’t cause global warming?
Say what????
https://www.youtube.com/watch?v=TnwtTQlfaQo
This link is broken: The Crisis Of Evidence: Why Probability And Statistics Cannot Discover Cause.
Thanks, Dan. Link fixed.
I think you must be wrong , because a leading expert in the field wrote in a highly regarded newspaper earlier this year,
“Typically, scientists apply a 95 percent confidence limit, meaning that they will accept a causal claim only if they can show that the odds of the relationship’s occurring by chance are no more than one in 20. “
Paul,
And who can argue with experts?
For others “occurring by chance” has no causal meaning. “By” chance means caused by chance, which is impossible.
Update: I had to look. Naomi Oreskes. Sheesh.
Your “Michael Crichton say the same thing?” link gets me to somewhere that says “REQUEST FORBIDDEN”.
Tried on Chrome and Firefox…
Causality is a deterministic (predictable) process but statistics applies to random (unpredictable) processes, like flipping a coin, rolling a pair of dice or drawing a card from a deck. I always claimed you can’t prove predictability using math that applies to the unpredictable.
“UFOs cause global warming—and didn’t Michael Crichton say the same thing?”
Actually, wouldn’t that be the other way around: global warming causes UFO sightings?
Likely most UFO sightings are some kind of misunderstood (by the viewer) weather phenomenon, so it actually makes sense that they would correlate with temperatures.
@Ray,
There is no such thing as random processes in nature. All the processes you mention are in fact predictable if you had sufficient information about starting conditions. Uncertainty / randomness are just measures of ignorance/lack of information.
All,
My enemies have taken advantage of my absence. Many more items are now fixed.
Classical statistics with its p-values and hypothesis testing are pretty much useless in finding causes. Agreed. However studies of correlations between variables is not useless. I maintain it’s necessary approach if only to verify a particular hypothesized cause. And to do this requires statistics, and probability, no?
An experiment to control X to see if doing so changes Y in a predictable manner, introduces another variable Q (= changed X). The correlations between X,Y, and Q are used in the determination.
But the experiment isn’t necessary if Q effectively is present. What if a variable is discovered that makes X & Y independent when conditioned on it — IND(X,Y | C). Then I think it would be justified in concluding C is a common cause of X and Y. For all practical purposes anyway. Of course we can quibble over the word.
Pearl’s algorithms IC and IC* do this mechanically and extend causal relationships when possible — all through statistical analysis. But then, you say he’s wrong. Still would like to know where he’s strayed. Are these algorithms perfect? By no means but it’s often a good start.
FWIW: a somewhat better algorithm is TETRAD . Some of how it works is in:
Causation, Prediction, and Search, Second Edition
by Peter Spirtes, Clark Glymour, and Richard Scheines
ISBN-13: 978-0262194402
ISBN-10: 0262194406
http://www.amazon.com/Causation-Prediction-Adaptive-Computation-Learning/dp/0262194406/ref=sr_1_2?ie=UTF8&qid=1438617221&sr=8-2&keywords=causation+prediction+search
Evey effect is influenced by multiple causes. These causes may differ in their influence on the effect. In industrial statistics, one would first make as comprehensive a list as possible of possible causative factors. For example, tablet uniformity in a pharmaceutical might be affected by factors related to granulation (e.g., RPM, duration of impeller), drying (duration, load size, air temp, etc.), blending (blending time, rpm, etc.), and etc. There are lots of factors that could result in a lack of dosage uniformity among tablets.
Some of these are of minor importance and can be left “to vary as they vary.” Some may be dominant, like Arnold Schwarzenegger joining with the seven dwarfs in a tug of war. It doesn’t matter what Sneezy and Grumpy do. The rope moves where Arnold wills. We have called the gallimaufry of many small causes, no one of which is dominant, “random” causes; not because randomness is a cause but because the actual physical causes are too many and individually too minor to be worth teasing out of the overall variation. Collectively, they form the “noise” against which we try to find Arnold’s fingerprints.
So why do a correlation? It may be useful in indicating a possible cause. If the correlation sucks, there are probably no Arnolds in the mix. (Or else they neatly cancel each other out.) If there is a good correlation between X and Y, then X might be a cause of Y. Or Y might be a cause of X. Or Z might be a cause of both X and Y. Or it might just be a big fat coincidence. IMHO, the statistical analysis is not the answer, but it may be a way of asking the questions. You still have to roll over the rocks and do the causal investigation, but the stats maybe tell you which rocks are worth rolling over.
YOS, DAV,
YOS says, “IMHO, the statistical analysis is not the answer, but it may be a way of asking the questions. You still have to roll over the rocks and do the causal investigation, but the stats maybe tell you which rocks are worth rolling over.”
This is exactly so. We obviously learn causes and the natures of things and powers from examining data. Indeed, there is no other way to learn except through this “extended” sense data.
But it is still time for hypothesis tests to die die die.
Minor point: I wouldn’t say every effect is influenced by many causes, though this is true in most mundane cases.
DAV,
Can an algorithm discover a cause in some circumstances. Sure, if they causes are pre-known in the sense the possible causes are already. Probability (I have this in the paper) in that case can quantify likelihood of which cause is operative. An algorithm cannot perform induction, however, and thus can’t prove cause.
Prove is a strong word, as you know. Proving something is an incredible burden.
More to come when I can get to induction.
Pingback: Probability & Statistic Cannot Prove Cause | Reaction Times
Ah, but Global Warming causes UFOs. I have it on good authority from watching Arnold Swartzenegger in the movie ‘Predator’. The cute native girl, played by Maria Conchita Alonzo, told us the aliens only come in the hottest years.
“Felix, qui potuit rerum cognoscere causas” (Happy, [is he] who has been able to recognize the causes of things. – Virgil)
@Tom Bri
Sir, you are wrong. UFOs cause global warming. They change the climate so that every year is the hottest year.
They also have the ability to time travel. We know this because, every year the climate of 50 years or more ago is discovered to have become colder.
This makes the present even hotter allowing more UFOs to visit.
Of course UFO’s don’t cause warming…. don’t be ridiculous.
Infra-red radiation into outer space attracts the UFOs!
Briggs,
Can an algorithm discover a cause in some circumstances. Sure, if they causes are pre-known in the sense the possible causes are already.
Not if already known. If it was already known there wouldn’t be any point in using the algorithm.
Let’s see, if I tun on a Roomba and it wanders about, did I vacuum the room or did the Roomba?
If I design and implement an algorithm to searching for a cause, IT finds the cause and reports its findings. The algorithm is told what to look for (I.e., something with the characteristics of cause and effect). It isn’t given predefined answers. Using one, I can hardly say it was I who did the finding anymore than the designer of the Roomba just vacuumed the room. I may be the one seeing the significance of a discovery but certainly not the one making it. So, unless you are willing to admit the algorithm did the discovery, who is doing the finding if it isn’t me?
This all goes back to what I said the other day about the philosophy of only the user of a discovery tool can discover vs. the discovery being done by a tool. You are firmly on the side of the user so discussing otherwise is pointless.
An algorithm cannot perform induction, however, and thus can’t prove cause.
Induction never can “prove” — it just gives the most probable explanation (cause), yes? As for an algorithm not being able to do induction surely that’s nonsense. When you do induction you aren’t doing anything differently than the algorithm. Yet, when you do it, you claim you aren’t following a recipe. Really?
Perhaps UFO’s need to shed a lot of heat. All that interstellar traveling will heat the UFO, and that heat has to go somewhere. After all, supernova remnants are heated up too when that gas crashes into the interstellar medium, and one would expect UFO’s going much faster than 2000 km/s, which is a common expansion speed.
Regarding the warming, CO2 is a greenhouse gas, so adding some should make the atmosphere hotter.
Two posible causes. The next step is to put some numbers on the effects.
Obviously you’re right – correlation does not demonstrate causation – duh.
However… you say “Because he would also know UFOs can’t be a cause.” and I say “oh really?” (picture smirk here, please)
https://www.google.com/search?q=ginat+spaceships+entering+the+sun&ie=utf-8&oe=utf-8
(and, yes, those people are more likely to be just nuts than right nuts).
The main problem with this article is it throws the baby out with the bathwater. Perhaps Dr Briggs could answer this question: If an experiment cannot be used to discover cause, what do use to discover cause?
It is not difficult to point to failings in any set of methods, but what are the solutions or the alternatives? Although you mention in your article that you are critical of Popper’s theory of falsifiability, you don’t mention in what way? Are you skeptical of it? Or do you fully endorse the system but lament it cannot be used more strongly?
MattS, I’m not sure I understand what you mean by
“There is no such thing as random processes in nature. All the processes you mention are in fact predictable if you had sufficient information about starting conditions. Uncertainty / randomness are just measures of ignorance/lack of information.”
Now there are chaotic processes (e.g. the simple double pendulum) that one cannot predict a particular trajectory, just a set of possible trajectories, even when the initial conditions are “well-defined”. Do a Google search or see
http://www.math24.net/double-pendulum.html
I guess it hinges on what one means “ordinarily” by “sufficient information” and what that might have to be in practice.
DAV: As far as I can tell, neither you nor the Roomba vacuumed the room. Those things are highly overrated. Need work on the cleaning algorithms and a way to run them with having to spend more time cleaning the thing than it would have taken to just vacuum the room with a corded vac. With the corded vac, we know you vacuumed.
Sander: “adding some should make the atmosphere hotter”. Yes, but how many other factors in climate lower the temperature and maybe cancel out the CO2? If I put water in a pan and heat it, it will boil. Unless I have a fan that removes all the heat and keeps the pan from boiling.
Sheri,
Yeah a lot of things don’t live up to claims made of them. Hired a guy to cut my lawn and all I got was disappointment. At least the Roomba looks like it’s trying and not sitting on it’s ass lighting a Camel.
@Bob Kurland,
In highly complex situations, such as the simple double pendulum sufficient information could include the exact distribution of mass in the pendulum head down to an atomic scale, air temp, pressure, flows down to mm scale in the area in which the pendulum swings, seismic activity (at very high resolution and sensitivity.)
Now it may be that in some cases, and the double pendulum may be one of them that sufficient information is not obtainable by anyone other than God. That still doesn’t make the underlying process that drives the pendulum’s movement random in any real sense.
Arnie didn’t need a stat test to know “If it bleeds, we can kill it.” Arnie > p values. Begs the question why he can’t apply this logic to climate change…
DAV,
Better than sitting on a camel lighting its ass. 🙂
(Sorry, I’ll leave now).
DAV: Good point. At least appearance wise the Roomba does look busy. Plus you can kick the Roomba across the room the eighth time it gets stuck somewhere and you don’t get sued.
Bulldust: Arnie was bitten by the political vampires and cannot form clear thoughts anymore. A tragic, incurable bite it is.
Phil R: You can leave, but I thought that was funny. 🙂
Phil R,
It is. For those who don’t know, I was quoting part of an old joke. Glenn Beck made an update in 2012.
DAV,
Thanks. Don’t recall seeing that before, but that’s funny.
MattS–
“In highly complex situations, such as the simple double pendulum sufficient information could include the exact distribution of mass in the pendulum head down to an atomic scale, air temp, pressure, flows down to mm scale in the area in which the pendulum swings, seismic activity (at very high resolution and sensitivity.)
Now it may be that in some cases, and the double pendulum may be one of them that sufficient information is not obtainable by anyone other than God. That still doesn’t make the underlying process that drives the pendulum’s movement random in any real sense.
Matt, I’m an operationalist with respect to science. If a human being can’t predict the outcome, then it’s unpredictable. Whether you call the possible outcomes random or not random, depends on what you mean by random. But they are unpredictable by us. God can always predict the outcome of anything by virtue of his Middle Knowledge.
Assume we have knowledge of the cause!!! Ginormous assumption this is. Think about it. If we have knowledge of the cause, then ____ (fill in the blank).
I don’t know what it means to say “prove” cause, but statistics does not tell us how to come up with causes or factors that may cause a certain effect. We have to rely on our knowledge or informed guesses about likely causes of the effect of interest, and there is no guarantee that what we come up is indeed a cause. Isn’t this obvious?
What’s not obvious is whether statistical tools can prove causality. Some text books in social research discuss over this topic intensively. In short, science doesn’t prove anything, neither does Statistics. They are never set out to prove anything, though data and empirical evidence falsify ideas.
(To prove a statement is to provide deductive arguments (proof) for the statement. )
If the temperature at 8am and the price of a certain stock at noon are highly correlated based on the data from this past year, yes, I’d consider the temperature at 8am to decide whether to sell or buy this certain stock at noon. No, I don’t know why the stock price changes, and I am no Carl Icahn.
If I knew the cause why stock prices change, then _____ (fill in the blank… hmmm, predictions?).
Correct. No method, scientific or philosophical, can prove cause. Probability at best, can quantify confidence. At worst, it can completely fool you. If you’re an especially dim witted researcher (more the norm these days as science degrees are handed out in much the same way as taxi licences) you might end up confused on this point.
Geez Briggs:
like those whoread you dont already know that corelation isn’t causality….
I dont even like your use of the word “linked”
they aren’t linked, except by some type of proximity…temporal, spatial, whatever.
in any case, all a correlation tells us is that two things occurred…the link to causality is ALWAYS dependent on the quality of insight and hypotheses that propose a mechanism that actually does link the two phenomena in question.
Gender and incarceration rates….
Skirt length and economic growth…
This who dont understand this, cannot honestly claim to be trained in statistical analysis.
davideisenstadt,
I gather, then, that you have not read the paper. Nor ever used a hypothesis test or Bayes factor. You are in the happy minority.
I’ll be reviewing the paper, “Is young fatherhood causally related to midlife mortality? A sibling fixed-effect study in Finland”. Wee p-values say yes.
The irrefutable correlation! Proof positive of global warming!
https://scontent.fphl1-1.fna.fbcdn.net/hphotos-xap1/v/t1.0-9/11825886_900276213341963_2584038828767395468_n.jpg?oh=7894ce4f1991d1d0b52ecdf6a807352e&oe=56496411
Briggs:
I deal with this all of the time…most of time unsuccessfully…
Ever introduce a class to incarceration rates broken down by gender?
Clearly men are victimized by the criminal justice system.
Mortality rates and gender?
Marrried men die, on average 7 years before their spouses, why?
Because they want to, thats why.
Look, i just dealt with a student regarding the effing likert scale, a fraudulent meaningless construct from the get go. like a score of 3.57 means anything at all?
But all of this doesn’t mean that these misconceptions are correct, eh?
P value…i say BS! explain the link, make a plausible explanation for the correlation, and i will consider it…
Pingback: This Week in Reaction (2015/08/09) | The Reactivity Place
Pingback: Is Young Fatherhood Causally Related To Midlife Mortality? Wee P-values Say Yes! | William M. Briggs