Look at the picture, which is real data, but disguised to obscure its source. It is a physical measurement taken monthly by a recognized authority. The measurements are thought to have little error, which we can grant (at first); the numbers are used to make decisions of importance.
Question: is there a trend in the data?
Answer: there is no answer, and can be no answer. For we did not define trend.
What is a trend? There is no unique definition. One possible definition is the majority of changes: do more values increase or decrease. Another: Is the mean of the first half higher or lower than the second? Another: Are the values increasing or decreasing more strongly? For instance, there could be just as many ups and downs, but the ups are on average some-percent higher than the downs.
We could go on like this. In the end, we’d be left with a definition that fits the decisions we wish to make. Our definition of trend might therefore be different than somebody else’s.
Whatever we have, if we wish to declare a trend is present, all we have to do is look. Does the data meet the definition or not? If it does, then we have a trend; if it doesn’t, then it does not. Simple as that.
We don’t need p-values, we don’t need Bayes factors, we don’t models of any formality. We just look. Are there more than four coins on the table today (to suppose another question) as opposed to yesterday when there were three? There are or there aren’t. We just look. There are still three coins on the table today. Is three more then four? We do not need a formal model.
We need a model, which is the simple counting model. This is a model, but it cannot be said to be formal in any sense statisticians use models.
Now, keeping with our coin example, we will all agree that something caused the number of coins to be what they were. Perhaps several causes, of cause in its full sense of formal, material, efficient, and final aspects. If we decide there are more than four coins—by simple counting—it is clear that there would be a different cause or causes in effect than if there were four or fewer coins. Obviously!
There is no difference in the coin example than in our physical-measurement example. Yet the two are treated entirely differently by statistical trend hunters.
Statistical trend hunters will do something like compute a regression on the data. If the coefficient for trend is coupled with a wee p-value, the trend is declared to be present, else it is not. This is different in spirit from the definition of trend above. One definition could be an overall mean decrease or increase, as in a regression. But there is no sense to the idea the mean change is or is not there unless a p-value or some other measures takes a certain value.
How is it that coins are “really” greater than four even though we see three? How it the coins “really” aren’t three unless a function of the number of coins through time gives some value? How is it that the mean increase or decrease “really” isn’t there, even though we can see it, unless the p-value is wee?
It’s the same question that’s asked when after a medical trial, which showed a difference in treatments but where the statistician says the difference which was seen isn’t “really” there, because the p was not wee.
If the difference is not “really” there, but visible, the not-really-there difference is said to be “caused by chance.” Same with trends.
There we have the real lesson. It’s all about cause. Or should be.
Chance is not real, thus cannot cause anything. Yet some believe chance does exist, and that probability exists, too. If chance and probability exist, then causes can operate on them. Cause operates on real things. Cause must then act on the parameters of probability models, at least indirectly. How? Nobody has any idea how this might work. It can’t work, because it is absurd.
A complete discussion of cause and probability is in this paper. It is long and not light reading. That cause cannot operate on probability is another reason to reject p-values (and Bayes factors). For either of those measures ask us to believe that probability itself has been changed, i.e. caused to take different values.
(Incidentally, if you’re inclined to say “P-values have some good uses”, you’re wrong. Read this and the paper linked within.)
Cause is crucial. If a trend has been judged present, which happens when the p is wee, correlation suddenly becomes causation. The judgement is that the trend has a cause. It is true that all trends, however defined, have causes, but that is because every observation has a cause, and observations make up a trend.
Trend-setters say something different. They say the trend itself, the straight line, is real. Therefore, since the line is real, and real things have causes, the line must have a cause. That cause must have been a constant force of some kind, operating at precise regular intervals. If such a cause exists, as it can, then it should be easy to discover.
The problem is not that this kind of cause cannot exist, but the identification is too easily made. Consider the problem of varying the start date of the analysis. We have observations from 1 to t, and check for trend using (the incorrect) statistical means. The trend, as above in the picture, is declared. It is negative. Therefore, the cause is said to be present.
Then redo the analysis, this time starting from 2 to t, then 3 to t, and so on. You will discover that the trend changes, and even changes signs, all changes verified by wee p-values. But this cannot be! The first analysis said a linear force was in operation over the entire period. The second, third, and so forth analyses also claim linear forces were in operation over their entire periods, but these are different causes.
This picture shows just that, for the series above. For every point from i to t, a regression was run with linear trend, the trend estimate plotted, blue for wee p-values and decreasing trends, red for wee p-values and increasing trends, black for either increasing or decreasing and non-wee p.
The statistician would be convinced a negative linear cause was in effect for the first few months. A different cause for the entire series from i to n. Then it went away! No causes were present, except “chance”, for a while, then a positive linear cause appeared. And appeared again. And again, each time different, each valid for the entire series from i to n. It becomes silly in roughly 2013, where one month we are certain of a positive linear trend, and the next we are certain there is “nothing”, then certain again of another positive trend, then “nothing” again. And so on.
This is a proof by absurdity that cause has not been identified when a trend is accompanied by a wee p-value or large Bayes factor.
Correlation is not causation. But we can put correlation to use. We can use the correlational (not causation) line in predicting future values of the series. In a probabilistic sense, of course. Since liner causes might be in operation, perhaps approximately, then the linear probability model might make skillful predictions.
But since we almost never have the future data in hand when we want to convince readers we have discovered a cause, it’s best then to do two things.
(1) Make the predictions; say at time t+1 with value will be X +/- x, at t+2 it will be something else, etc. This allows anybody to check the prediction, even if they don’t have access to the original data or model. What could be fairer?
(2) DO NOT SHOW THE TREND. Showing it each and every time causes the Deadly Sin of Reification to be committed not just by you, but by your reader. He sees the line first and foremost. The line becomes realer than Reality. The stuff that happens outside the line is called “noise”, or something worse. No! The data is real: the data is what happened. The world felt the data, not the line.
Please pass this on to anybody you see, especially scientists, who use statistical methods to claim their trends are “significant”.