Class - Applied Statistics

Example Of Bad Trend Analysis

These examples are from the BAMS article “Continental US Hurricane Landfall Frequency And Associated Damage: Observations and Future Risks” by PhiliP J. Klotzbach, Steven G. Bowen, Roger Pielke JR., and Michael Bell.

The caption reads:

Fig. 2. (a) CONUS landfalling hurricanes by year from 1900 to 2017, and CONUS landfalling major hurricanes by year from 1900 to 2017. The dotted lines represent linear trends over the period. The p values for the linear trends are 0.33 for landfalling hurricanes and 0.61 for landfalling major hurricanes, indicating that neither of these trends are significant

If you haven’t already, read the article “Using P-Values To Diagnose ‘Trends’ Is Invalid”. This is a must.

See those lines over-plotted on the graphs of actual observations? Mesmerizing, aren’t they. Those lines did not happen.

This is an odd thing to say, and to insist upon; nevertheless it is true. The lines did not happen.

Take a look at 2 (b), at 2010. What happened in that year? Nothing. What does the line say that happened? Something.

“Come on, Briggs. Everybody knows the lines didn’t happen! Don’t mislead people. They’re just there as a guide.”

That so? A guide to what?

“A guide to the trend.”

That so? What’s the “trend”?

“Well, it’s that line, drawn by a regression with a non-wee p-value. Which means a horizontal line, and not the line drawn, can’t be rejected.”

I’ll assume you know what you’re saying when you said that, but you lost me. Are you trying to say that there was, as the line suggests, a downward multi-year linear force causing the number of major hurricanes to be less?

“No, I mean that a constant force causing major hurricanes can’t be rejected.”

Never mind that double-talk. Let’s assume the p-value was wee and the line therefore “real”, whatever that means. In fact, what does it mean? That a downward physical cause, a decreasing cause, a force that changes in just the little bit year-by-year indicated by the line operates on the atmsophere?

“Well, not—”

Hold up. In 2010, even if the line was horizontal, nothing happened, right?

“Right.”

That means that if the line, horizontal or decreasing was real, there should have been something like less than 1 major hurricane, but that something stopped this less-than-one major hurircane. Another cause or causes are out there that negated the force measured by the line?

“Something is causing major storms, of course.”

Certainly. We’re only arguing whether these trend lines have discovered cause. We don’t need them to see the data, which is right there in front of us. Have the storms increased? Sometimes yes, sometimes no. What caused that blip around 2005? What caused the blank right after? It wasn’t the line.

“That line could be discovering a force, and one that is decreasing in a strictly linear way, weakening, as you say, year by year.”

Sure, it could. Why not? It could also be false. How can we tell?

“By the p-value.”

But every use of a p-value is fallacious or a mistake. As has been proved. Anyway, it must sound strange to claim a mere statistical test can discover such a subtle and peculiar cause.

No, what we can do is to use that line, or something like it, to make predictions of future numbers. If that line or that something makes skillful predictions, in a technical sense, then we have something. If not, then not.

In any case, that line is not proof that hurricanes are doing anything. The observations alone are all we need for that.

To support this site using credit card or PayPal click here

8 replies »

  1. The trend lines reinforce the observational analysis that nothing has happened with hurricane activity over the last century that would warrant the government curtailing our rights to use fossil fuels. Without the trend line, it’s easier for Global Warming of Doom leftists to focus attention on the 1985 and 2004-5 spikes to advance their agenda that climate disasters are increasing, and that the government must control and limit the use of fossil fuels to avert more disasters. The trend lines make it easier to refute the GWOD propaganda.

  2. ThiA second error is to restrict the analysis to hurricanes making US landfall. IF we are interested in the hurricane-making process, where the hurricane goes is a different problem, akin to predicting where a child’s spinning top will head off to.

    I also object to using a bar chart instead of plotted points. The visual technique obscures the zero years. [This was actually done by a customer once. He took the average of the non-zero points for defectives in a lot and submitted a complaint to us on that basis.]

    Also, the methodology assumes successive points are independent. But they may likely be serially correlated. [The best predictor of this year’s count is last year’s count.] Why not CUSUM or EWMA?

  3. Well, as far as tropical weather systems go; the causes are: 1. Water temperatures, at depth, at least 77F/25C, 2. an established low pressure area, either near the surface or mid-level, 3. a high pressure area aloft, and 4. little to no vertical wind shear. Number 4 is the most uncertain, I think; and note that all 4 have thresholds that must exist or exceed a value.

  4. We could follow our host’s earlier suggestion and simply look at any two time points and calculate the level change, weighting for the time between them. To be unbiased, we could substitute every number and look at all of those slopes, and summarize with, say, a median of these numbers. Call this the “Substitute every number” (Sen) estimator.

    This is a simple predictive instrument, with no need to talk about causes, distributions, or parameters or other asymptotic/metaphysical entities. It would not breakdown very quickly but if we were worried about that we could get the median at each point, and take the median of those points!

    We could then look at all (or some) of the shuffles of just these observed data and see the rank of the above median (repeated median) in the list. I would call this (normalized) rank the “Darwin Pea” rank. If the rank is above most of the shuffles, I would then say “That’s big. Might want to look at that some more.”

    It doesn’t cover YOS point about framing the question. This is more of a “Shut Up and Calculate” approach to data prediction, with a hat-tip to R. Feynman and D. Mermin.

  5. I don’t understand the focus on cause. If there is a known pattern but no known cause, then you can make predictions based on the pattern alone. If there is a known cause but no known pattern, then you can’t make predictions at all. The only thing that matters for practical purposes is the pattern.

    Take for example, the difference between the Medieval theory of impetus and Galileo’s theory of dynamics–both theories were current at the same time, and Galileo allowed that impetus may be the cause behind his equations.

    Impetus is a cause of unforced motion; it is something that builds up in a moving object while it is being propelled by a force and continues the motion of the object after the force ceases. This his how an arrow continues to fly after it has left the bow. This theory of causality gives you no power to predict anything.

    By contrast, Galileo’s theory of dynamics is just a set of equations that describe a pattern of nature. Galileo’s theory can be used to predict motion, the theory of impetus cannot.

  6. Causality from the trend line is because that is the argument of the AGW crowd. If the hurricanes are increasing, then the cause is AGW. Of course, same is true if they are decreasing, but let’s not go there. The reason causality is discussed is because that’s how a huge percentage of the population views the trend—as proof of some cause of the phenomena.

    Yes, the trend may be useful for prediction, unless of course, the cause of the phenomena suddenly causes the phenomena to veer from the trend and run far amok. Trends tend to be short-lived, so prediction is marginally useful even then.

  7. I didn’t say “trend”, I said “pattern”. Galileo’s equations of motion describe a pattern–one that is not only not short-lived, but apparently a permanent aspect of nature. Predicting based on a trend is risky; predicting based on a pattern is how all of technology and practical craftsmanship works.

  8. technology and practical craftmanship involves the ability to do repeatable experiments and observe the results, and change your technique. None of that is true with climate studies. In situations where you do not have the ability to control inputs, vary inputs, you have to be much more uncertain with your interpretation of your observations, precisely because you cannot know the cause

Leave a Reply

Your email address will not be published. Required fields are marked *