Statistics

How To Properly Handle Proxy Time Series Reconstructions

This is all made up data, so as not to hurt anybody’s feelings. Also, this is a sketch. Everything can’t be done in 700 words.

We are interested in the time series T, which represents values of some thing taken at progressive time points (these needn’t be regular). But we can’t measure T. We can, however, measure a proxy of T, something “correlated” or associated with T, something which might causally be affected by T. What’s a proxy? Something like this:

Figure 1

Figure 1

Imagine the proxy is some chemical measurement inside tree rings, coral reefs, or whatever and T temperature. Somehow we have taken simultaneous measurements where both the proxy and T were available. Step one is to model the relationship, which is shown by the over-plotted line (a simple linear regression). Pretty good fit, no?

It ought to be, because this is an oracle model; which is to say, the model here is true because I picked it. In real life, the model itself is usually a guess, meaning everything that follows will paint a picture of confidence which evades us in reality.

Next thing is to guesstimate T where we have no T but where we have the proxy. Like this (the proxies aren’t shown, but I used the perfect model fitted above to predict T):

Figure 2

Figure 2

Very well, this looks like a reasonable prediction of T given new values of proxy (using the same regression). But every good scientist knows that error bars should accompany any prediction. Here’s what people using time series usually do:

Figure 3

Figure 3

The fuzziness comes from looking at the error, the plus-or-minus, of the relevant parameter inside the model (standard 95% bounds). Looks like a tight prediction, no? Even after taking into account the uncertainty of the parameter, we’re still pretty sure what T was. Right? You guessed it: wrong.

For that, we need this:

Figure 4

Figure 4

The wider bands show the plus-or-minus of T, the prediction interval of the real observable (same bounds). There is no use plotting the uncertainty of the parameter as above, because the parameter doesn’t exist. T exists. We want to know T. This is the best guess of T, our ostensible goal, and not of anything else.

I would like to shout that previous paragraph right up next to your ear until I see you nod.

Notice how much, how dramatically larger are the intervals? How less certain we really, truly are? If you noticed that, you have done well. But don’t forget that this picture is too optimistic, because the proxy-T model was known. In real life, we won’t usually know this and so have to widen the final error bars.

By how much? Nobody knows. This is key. If we knew, then we could know the model and we wouldn’t have to widen the bars. But since we do not know the proxy-T model, we do not know how much to push out the envelope. Meaning that if we accept the numerical bounds as accurate just because they are numerical, we will be too certain. Worse, in our quantitative-induced euphoria, we’ll forget that we should be less certain. Not all probability is quantifiable.

Now another thing people like to do is to plot a straight line over the guesstimated T and speak of whether there was a “statistically significant” increase or decrease in T, or they’ll use the line to say “there has been an X average increase in T” or some such thing. This is almost always folly, not the least because these judgments eschew the uncertainty we have been at pains to illuminate.

Plus there is no reason in the world to do this unless you expect that straight line to skillfully predict future values of T. How do you know if this is true? Hint: you don’t. After all, something like this can happen:

Figure 5

Figure 5

The new T (over the entire period and not just the time of the proxy) was generated in advance (as were the proxies, which recall have a specific known relationship with T). I picked this one (T is a kinda-sorta a “long-memory” time series) because of its vague resemblance to actual time series we have all seen before.


Categories: Statistics

8 replies »

  1. “… or they’ll use the line to say “there has been an X average increase in T” or some such thing.”

    All that’s needed is to take the medians of a handful of points around X1 and X2 and to see the answer. But ya gotta admit the line looks more sexy and scientific.

    “I would like to shout that previous paragraph right up next to your ear until I see you nod.”

    It’s difficult to nod off with someone shouting in your ear. There will likely be no nodding until after you become hoarse.

  2. Is the straight line plotted in Fig 5 something that was actually calculated from the data points in Fig 2? I haven’t tried doing any kind of fit yet, but it looks a lot steeper than what I would guess “by eye”.

  3. Alan,

    Just a standard regression fit (the extrapolation). Matches “by eye” only by coincidence.

  4. Yes, your line does match what I get from calculating it. My “eye” claims to have been fooled by the change of scale from where it saw the data first and by the introduction of new lower data on the right in the compressed graph, but I’m having none of that and I am sending it in for re-calibration anyhow.

  5. Of course you are right that it is a bit presumptuous to extrapolate from a data set where the predicted trend over the data interval is barely 1/5 of the total range of values in that interval. Does anyone actually do that?

  6. Alan,

    I just took the simulated values and fit a simple linear trend to the first 50 of them, then showed the rest of the simulated values with the same trend overplotted. Not much can be gained from staring at this example, because I picked this one to show how things can go bad. Whether they actually go bad in any real series is a separate question (actually separate questions).

    And of course we regularly hear of climate forecasts of “global mean” temperatures out a century in advance.

    Plus, the trend bit is an afterthought. The difference between parametric and predictive uncertainty is the real story.

  7. I call it Highway Hypnosis Syndrome.

    When people — by this I mean scientists who construct these things — concentrate on the bright-yellow center line, they miss that the car’s actual path of travel along the highway can safely pass along any stretch of the width of [empty, otherwise untraveled] road

    JJB

Leave a Reply

Your email address will not be published. Required fields are marked *