Before us are the observations X1 to X156. Recall we are assuming that each of these X has been measured without any error. Given that we observe X1 = 0.43, the probability X1 = 0,43 is 1 or 100%. Now let’s have some fun and ask some more complicated questions of our data. We can ask anything we like.
How about this? What is
(9) Pr(X156 > X1| Observations)?
Well, all we have to do is look. It appears from the plot, and a glance at the data confirms, that this probability is 0, or 0%. Why? Well, because X156 was not larger than X1. How could that cause any controversy?
But notice that (9) is an entirely different question than this one:
(10) Pr(X156 > X1| M1)
or this
(11) Pr(X156 > X1| M2),
It could very well be that (10) is greater than (say) 90% but (11) is less than 10%. It could even be, depending how rigid the models are, that (10) equals 1 and (11) equals 0; that is, the models could say the exact opposite of one another. And this is not a rare situation: indeed, we saw it last week when M1 was a climate model and M2 was a probability model. Don’t forget that we already agreed with eq. (3): different models produce different probabilities for Xi in general.
How about this question?
(12) What is the probability that temperatures (as measured by X) increased?
I don’t know and neither do you. Why? Because this question is ambiguous. Just what exactly does “increased” mean? Is it asking if X156 > X1? That’s a kind of increase. If so, then we can compute an answer (given our observations, it is 0, or 0%). Does “increased” mean that X increased at least once? If so, then we can compute an answer, which in this case is 1, or 100%, the opposite of the first definition. If you say that “increased” means “increased generally” then you have to supply the definition of “generally.”
Suppose “increased generally” means that X increased more often than it decreased or stayed the same. We can easily compute this, given our observations. And this is a different probability than if “increased generally” means that X increased or stayed the same more often than it decreased. If you do not see this, stop here and ponder until you do.
We could go on and on. “Increased” could be taken to me increased by a certain amount, or never decreased more than another fixed number. There are more possibilities, too, but we’ll skip them. All we need remember is that the probability of these questions can all be different. There is just no evading these kinds of definitions (even if you want to). So before you find your blood boiling and you hear yourself shouting “Denier!” make sure you understand what question you and your enemy are answering.
Let’s pick one of these definitions so that we can move on: let’s say that “increased generally” means “X increased or stayed the same more often than it decreased.” Now, what is (12) with this definition in mind? Well, all we have to do is glance at our observations: the probability, given these observations, is 0, or 0%. Let’s write this out in notation, to make it crystal clear:
(13) Pr( X increased | Observations) = 0.
Of course, (13) assumes our definition of “increased”, which to be precise we should write inside the parentheses; however, we’ll let it slide for ease of reading.
We seem to be done. We wanted to know if X “increased”, we agreed on a definition of what “increased” meant, we looked at the observations, and we knew the answer. We even agreed that each X is measured without error. Except for disagreements about the word “increased”, there would seem to be no room whatsoever for controversy.
There is, of course, plenty of space, but only because people confuse just what probability is being computed. Because some people, when hearing “What is the probability X increased”, instead of computing (13), compute this
(14) Pr(X increased | M1)
or this
(15) Pr(X increased | M2),
etc. There is no contradiction for using the past tense here, for we can just assume, as people do, that “increased” means “would have increased”. Now it is perfectly possible, as before, that (14) could be greater than, say, 90%, and that (15) could be less than, say, 10%—even though, in fact, X did not increase (by our definition). And we don’t have to settle for just these two models: we have an infinite supply of models, so that we can compute
(16) Pr(X increased | Mj)
for j = 1, 2, … Each (or most) of these probabilities will be different. Which is the correct one?
Well, that is a separate question. For real temperatures, I have no idea which model is best. Many people, however, claim they know exactly which model is the best and truest and most pure. Well, maybe they are right. They say that they have a model which takes into account, in just the proper way, forced versus unforced feedback, the effects of variable sunlight, the thermodynamics of this and that gas, and on and on. Suppose the folks making this claim are right and that their model is the “best”, where we can leave the notion “best” float for now. Further suppose that
(17) Pr(X increased | Mbest) > 90%;
that is, given the best model, the chance that X would have increased over this series is 90%. Given Mbest, this probability is true. But, alas, so is (13). It is still the case, no matter what (17) tells us, that (13) holds, that temperatures did not, in actual fact, increase. This is just too bad for Mbest. Since we are supposing that Mbest is best, all we have learned is that our model has rather severe limits and that X is not predictable to the extent we would like it to be. Tough luck for the model! Sometimes the universe is predictable and sometimes she isn’t.
But then, it is unlikely that Mbest really is best (just as it is even more likely that the model you, the reader, have in mind is best). I said I don’t know what model really is best, but I do claim to know, and I will shortly prove, that the way to think about temperatures is flawed, because we don’t consider that X is measured with error. We’ll tackle that next time. We’ll also talk about “significant” and “linear” increases, etc.
It would be nice if X1 and X156 were labeled on the graph otherwise P(X156>X1|Observations)=0 is hard to ascertain. How do we know you aren’t denying the obvious? 🙂
DAV ????
X1 is the left-hand of the line and, since there are 156 observations, X156 is the right-hand end of the line. Or, since this is obvious, perhaps you’re making a not-so-obvious joke.
So (9) actually reduces to: what’s the probability that -2 is more than 0.5? Let’s see, divide by 2 and – Infinity again!
Hmmmmm……
All,
I notice that now that the going has gone rough, the comments from the perpetually outraged have dropped to 0. I’ll need your help to let me know if this series is discussed elsewhere. Like DAV suggested yesterday, it probably won’t be.
But I admit to being happy about the series, particularly Parts IV & V (the last!). I hope by Friday to have (a version of) this in PDF form, perhaps on Arxiv.
Briggs,
Please post a link to the data so the kids can play along! For example this kid likes the Kendall family.. That avoids those pesky errors in time,too.
BillR,
Good idea. Now on Part I appears this update:
Update Link to the data (CSV file), for those who like to touch.
If our model is that people will arrive at this post without reading the previous post or have forgotten the previous post then additional labels on the graph pointing out where X1 and X156 are located would be useful. DAV is right.
If our model is that more information on the graph is better than less information on the graph then DAV is right.
Briggs wrote, “let’s say that ‘increased generally’ means ‘X increased or stayed the same more often than it decreased.'” My understanding of this is that for every pair of points
(Xi) and (Xi+1)
the set
((Xi – Xi+1) is less than or equal to zero)
has more members than the set
((Xi) – (Xi+1) is greater than zero).
Apparently the tags don’t work for comments.
Just last week, we received our daughters’ grade reports. Mr. JH dug out all the reports since 9th grade and asked my daughter #2 to comment on her own GPA performance over the years. He told her that there was a troubling and unacceptable overall decreasing trend. (I can’t tell you what model he used, but he had carried out some sort of modeling.)
My witty, trying-to-get-away daughter pointed out that he did not read the reports properly because her score on religion increased from B+ in 2008 to A in 2011 (only because she loves Father Don, her current religion teacher.) She had made a correct observation, but…
Well, she earned herself a lecture on how important it is to do well in school.
Great post. Clear and well defined.
The slope of the OLS regression line for GMST being positive is a perfectly reasonable definition of “temperatures have risen”. It is one that most climatologists will be happy with as it is an indicator of a general trend in the data that isn’t highly sensitive to the random variability of individual obsevations. Perhaps it would be easiest to explain the point you are making by starting at a point that is familiar to climate science, rather than one that will seem odd to them (“X increased or stayed the same more often than it decreased.†) with good reason.
Let beta be the slope of the OLS regression, then assuming that the data are measured without error p(beta>0|observations) is either 0 or 1, beta either is greater than 0 or it isn’t, just as for the case of the measure you have used. There is only one OLS regression line given the observations.
Yes, I do know that OLS has a probabilistic intepretation, but so does “X has increased or stayed the same more often than it decreased” (the data were generated by a stochastic process that increases temperature at each time step with probability q and decreases it with probability 1-q and by “the climate has warmed” we implicitly mean that q > (1-q)).
Beta is a statistic that summarises a feature of the data, just as q is. Both have a statistical/probabilistic interpretation, but that doesn’t mean we actually using that interpretation.
@Dikran: When you say “the data were generated by a stochastic process”, which data are you referring to?
@Dikran: I don’t want to clog this thread, but I believe that in the first thread about Temperatures, you got distracted by a couple of individuals and didn’t reply to my question of 3 February 2012 at 3:28 pm. Could you please go back to that thread (On Global Warming Apoplexy: Temperature Trends) and answer, so as not to clog this thread? I’ll post it again at the end of that thread, for your convenience. Thanks!
I am not crazy about this definition. Basically, you count the numbers of non-negatives and negatives for the series y(t)= x(t)-x(t-1) (x here are the realized data values). If there are more non-negative numbers in {y(t)}, then given the observations, one concludes that X (variable name) increased. (The nonparametric sign test comes to mind! ^_^)
Note that this definition doesn’t take into account the magnitude of the increase or decrease. I can easily come up with a time series that has visible increasing trend but (13) holds based on the definition.
For example: x=c(14, 13.89, 13.88, 14.21, 14.2, 14.19, 14.5, 14.44, 14.51, 14.5, 14.49, 14.51)
Two points determine a line. A nonnegative increase means a nonnegative slope when connecting two consecutive points. A decrease implies a negative slope. For instance, the slope is negative when connecting the two points ( x(1)=14, time=1 ) and ( x(2)=13.89, time=2 ).
So, one can also look at the definition in a different way. That is, “X increased if there were more non-negative slopes than negative ones (when connecting data points).â€
Still, not a favorable definition, for me anyway.
@Dikram, I agree that the slope of the OLS regression line is a reasonable definition of whether the data has increased or decreased. However, I also appreciated the point that the post was making – which is that definitions matter.
In my experience, a lot of arguments are due to two people having different definitions for the same terms. Until that is identified, the people will be arguing past each other. So defining terms is an important step. Assuming that the other person uses the same definitions just leads to potential angst that would be unnecessary if the definitions had been spelled out and agreed at the start.
Sometimes the definitions are easy to agree upon, but other times they’re not. Even when people think they agree on the definitions, there can be special cases that they never considered. As I work at the edges of the healthcare industry, my best example is gender. There are FOUR genders in the computer systems that I work with. Besides male and female, there is also unknown (eg. a badly burnt victim in emergency will have an unknown gender at that time) and also indeterminate (which is a neo-natal condition affecting some babies, in that their gender – male or female – can’t be identified until up to three weeks after their birth). I am, of course, ignoring the issue of people in the middle of a sex change operation, and hermaphrodites…. And who thought that the definition of gender would be trivial?!
In addition to the two points which I identified as troubling about your previous post I am also puzzled by your continued discussion of the “probabilities” of events that have already happened, and by a lack of clarity about what you mean by “probability” in the first place. Failure to precisely identify the underlying repeatable experiment or “game” is the source of many an apparent paradox in elementary probability theory and it appears that you are either falling into (or setting up) a similar trap here.
Alan Cooper,
If you have a lottery ticket and I know what winning numbers were drawn but you don’t, what are the chances that you hold the winning lottery ticket? A past event yet has a probability value other than one or zero.
Even when known, the probability value will have only one of two values. It’s still there.
As for clarity in the definition of probability, that’s a deep philosophical question. Go checkout Von Mises, et al. Matt is using it here as a measure of uncertainty.
I should point out that the chances of you winning really depends on the question. The probability of you winning given what I know may not necessarily equal the probability of you winning given what you know. They are two different questions.
For example, what I know may be the winning numbers and whether I selected you at random and if I already know the winner. The probability of you winning given what you know likely is hasn’t changed much from the day you bought the ticket.
Raw probabilities, like P(X),shouldn’t be used. It’s a habit hard to break.
Oh my. That second sentence sound silly. “What you know” vs. “What I know” .
@GraemeW, I agree, definitions are important, which is why if we are questioning projections made my climatologists it is only fair to use a definition of warming that they would use, not one that they wouldn’t (and for good reason). The problem isn’t the statistics, the problem is a lack of consideration of the climatology in designing the analysis.
The key to success in staistics is understanding the data and the purpose of the analysis. In this case we need to have some understanding of climatology and the aim of the models and discuss with the climatologists why they use linear trends, rather than just assume that we statisticians know best.
Dear Mr Briggs,
I downloaded the data and I have a couple of questions.
I fitted a third-order polynomial like my friend Roy and now I have a linear parameter that’s negative and a cubic parameter that’s positive, both highly significant! Is it warming or cooling?
I can’t make a nice graph like yours unless I have years with 13 months in them. That can’t be right can it?
Keep fittin’ ’em, Rich! We’re bound to stumble upon one which fits perfectly.
To everybody else; we do know that this data is fictional, right? As in entirely made up so as not to offend anybody.