Statistics

What A Prediction Is And What It Is Not: Part IV

There is a technical distinction between a scenario and a prediction, though the line is fine and often disputable. Many scenarios are merely intolerably loose, and therefore useless, predictions. The difference is that a prediction lays out its conditions for all to see, while a scenario keeps at least one condition hidden. The conditions are both on the “left hand side”, i.e. what conditions must hold for the prediction to be in force, and on the “right hand side”, i.e. what is the measurable, observable event that is being predicted.

An example of a scenario is the so-called Doomsday Clock touted by the Bulletin of the Atomic Scientists, a group composed of folks from the left wing of science. On 10 January 2012, the BAS announcedDoomday clock

It is five minutes to midnight. Two years ago, it appeared that world leaders might address the truly global threats that we face. In many cases, that trend has not continued or been reversed. For that reason, the Bulletin of the Atomic Scientists is moving the clock hand one minute closer to midnight, back to its time in 2007.

The BAS said, “Two years ago, it appeared that world leaders might address the truly global threats that we face. In many cases, that trend has not continued or been reversed.” By “threats” they mean climate change, energy use, terrorism, and other matters. Nuclear weapons being loosed is on the list, but seemingly as an afterthought.

The BAS clock is a scenario for two reasons. (1) We do not know what will happen if the clock does strike midnight: the clock is supposed to represent “doomsday”, but that term is meaningless (in secular terms). There is also no date by which we can verify the prediction. (2) We do not know each condition that went into the decision to shift the clock. We know some of the conditions, because the BAS announced some (“harmonized domestic regulation [of CO2]”), but we do not know them all (the BAS publicly stated their conditions were only “a minimum” set).

What we have at the BAS is a secular clerisy saying, “Do something: we won’t tell you what, but it had better be something. If you do this something, we’ll turn back our frightening clock. If the clock strikes midnight, something bad will happen: so bad we cannot tell you what it is.” I mention in passing (and with no value judgment) that humanity has suffered through both (repeated) climate change and the use (and stockpiling) of nuclear weapons, yet it has not approached annihilation.

The BAS clock is thus political theater, capable of convincing only those unwilling to put in any thought about what the clock means, and it is quite useless in making any real-world decision.

The Intergovernmental Panel on Climate Change issues what it openly calls scenarios on the path of “emissions” and their effect on global temperature (all data and quotes taken from this public website). Are these genuine scenarios?

Without burdening us with details, it is possible to operationally define what “global average temperature” (GAT) is, even though the term has little physical meaning (no thing experiences a GAT; plants, animals, rocks, experience temperatures; nothing experiences a global average temperature). This definition of GAT is statistical.

A forecast of the GAT is a scenario unless the operation definition of what the GAT is is known (and thus the scenario becomes a prediction). That is, the list of (ever in flux) surface stations, satellites, buoys, etc. must be known; plus all the statistical manipulation that is used to massage the data from these sources must be explicated for a prediction to hold. The GAT is a prediction, but the loose statements which litter the media about other weather effects are generally scenarios.

Other associated scenarios are the bad things that await us if the GAT increases, which is to say, if the “climate” changes. Greenpeace is a serial offender and issues steady streams of scenarios. For example, they would like to “promote sustainable agriculture” which they claim is endangered because of climate change. This is a scenario because by taking the word at its plain English definition, agriculture is already “sustainable” because the fruits of the land are, in fact, sustaining billions. Incidentally, Greenpeace knows this and so is forced to begin their scenario with the words, “Over the past 50 years, we have nearly tripled agricultural outputs.” (they also say, “lack of food is not the cause of hunger” which is again at odds with agriculture not being “sustainable”).

“Sustainable”, then, is always a scenario word.

The IPCC’s energy-use scenarios are, however, predictions. They are predictions which seek to evade the force of predictions by calling them scenarios. In their words:

Scenarios are alternative images of how the future might unfold and are an appropriate tool with which to analyse how driving forces may influence future emission outcomes and to assess the associated uncertainties. They assist in climate change analysis, including climate modeling and the assessment of impacts, adaptation, and mitigation. The possibility that any single emissions path will occur as described in scenarios is highly uncertain.

The IPCC admits that “Any scenario necessarily includes subjective elements and is open to various interpretations”, which is of course part of the definition of a scenario. But their scenarios are predictions. The IPCC directly states that if energy use is this-and-such, then that-much warming will occur; and that if energy use is instead that-or-so, then this-not-that much warming will occur. It is true that the IPCC does not state in exact terms what are likelihood of each condition is, but this does not matter because if one of these conditions obtains, the specific prediction of warming is then in force.

The IPCC’s ploy is obvious: if these predictions are a bust then the theory/model which gave rise to the predictions will not have to be abandoned or radically modified. That is, if the prediction fails, the IPCC hopes to say that it was only a scenario and was on no use whatsoever in making any decisions. But this is false: unlike the BAS, the IPCC clear states that the action it requires is reducing carbon-based energy use. They intend fully that these so-called scenarios be treated exactly as predictions.

Who was it that said something about a beautiful theory being murdered by ugly facts? It is this wholesale slaughter that the IPCC would escape by claiming, after the fact, that what actually happened was just not part of any “scenario,” therefore its theories must live. Theories live or die by their ability to make skillful predictions or not. And this is so if the predictions are mislabeled scenarios.

We’ll look at some detailed prediction/scenarios at another date.

Categories: Statistics

15 replies »

  1. I’ve always seen the “Doomsday Clock” as the rethorical devices some scientists use to express their panic over the complexity and dangers of the modern world. They do not understand it, and so they obviously conclude that it will go kaboom, and pretty, pretty soon at that – 5 minutes to be exact.

  2. This has been a brilliant series Mr. Briggs. Your analysis is surgical in its precision.

    Maybe the BAS Doomsday clock should be tied to some super-string based quanta of time. At least that way they could blame the uncertainty on theorectical physicist, and accuse anyone who questions its meanining to be ignorant of the science.

  3. “The difference is that a prediction lays out its conditions for all to see, while a scenario keeps at least one condition hidden.”

    Well, I’m about to give up, but one more try . . .

    Yes, we could redefine the word “scenario” to mean simply a prediction with one condition hidden, but that hasn’t really gotten us anywhere beyond noting, in a particular case, that such-and-such prediction incorporates a hidden condition.

    Further, we can lay out scenarios with incredible precision and with all the conditions defined and expressed. It has nothing to do with whether there is some unexpressed condition or not.

    The issue is not conditions. The issue is not specificity. The issue is certainty (e.g., “will” versus “would” or “could”).

  4. I should add that I appreciate Briggs taking time to put together this series of posts, with some great gems, like the following, with which I wholeheartedly agree:

    “What we have at the BAS is a secular clerisy saying, “Do something: we won’t tell you what, but it had better be something. If you do this something, we’ll turn back our frightening clock. If the clock strikes midnight, something bad will happen: so bad we cannot tell you what it is.” I mention in passing (and with no value judgment) that humanity has suffered through both (repeated) climate change and the use (and stockpiling) of nuclear weapons, yet it has not approached annihilation.

    The BAS clock is thus political theater, capable of convincing only those unwilling to put in any thought about what the clock means, and it is quite useless in making any real-world decision.”

  5. Eric,

    Not quite. As I said up front, a prediction, a hard prediction, is an action or decision. No probabilities attached. Those that come with uncertainty I called (for lack of a better phrase) soft predictions. You turn soft into hard by performing an action based on the uncertainty quantified in the soft prediction.

    More about this to come.

  6. I like the “doomsday clock”. And I agree with Luis Dias. I look on the clock as a harmless but necessary device allowing academics with poor interpersonal skills to express in a limited way their inability to cope with modern life. That very few logical persons – [NOTE: “The Media” are by morphemic analysis self-excluded from that category] – take it seriously is evidence the clock serves it purpose.

    The best part about a doomsday clock is – contrary to the laws of physics [science?] – it can be sped up, slowed down or even set back, all by the power of one’s intellect. Is this wistfully “god-like” thinking, or what?

  7. Without burdening us with details, it is possible to operationally define what “global average temperature” (GAT) is, even though the term has little physical meaning (no thing experiences a GAT; plants, animals, rocks, experience temperatures; nothing experiences a global average temperature). This definition of GAT is statistical.

    This statement is true, yet the climate science tribe endorses and promotes the concept of “teleconnections” that says individual organisms do experience and correlate to GAT. Does this false premise have different implications for predictions and scenarios?

  8. soft predictions

    Seems to me a soft prediction is a scenario unless the prediction is the probability itself otherwise it is semantically no different than substituting (may, could, might) for the probability. That implies some mechanism be available to test the proposition (i.e., that the probability is X is Y).

    Does the weatherman make predictions? “There is a 40% chance of rain tomorrow”. If my understanding is correct that means there is a 40% chance that someone within a vague (usually, metropolitan) area will experience rainfall. And if no one does, hey, there was just a chance (even if it was 90%). How does one verify such a “prediction”? As if anyone really cares about the actual number.

    Well, with rainfall — maybe it can be tested but it gets harder (read nearly impossible) for one-time and rare events.

  9. The doomsday clock seems more analogous to a risk rating than any actual scenario or prediction. It seems very similar to what Standard and Poors does.

    S&P will tell you that an A rating is better than a BBB rating, but that is about as far as it goes. They will also say that an A rating is not comperable accross the spectrum of bond types — an A rated Corporate bond cannot be compared to an rated municpal bond, or an A rated Mortage-backed bond.

    At least there is enough history in corporate bonds that we can look at historical default rates of A rated companies, to get a sense of what ‘A’ means. Until 2007 there was no history in the world of Mortage-backed bonds. And of course there is no history of doomsday events to measure against the Doomsday clock.

  10. Eric & Briggs, debating above

    (Briggs reply includes: “As I said up front, a prediction, a hard prediction, is an action or decision. No probabilities attached. Those that come with uncertainty I called (for lack of a better phrase) soft predictions. You turn soft into hard by performing an action based on the uncertainty quantified in the soft prediction”),

    are addressing SEMANTICS — which illustrates that when one strives to parse & define a topic so incredibly nitpickily precisely an entirely new factor gets introduced into the mix to further garble things: language & the ability of the persons involved to understand the statements used to “precisely” define terms, and, to interpret the statements made by others doing the defining.

    In other words, in the effort to add clarity the ability of the parties involved to use & understand a common language, which seldom is ever on a parity, itself contributs to obfuscation. When that happens, such as repeatedly in this blog essay series, it’s a sure indication that something is going overboard.

    This is a very common/recurring issue in law, with the interpretation of various statutes–and the observed outcome that, all too consistently, the clear conclusions diverge, often radically, from teh philosophical/value-based result most laypeople would expect.

    A just-concluded US Supreme Court case addresses habeus corpus issues & interpretation issues form the core of the decision of just the sort, in principle, discussed in this recent series of blog essays. This case will be of concern to every U.S. citizen concerned about how some basic freedoms (6th amendment, etc.) have been recently getting whittled away. Even more entertaining, however, is that Justice Scalia has a dissent that covers most of the document and includes ample phrases in his trademark call-it-as-he-sees-it style we always look forward to:

    http://www.bloomberglaw.com/public/document/GONZALEZ_v_THALER_No_10895_2012_BL_6147_US_Jan_10_2012_Court_Opin

  11. Yeah, 49er, I also find the godlike abilities of the doom clock maintenance people curious and funny. Can we still even describe it as a “clock”? Probably a very relativistic, quantum mechanics’ style of clock, filled with weird spooky behaviors… yeah that must be it.

  12. I have a little problem with the claim to be able to calculate the GAT. Weather stations do not calculate the average local temperature. The oceans cover 70% of the earth and there aren’t any weather stations providing temperatures over the ocean surface. How can Hansen calculate a GAT? The GAT is an excellent example of “garbage in gospel out”.

  13. Briggs: “Not quite. As I said up front, a prediction, a hard prediction, is an action or decision. No probabilities attached. . . . More about this to come.”

    OK, I look forward to your further thoughts.

    In the meantime, I’m still failing to see why a scenario can’t be as precise and well defined as a prediction. If I’m analyzing potential launch windows for a craft heading to Mars, I might run dozens of scenarios — all of which are extremely well defined, and very precise in their calculations. If we launch at a particular time, with a particular rocket, with a particular burn, on a particular trajectory, and so on and so on, then we will arrive at Mars at such-and-such a time, in such-and-such a location, etc.

    These are all scenarios. They are all possible future outcomes. None of them may come to pass. Yet they are very precisely and fully defined.

    I’m sure you have an excellent point in the works and I need to spend some more time thinking about what you’ve written thus far (and also look forward to your next post). But in the effort to make your larger point, I’m wondering if there isn’t a risk of twisting the definitions of words to get where you want to go, based on a desire to have a once-and-for-all definitive distinction between the two words that will then support the larger point . . .

  14. Isn’t scenario planning what we do to build strategies in response to future uncertainty?

    My conclusion is that a prediction is a subset of a scenario. Sorry, I am too lazy to explain it today.

    Conditional prediction, non-conditional prediction, vague prediction, soft prediction, hard prediction… any other kind of predictions? Funny. All predictions are conditional.

Leave a Reply

Your email address will not be published. Required fields are marked *