Say Eyjafjallajokull three times fast. I wish I could say it once. I’ve heard it in so many different forms since it blew its stack that I can’t be sure how its pronounced.
But pop off it did, spewing, as volcanoes are wont, a mass of tephra, gas, and other nastiness into the atmosphere.
You’ll have already heard that the extra-state bureaucrats that run Europe closed down the air space over that continent. This was probably wise immediately after the eruption.
But the closure order was allowed to linger, stranding many and costing much. Airlines complained, as you’d expect them to; some flew test flights to show all was well. Angry voices were raised.
The question is: why did EU bureaucrats maintain the impenetrability of their air space so long?
According to Christopher Booker it was because of an “over-reliance on an inadequate computer model”.
Here’s the meat (emphasis mine):
Within two days, the amount of ash over northern Europe was at barely one per cent of the official danger level. But the authorities were locked by international rules into a rigid bureaucratic system, based on a computer model, which gave them no alternative but to close down air traffic for days longer than was justified. The real flaw in the system was that it made no provision for testing that crude computer model against actual real-life data, which could have shown that the computer was vastly exaggerating the risk.
Having to fit together a jigsaw puzzle of non-interlocking bureaucracies did not help:
Responsibility for responding to the Icelandic eruption lay with a bewildering hierarchy of national and international authorities, starting at the top with a UN body, the International Civil Aviation Organisation (ICAO), working down through the European Commission and Eurocontrol (which is not part of the EU), to national agencies, such as our own Civil Aviation Authority, the National Air Traffic Service and, last but not least, the UK Met Office, owners of the relevant computer model.
In music, a particularly lovely or revealing theme always bears repeating. So it is in prose: our betters who run the EU made no provision for testing that crude computer model against actual real-life data.
I’m tempted to, in the modern musical style of repeating a phrase ad nauseum, cut and paste this statement until it is pounding in your head. For it tells us all we need to know about what’s wrong with governments that wants to rule us via reliance on superior brain power.
The idea of creating a model of volcanic particle dispersion is sound, interesting, and valuable. The fact of foisting it untested on a people as the deciding factor in governmental decisions is silly.
Undoubtedly, the scientists who coded the model, who drew up its equations describing the gentle curves of air flow, love their creation. They would hate to hear anybody call her “ugly” or “useless.”
Their reaction, upon hearing these slights and slurs, would be, as it always is with jealous lovers, to strike back. “We are scientists!” they would say, “With PhDs. We work hard; we can integrate manifolds blindfolded; we work hard. Our model has been peer-reviewed and all of us find her beautiful!”
And they would also say, “Our model runs on a computer!” Gasp!
These chants—so-called because they work as well as any incantation—cause bureaucrats to cower and defer. “Those scientists are smarter than we; therefore, we must not question them” is their cloak. Pulling intellectual rank works better than holy water on a vampire.
I have not seen every rule and regulation that the agglomeration of European government agencies have created to lord over commercial air travel, but I would bet $5,000 at even odds that nowhere is there a proviso that says something like, “In order to be considered reliable, all models will be tested on independent data and demonstrated to be skillful.”
Where, as always, “independent data” is defined to mean data that was not used in any way in the design or testing of the model.
And so we consider Cap & Trade & Tax & Spend—now on hold because the Obama administration wants to stir up leftist sentiment over immigration as a means to stop the bleeding of Democrat votes.
Do we ever hear how, if Cap & Trade is implemented, it will be phased out if the models turn out to be in error? The creators of climate models are so sure of themselves that the mere thought of error is anathema, heretical.
But do all models speak ex cathedra? Not hardly. Yet since most are convinced that “things are worse than we thought” these laws may be inflicted upon us.
We should insist that a cut-off switch be grafted onto them. This is an appealing political tactic. When passage appears inevitable ask for the commonsense inclusion of tests of model validity.
Insist that the laws automatically expire if the temperature of the Earth (suitably defined) does not increase, or the models lack skill.
There are no reasonable objections to this request.
The model used by the Met Office in this fiasco is the NAME model described here:
http://en.wikipedia.org/wiki/NAME_(dispersion_model)
But it uses Lagrangian Dispersion models, and Random Walk techniques, and Empirical Turbulence profiles, and Puff Techniques, and (gasp) Monte Carlo Methods, so that’s all right then.
Two space-shuttle accidents are at least partially explained as the result of over-reliance on computer models — one an impact code and the other a heat transfer program. The Sleipner platform failed due to misuse of a finite-element code. The 2003 power outage in the Northeast resulted from failure to restart a network analyzer after a software update. We have no idea if the Toyota accelerator problems resulted from a software issue, but everyone thinks they did, which is just as bad.
No problem. Let’s run the entire economy according to output from an unverified, untestable, flaw ridden code. The Easter islanders had their stone heads — we have virtual idols.
Models help us identify what we do not understand. If you do not test them against reality, you do not know what you do not know.
“Unknown unknowns” again. We are back to “our betters” being intellectually unable to conceive of limits to their “knowns”.
The Wizard of Oz explains how our world really works. Huge government funding for science and technology does not correspondingly multiply the number of wise deep thinkers — it awards diplomas to vast armies of zoo animals. There are hairless chimps everywhere with PhDs for brains who earnestly believe that developing computer models is science, so their pointy-haired bosses have to believe that too.
On a happier note, here is a charming lesson on the pronunciation of that Icelandic name:
http://www.youtube.com/watch?v=4HXUws4uCMU
What we have is more of a political problem than a statistical problem. If a bureaucrat gives the “all clear” and that is followed by an incident, there will be hell to pay. What margin of safety should he work with? He needs 99.99% confidence, and that just isn’t going to happen.
“The creators of climate models are so sure of themselves that the mere thought of error is anathema, heretical. ”
If the climate modelers knew what they were doing there wouldn’t be twenty some odd climate models all giving different answers. When someone claims the ability to write a computer program to foretell the future a century from now, I disbelieve them.
Boeing performed more than 15,000 hours of wind tunnel tests as input to and verification of their aerodynamic models.
Beware the robots.
When a machine tells you what to do, ask yourself, “What’s in it for the machine?” Or better yet, for the machine builders?
But my warning, and Briggs’, will fall on deaf ears. It is already too late, way past the tipping point. The robots are in control. Resistance is futile.
Is that a bell ringing? I must respond. Beep, beep, beep. I am here, I will do your bidding oh Mechanical Master. I twitch at your command. Ground me, freeze me, make me dance. Do what you will to me oh mighty Robot Lord.
I cannot agree more with your “cut off switch” concept. A bill’s sponsor(s) – modeled or not-should be required to list the assumptions made to justify any new legislation. The minority party should have the right to craft tests for these assumptions. If the test(s) fail(s) at some future date or data collection point etc it would “trip” a requirement to correct/amend the law/regulation – a nice system check and balance. (There should also be a similar requirement for regulation coming from the regulatory agencies.)
It may also force more precise language as to what the legislation is trying to achieve and keep the interpretation out of the courts and the bureaucracy. It would be interesting to pick a recent piece of legislation and define potential “test” parameters.
I have the sense from some of your earlier posts you don’t like Popper– but isn’t a cut off switch a quest for falsification?
Pingback: Tweets that mention William M. Briggs, Statistician » What Can Volcanic Dust Models Tell Us About Cap & Trade? -- Topsy.com
I remember once working with a six DOF missile flyout model to determine lethal range. We finally determined that range wasn’t the issue, the pilot had to carefully select the random number seed for the Monte Carlo simulation.
The European Commission yesterday outlined a comprehensive package of measures to provide relief to the air transport industry in the aftermath of the volcanic ash-related airspace closures earlier this month and committed to “coordinated European action” to reconsider response procedures for future volcanic eruptions.
…
Kallas said preliminary estimates by the EC put losses related to the airspace closures at between €1.5 billion ($2 billion) and €2.5 billion for the industry, including airlines, airports, air navigation service providers and other aviation-related businesses such as tour operators. To alleviate the financial burden, the EC is proposing that member states rapidly implement measures to grant loans and loan guarantees at market conditions, and even provide state aid “for net losses directly linked to the exceptional occurrences.”
http://www.atwonline.com/news/story.html?storyID=20146
I hate to spoil the party but it wasn’t the Eurocrats that shut down the various air spaces but rather each of the EU member countries independently (for some values of independent). The German government was amazing to watch as it first declared the “no risk” policy and then backed off when the airlines started making noise. Strangely, just as information was actually being gathered concerning the actual location and concentration of the cloud over Germany, the airlines started a number of VFR flights.
The basis for the decision to close the airspace was, indeed a computer model, or rather various models. The claim that these models are untested is, however, inaccurate.
According to the MET site (picking on the MET because it’s in English)
More detail showing satellite and LIDARobservations vs. model forecast.
LIDAR measurement data was late in coming in large measure because the funding for the monitoring network was cut to save money. However, even if the network had been intact it would not have been adequate to the task of deciding where and when it was safe enough to fly. The area involved is large and the number of stations small.
Given the fact that Europe is located very near a large active volcanic hot spot it seems amazing to me that we are so completely unprepared for the inevitable but there you go. On top of that, it seems that the aircraft engine manufactures have not defined maximum limits for safe operation so that even complete dust data would not have provided sufficient information to decide when it was safe to operate except in clear air.
I await the eruption of Katla with bated breath.
William
.
Say Eyjafjallajokull three times fast. I wish I could say it once.
.
I can and you can . Yes we can .
It helps knowing following things :
Рit is written correctly Eyjafjallaj̦kull
– ll is pronounced tl like in the english cu-tl-ass
Р̦ is pronounced like the german ̦ like in L̦we
– j is pronounced like the russian j like in kratkoj (or english y-es)
– y is classically like i like in most languages
How do I know ? I visited Iceland (a beautiful country) with my daughter who speaks 9 languages and spent part of her time over there talking with icelandese about their grammar and semantics .
So now an easy method in 3 steps :
1) Say Heya several times . Drop the h and keep eya .
2) Say fiat – la (like in fiat lux) 3 times . Fiat-la .
3) Say yö-kutl 3 times . If you don’t know how to say ö , say something between o and a deep drawling e . If you can’t , do a simple vague o and hope that an icelandese won’t notice .
.
Now say the 3 parts 1 after the other . Decrease the intervals eya РFiatla Рy̦kutl .
And now increase the volume and drawl it in one piece .
There you are , you just convincingly pronounced Eyjafjallajökull 🙂
Throw in how science fiction (most notoriously the Star Trek universe) has computer models that *always* accurately represent reality without fail. Those holodecks never go wrong.
Matt:
Roger Pielke has a very interesting post on the role of models: http://pielkeclimatesci.wordpress.com/2010/04/28/comments-on-numerical-modeling-as-the-new-climate-science-paradigm/
Perhaps this is another example of the adage “When you only have a hammer, you see everything as a nail!!”
Here’s an interesting ‘climate modeling done the right way’ (as indicated by the need to “match observations of temperature trends with model predictions” — presumably to mean that the model wasn’t “tweaked” to force-fit it to the observations) summary paper from NASA GISS (in 1998):
http://www.gsfc.nasa.gov/gsfc/service/gallery/fact_sheets/earthsci/eos/global_warming.pdf
And here’s a link to a report indicating that a fundamental driver of the carbon cycle, soil microbes & fungi, which account for something like 8x’s human-generated carbon greenhouse gasses, actually behave fundamentally differently than was assumed:
http://today.uci.edu/news/2010/04/nr_soilcarbon_100426.php
as reported by:
http://wattsupwiththat.com/2010/04/28/new-ground-truth-microbiotic-negative-feedback/#more-19014
Tom,
I have it! I think. But I’ve no one of Icelandic descent to practice it on.
Pat,
A cut-off switch is not equivalent to falsification. A jury might pronounce a defendant “not guilty” when not all are convinced (beyond a reasonable doubt) of his guilt, yet that jury would not claim to have falsified the theory that the defendant committed the crime.