Climate Model Uncertainty: Part I

I find it bizarre that you continually misrepresent what climate models do. They do not ‘match the data used to build them’ in the sense that you mean (presumably the global mean temperature anomalies) since those data are *not* used to build them. What is used are the climatologies (the decadal+ averages of observables along with their statistical properties), not the year by year variations of the weather. They do not “take current observations as input” in the sense of weather forecast models. Transient runs are initiallised with pre-industrial conditions and are free-running after that, driven only by external (and independently derived) changes in atmospheric composition are surface features.

Try actually downloading a climate model and see for yourself how they work. Or, you know, try reading a paper. Even your description of weather forecast procedures are wrong.

Hi Briggs,

I believe this recent post (and its links to the discussion on Steve Easterbrook’s blog) on parameterization, calibration, and validation of weather/climate models is relevant to the the issue of climate model uncertainty you bring up here.

BTW, I disagree with Gavin that inputs into the climate models are “the climatologies”. I think the distinction between input and output needs to be kept clear.

George

Matt,

Back in my youth I took a graduate-level forecasting course (ARIMA). The one point that was continually stressed was to hold out at least 20% of the existing data prior to estimating each model’s parameters so as to be able to compare predictions of competing models on the basis of how well these stacked up against the hold-out sample. No offense to Gavin (above), but every representation I’ve seen on GCM results seems to focus on how well the models fit ALL of the available data. This implies in my mind that they are continually re-estimated, or “tuned”, as new observations become available. (For now, let’s not get into the issue that they are probably tuned to inaccurate / corrupted surface data records)!

The point here is that “tuning”, as you have continually pointed out, is a trivial exercise. And in the case of GCMs, if the assumption going in is that CO2 done it, then all tuning (i.e., re-estimating) does is drive the model’s feedback and sensitivity parameters to the extent necessary to arrive at the observed (i.e., known) inputs. In short, if Gavin wants to buttress support for the GCMs, he needs to demonstrate that the models were estimated using data from one period and that the output predicts hold-out data that was not available to the model during estimation.

PS – What I’m really curious to see done is to have Gavin turn over his parameterized model to an independent group, who would then re-run it with a constant CO2 input of 280 ppm…

It would be a useful test, would it not, to tweak the models on, say, the 1910 to 2009 data and then get them to “predict” the 1850 – 1909 ‘climates’, which could then be compared with the observations. How do they do on such tests?

Suppose I built a model of a physical system. The inputs to my model would include equations of Newton, Galileo, et al. Some variables such friction, I may not be able to measure in my lab. So, I leave that as an open dial on my model, and I tease that dial to fit my observations. Ideally, I would test my model on new data. But, if new data is impossible to gather, and I have high confidence that my equations are correct, and I think I have accounted for all of my variables, can I publish my results?

In my former life, I traded mortgage-backed securities. There are 2 big unknowns in the world of MBS — if you will be repaid, and when you will be repaid. The government guarantees the majority of the market, so the “if” was a less relevant question than the “when.” Wall St. produces models that will forecast a mortgagor’s propensity to pay ones your mortgage each month for the next 30 years. Factors in the models include the rate, the size, the age, LTV, the location of the house, the environment in which the loan was issued, the path of interest rates since issue, and a multitude of potential interest rates, now and into the future. About once a year, each bank tweaks their model, without really addressing how the 2009 prepayment experience is has any bearing on how will pay in 2011. At least, in the case of MBS, your maximum loss is limited to the size of your initial investment.

Briggs–
Gavin has a point about the initialization. Really, he does.

Modelers in engineering do similar things when running models for steady state processes. (Say, heating a big tub of goo from the bottom of a tank.) You write a model that you think correctly parameterizes all phenomena at steady state. The model will break the volume of the tank into a bunch of grids, apply the equations we think apply to each grid in the tank.

When running the code, you can initialize the temperature in the tank one of two ways:

1) Set all temperatures to a constant value. Maybe you’ll pick room temperature and say the velocity is zero everywhere in the tank. You know this is a bad guess.
2) Guess the correct temperature and velocity in every grid cell. You could guess this based on someone’s measurements. You expect this is a better guess.

Now, given your guess, you run your code. To the extent that your first guess does not match the answer your model gives at steady state, heat will travel from one grid cell to another by convection or conduction.

Now, you expect that when you set the temperature and velocity to zero, that your first iteration will cause temperature to rise some places (especially near the heat source). On your second iteration, you might see the warm goo try to rise. You keep iterating until the answer after interation “N” matches the answer after iteration “N+1” within some criteria.

Oddly, if the observations you used were right but your models is wrong heat and mass will flow. So, the “right” answer will start to deviate from the observations. You must permit this to happen. You let the model continue to run. You keep iterating until the answer after interation “N” matches the answer after iteration “N+1” within some criteria.

In principle, if the solution is not very sensitive to initial conditions, your model should give you the same answer under both conditions. However the first method should converge more rapidly, and reduce the amount of computer time required to obtain a solution.

Mind you: there are physical systems that can have several different meta-stable steady-state solution. These might give different answers if you started with different initial conditions. Problems that are very sensitive to initial conditions would fall in the category of very interesting ones. In an industrial application, engineers would likely consider the multiple possible steady states a problem and would likely want to design the system to eliminate this sensitivity. Luckily, with many, many systems, the multiple steady states cannot occur.

But the general notion suggested by Gavin– where models are initialized with data and then spun up– mostly works, has analogs in engineering, and mostly is not particularly sensitive to initial conditions. The specific problem with using observations to set initial conditions is a potential problem, you are but as a practical matter, it is probably not important.

That said: there are more practical problems associated with modelers being able to peek at the 20th data when creating their hindcast.

There are posisbilities that the modelers can tweak their “boundary conditions” and “forcings” within the range constrained by already observed known data in a way that permits better agreement between transients and data.

For example, if a modeler A creates a model (say model E) and modeler B wants to get the best possible match to 20th century data with that model, modeler B could do some sensitivity tests varying aerosols, solar, co2 etc. They would fully explore the behavior of model E. (Why, they might even report this in the literature as interesting in and of itself. 🙂 ). Then, knowing more or less how the model responds to forcings, modeler B can pick higher or lower aerosols within the large range supported by the data. Say high aerosols is better for their model: Use that, cite that paper instead of the paper with the lower forcings. They can pick from several solar forcing estimates for the 20th century etc. They can’t force perfect agreement, but they can certainly improve it relative to someone who had to predict the 20th century climate without knowing what it looked like before running the transients.

Oh, and for what it’s worth, the modeler might do this all more or less unconsciously without thinking they are tuning. They can protest the tuning is not done in the same way that econometricians tune their models. Quite right.

They might argue that assuming the parameterziations in “model E” were right, then that points towards the specific aerosol/solar etc forcings that result in the best match being right. Of course, the alternative is the model parameterizations are biased and the good agreement will vanish the moment the modeler tries to predict future unknown data. The fact is, that combination of parameterizations and forcings were selected with knowledge of the observations that would ultimately be used to test the simulations. So, there is some feedback– though not as direct as fitting in a linear regression.

dearie me:
It would be a useful test, would it not, to tweak the models on, say, the 1910 to 2009 data and then get them to “predict” the 1850 – 1909 ‘climates’, which could then be compared with the observations. How do they do on such tests?
I’m not sure what you are suggesting. But given the class of models GCMS are, this would involve running it backwards. You can’t do that because the second law of thermo does not permit you to run these sorts of physical models backwards in time.

For example: Suppose you come home and discover a drinking glass full of room temperature water sitting on the counter. (More precisely, indistinguishable from roon temperature within measurement precision.)

Given that ultimate condition, and believing the room was at a constant temperature for the previous 5 hours, can you run a model to discover the temperature of the water 5 hour ago? If you told an engineer that the water had been 90 F at 8am and the room temperature was 70F they could do a pretty good job estimating the temperature as a function of time going forward. But there are an infinite number of initial conditions that would result in the water temperature being indistinguishable from room temperature when you arrive in the kitchen at 1 pm.

Gavin can’t run Model E backwards. No one can.

Lucia,

In your reply to dearie me, I understand your explanation and agree that you can’t run the model backwards. Can’t you, however, develop your model parameterizations using the 1910 to 2009 data set and then run the model (understanding that the base conditions e.g. CO2 concentration will need to be adjusted) feeding it data starting in 1850 and make forecasts you can test against the actual results later in the data set?

You can ask Gavin about GISS Model E aerosols forcing.

Here is a chart comparing aerosols (direct and indirect) forcing to the GHG forcing as reflected in the GISS Model E simulations.

It was until 1970 that the GHG forcing became greater than the aerosols forcing. In the 1890s, aerosols negative forcing was -170% of the GHG forcing.

http://img101.imageshack.us/img101/6042/modeleaerosols.gif

You could call the aerosols a fudge factor but it more that they are fudged period. Direct and indirect aerosols forcing.

http://img83.imageshack.us/img83/7408/modeleforcing.gif

Lucia, your examples happen to involve models or situations where the long run solution is a steady state. But surely climate is nothing like that? I’ve written lots of models that I could run either way in time: appeals to the Second Law wouldn’t matter a button for them. The cases where I couldn’t have run them either way typically involved some sort of mathematical stunt to avoid a computational difficulty – a shock, or near-shock, say – that precludes running them backwards. In other words the problem has not been a function of the physics but of the imperfect craft of mathematical modelling. Had I realised in advance that their might be a virtue in running them backwards, I’d have found some other remedy for the computational difficulties. Be that as it may, Steve’s point gets around the difficulty anyway.

I’ve always wondered why there are twenty some odd climate models in existance. If these modelers knew what they were doing there should be only one climate model and the output should agree with the measurements.

Some models only work backards.

An accident investigator can look at the results and detrimine what the intitial conditions must have been. But, no one could have looked at the intitial conditions and say that an accident would have been likely.

As Lucia said – “The fact is, that combination of parameterizations and forcings were selected with knowledge of the observations that would ultimately be used to test the simulations”.

Whether or not these selections were made “unconsciously” is irrelevant at this point – the “tuning” horse is out of the barn and no withholding of previously observed data or hindcasting can validate or falsify the models. We could, of course, wait on new observations, but this would take many years (decades?) before a statistically meaningful test could be run. (Assuming the modelers can be retrained from re-tuning their models in the interim)! Logically, then, the only way to validate the models is to test their specification, namely the forcing due to CO2.

If the modelers have this specification right, then running the models using pre-industrial levels of CO2 should result in a slight cooling consistent with the “consensus” estimate of warming due to emissions during the industrial age. If not, that is if the warming is due to forcings that are excluded or not correctly specified in the models, then running the models should result in a pronounced cooling due to the incorrect CO2-related feedbacks and sensitivities built into the models by tuning.

I’d like to see this simple test done, but assume Hell will freeze over before Gavin et al turn control of their supercomputers over to independent outsiders.

Climate models predict what would happen in reasonably long term climate trends at times beyond the time they were developed from. They are one way in temporal output, but they certainly can have the initial and boundary conditions of a previous time (if it can be accurately determined) used to develop them, then see if the present climate correctly results (but this is valid only if the most recent data was not used to help develop them). If all known data is used to develop the code, the only way it can be verified is to wait and see if the predicted trend occurs at times beyond that which was used to develop it. So far the time beyond the development period is too short to judge the models, but they seem to be heading the wrong way. Since all of the models seem to make similar assumptions such as constant relative humidity, and ignore many possible forcing such as Sunspot effects on cloud formation, they may agree with each other all they want, but all may be bad models.

Incidentally, these kinds of models are sensitive to initial conditions. This is exploited—well, acknowledged and lived with—in the operational sense with ensemble forecasting.
.
Absolutely correct William .
I remember somebody (Dan Hughes ?) having quoted G.Schmidt that the climate models present <b<POSITIVE Lyapounov coefficients .
I don’t know if the quote is accurate but if it is , it only shows misunderstanding of chaotic systems .
In any case the resolution of the climate models is too coarse to estimate Lyapounov coefficients .
On top this estimation works well only for purely temporal systems (e.g space independent) , spatio temporal climate models couldn´t do the estimation even if they had the right resolution what they have not and will never have .
So even if it is virtually certain that there are positive Lyapounov coefficients for climate equations , as nobody can write them accurately down we can’t know what they are .
And even if we had the equations , it´s clear that no climate model could be able to make this computation .
.
Anyway …
What we do know is that the “weather” is highly sensitive to initial conditions .
Climate models dealing with EXACTLY the same physical laws as those that govern the weather must exhibit the same sensibility .
Averaging (spatially or temporally) changes nothing because as (hopefully) everybody knows an average of a chaotic variable is a chaotic variable .
The only thing averaging does is that it eliminates high frequency components but the low frequency components are still chaotic albeit , trivially so , at larger time and space scales .
There simply is NO mathematical transformation be it an average or whatever else that would magically transform a chaotic variable in a non chaotic variable .
.
So it is precisely because the atmosphere-hydrosphere-cryosphere system is highly sensible to initial conditions at the fundamental level of the physical laws that it stays so after all and any mathematical transformations whether they “smooth” , “average” , “rotate” or whatever .
That´s also why what you say is correct – the only way to simulate such a system is to systematically feed it with updated numbers to make sure that it stays on track .
That´s exactly what weather models do .
.
Once that it is demonstrated that due to its properties , the system can’t be deterministically forcast beyond a very short horizon , comes the question if there is not SOME invariant probability distribution function that would at least allow to estimate the probabilities of future dynamical states of the system .
For spatio-temporal chaotic systems the answer on this question is generally no .
All this has really nothing to do with steady states or small deviations from some mythical “equilibriums” .

The MAGIC models used to “prove” acid rain are the best example of how model predictions go horribly wrong when only the physical processes that fit the agenda are allowed to be incorporated into a model. The acid rain models assumed the primary introduction of acidity to surface waters was from SO2 emissions. The acid rain models said if we controlled SO2 we would see pH improvements in 20 years. We haven’t because the primary source of acidity was never acid rain but natural soil derived organic acids. What is particularly sad is some scientists now claim the failure of pH recovery in surface waters is the result of climate change accelerated production of organic acids. ( At least they are admitting to the role of organic acids but too late for the scientists whose careers were destroyed for questioning the acid model assumptions)

Acid rain was the template for climate change especially in naming a crisis after a natural phenomena. All rain is acid – always has been- always will be– climate changes- always has always will. How could any scientist when pushed say acid rain did not exist? (“natural rain” is approx. a pH 5.5). However an admission as to the “reality” of acid rain was taken to support an agenda far beyond the scientific reality of the acid rain threat.

P.S
I think that one would also eliminate once for all analogies with aerodynamics .
Those analogies are so deeply misguided that they are not even wrong .
People say – “you see planes fly even if the flow is chaotic . So the models have an ability to provide useful predictions” .
Well to get something useful from a chaotic system model one has to DRAMATICALLY restrict the phase space domain .
That happens either by looking at very short times (that´s what weather forecasting does) , at very small spaces (that’s what aerodynamics does) or at very small ranges in dynamical parameters (that’s what CFD does) .
But the climate ?
It must take the biggest space available (the whole Earth) , look at huge times (centuries) and work with the full natural range of the dynamical parameters .
It´s the worst possible situation where no simplifications can take place and the system has to be treated as a fully chaotic system .
Clearly nothing to do whatsoever with a flow of air around the few meters of a plane wing .

Date free models built with no observations are common. They have no utility except possibly as thought experiments.

Gavin and Lucia seem to think that climate models are a) thought experiments, or b) use minimal actual observations but we just don’t like to talk about them.

Either way, the disconnect between the models and reality (the observations) is deep and wide. So be it. I accept that. The models have no predictive power vis a vis reality, since reality is not important to the modelers. They have some other purpose in mind. They even eschew the word “prediction”.

Great. Except that some powerful politicians and moneyed interests are using the hypothetical models to impose draconian regulations on society in the mistaken notion that the climate models have actual predictive power. But the modelers insist that they don’t, at least they said so above. (I would be remiss not to point out that climate modelers are not so blithe and forthcoming about the unreality of their models when testifying before Congress).

Why then don’t the modelers speak up, and tell the politicians that their models have been completely misinterpreted, that prediction is not what the models do, that any and all model outcomes are hypothetical, and that imposing penalties on society on the basis of hypothetical models is wrong?

Because the if they don’t make that clear, the modelers will someday be dragged kicking and screaming through the streets, as oppressed society blames THEM, not the power-hungry money-grubbing politicians.

The very idea that scientists are somehow apart from society, as well as reality, is naive, disingenuous, and dangerous to all concerned. Some modelers apparently want it both ways: their models are just thought experiments, and their models predict actual dire futures. Not smart. Blame will be laid and is being laid. The future for such dissembling modelers is not bright, regardless of the future of the climate.

Gavin – take a look at the IPCC 2007 report.
It is clear from the main temperature charts with reducing reducing error bands with time, that the models used were tuned to the period 1976 – 2000.

Your models should be restricted to data from 1880 to 1975 and specifically tuned to the period from 1943 to 1975.
Then use them to predict 1976 to 2010.
That’s 35 years, which is long enough to avoid the “that’s not climate, only short term weather fluctiations” song so often used to fend off criticism.

I bet you won’t show us the results.
Good luck and keep your overcoat handy.

” What I’m really curious to see done is to have Gavin turn over his parameterized model to an independent group, who would then re-run it with a constant CO2 input of 280 ppm…”

We know what would happen. Without exogenous forcing (e.g., volcanoes) not much of anything.

Briggs,

“…nobody–not a soul –would build a wing based solely on the basis of a theoretical model, fix the wing to a plane, and then strap themselves in and take off with a load of passengers.”

Except maybe the Wright brothers, minus the passengers of course. 😉 And all those other crazy flight pioneers who jumped off barns, cliffs, etc.

I really enjoyed and agreed with this post. I’m in financial services and see far too many models that have been developed and then tested “in sample,” and then reporting incredible predictive powers. The slicker “snake-oilers” will tell you they used “monte carlo” simulations which is supposed to prove infallibility. Unfortunately like all modelling techniques, monte carlo can only test within the parameters it is fed. Thus the models couldn’t predict the most recent market meltdown any better than they predicted the market meltdown in 1987.

It would seem that in climate science the number of variables, and our limited understanding of said, creates a system even more chaotic than the worldwide financial system. I just don’t see how they can get it right. Climate Science today seems to be dwelling in the extreme tails of a bell distribution instead of within the mean +/- (albeit very generously given) a full quartile.

Keep up the good work! It’s very much appreciated.

P.S. Look forward to Part II!

Pat Moffitt: at 4:19 pm
My 80s USAR enlistment duties took me to Heidelberg each year. Among the data I reviewed were descriptions and warnings re European ¨forest death.¨ My civilian employer, a newsprint mill, offered education benefits for study related to my work. I chose to pursue independent study, at my alma mater – VCU, on acid rain.
My career was not ruined. My GPA, however, plummeted: only years later did I realize that my failure to find a connection WAS my finding.
Thank you, Briggs et al. Keep up the great work-

Apologies for the noob question, but having digested William’s (or do I say “Briggs”?) first post and believing I understanding the point, I’d like to know more about the process used to build the anomaly grid in the first place.

I’ve tried JFG’ing it, but I’m struggling to get to something clueful. Any links would be appreciated.

Climate I: Is the debate over?

http://www.tvo.org/TVO/WebObjects/TVO.woa?videoid?71356252001

Why do I think climate models are useful? Because they demonstrate skill and have made predictions that have turned out to be valid.

Some examples:
1) Hansen et al (1992) predicted the impact of Pinatubo aerosols on temperatures accurately before they happened. Subsequent research has shown that this would not have been the case if the water vapour feedbacks or ocean heat uptake had been very wrong. Matches to radiation anomalies, WV, stratospheric temperatures and wind pattern changes to observations are all excellent (Hansen et al, 2007, Shindell et al 2004).
2) Rind and Peteet (1985) showed that an estimate of ice age ocean temperatures were inconsistent with a suite of land based proxy information using an atmospheric GCM. They were right.
3) Hansen et al (2005) and Domingues et al (2009) showed that predicted ocean heat content anomalies matched the current best estimates. Note that they didn’t match the ‘best’ estimate at the time the models were run. Models instead predicted that that estimate was flawed (AcutaRao et al). (i.e. the models were *not* tuned to get the ‘right’ answer).
4) Hansen et al (1988) shows substantial skill over a ‘no change’ forecast for global mean temperatures in the 1984-2009 period.
5) Model climate sensitivities which are emergent properties of the underlying processes (and are not easily tunable) match constraints derived from paleo-climate (Annan and Hargreaves (2006), Kohler et al 2010)
6) Models designed for late 20th Century conditions reproduce large parts of the orbitally driven climate changes in the mid Holocene (increased rainfall in the Sahara/Sahel, increased summer time warming etc).
7) Models designed for the late 20th Century show good matches first time out for the climate impacts of a slow down in the North Atlantic ocean circulation 8200 years ago driven by the final collapse of Lake Agassiz (LeGrande et al, 2006).
8) Coupled Model estimates of the Last Glacial Maximum climate are all in the right ballpark (given of course that boundary conditions are not known precisely). This would not be the case if climate sensitivity was either negligible or extremely high.

I could go on, but none of these things would have occurred if climate modelling was fundamentally impossible. Many of them involved challenges to what was thought of as the ‘best’ observations at the time, many of them are true ‘out of sample’ tests.

If all we were doing was constructing best fits to current observations and updating them as new information came in, none of this would work. The science is done when we have a conflict between the models, the observations and occasionally how the comparison is being done. The MSU data are a good example – the MSU2 record did not seem to match model estimates of mid-troposphere temperatures, but then people realised that you had to account for the tail of the weighting in the stratosphere where it is cooling, MSU-LT initially showed cooling in contradiction with model estimates that suggested it could not be that different from the surface records (mainly ocean temperatures). Turns out there were a number of problems in the MSU analysis etc.

As I suggested above, please read the papers that discuss the building and testing of models – it is nothing like the cartoon you suggest – i.e Schmidt et al (2006).

And although this might not answer every precise issue raised here, this FAQ covers a lot of it: http://www.realclimate.org/index.php/archives/2008/11/faq-on-climate-models/

Then download the code and look for yourself for the hidden subroutines that you seem to think must be reading in observed temperatures and fixing the physics accordingly (good luck with that one): http://www.giss.nasa.gov/tools/modelE/

Try running it yourself and see how quickly the initial conditions of the ocean are forgotten (about 100 years for the upper ocean and atmosphere, 500 years or so for the deeper ocean drifts), or how quickly weather patterns diverge if you change a single digit at the start (a much easier test for the home modeller).

Try downloading some output: http://data.giss.nasa.gov/modelE/ and seeing how you can get a similar climate signal regardless of the weather for the response to a volcano or the long term trend in greenhouse gases.

Try and learn something instead of just making up stories to convince yourself that the models aren’t worth bothering with.

A while ago I downloaded the Hadcrut3 gridded dataset. It was “anomalies” and I wondered how to get the actual values. There’s another dataset “Absolute” which gives, for each gridcell in a month the values subtracted to produce anomalies. I got even more confused. The full dataset has missing data for something like 40% of the gridcells but the absolute set has a value for every gridcell every month. I found out a little later that the cells of the absolute dataset were derived from the observations by running a climate model that matched the available data and using the model’s values for the remaining empty cells.

So all of the anomaly data incorporates model data. This is then averaged over the Northern and Southern hemispheres and then these are averaged again to give the “Global Average Temperature Anomaly”. Which is then presented as observational data. Which is compared to model outputs.

There’s no escape from models.

I don’t have a science background and so the vast majority of the content contained in the above 30 comments is well over my head.

There is something I’m quite equipped to note – and understand – however. And that’s that the comments from Gavin don’t just discuss the subject matter. They veer off into nastiness, condescension, and accusation.

from Gavin comment #1: “Try actually downloading a climate model and see for yourself how they work. Or, you know, try reading a paper.”

Gavin comment #2: “Try and learn something instead of just making up stories to convince yourself that the models aren’t worth bothering with.”

Earth to Gavin: Talking down to people as if they’re four-year-olds hasn’t worked for climate scientists so far. Why not try a little courtesy and respect?

Climate Change 2007: The Physical Science Basis
Summary for Policymakers

by Vincent Gray

My disillusionment with the whole process began very early. The very first Report was dominated by an attempt to persuade the value of computer models. Climate data on the supposed warming were largely confined to the end of the Report, presumably to draw attention from their lack of confirmation of the models. This was concealed by claiming that the size of this warming was “broadly consistent” with the models.
I could claim my first success, that after my comment, all subsequent IPCC Science Reports have placed the climate data at the beginning. But that was not the only cause for suspicion. In the first draft of the IPCC WGI 1995 Report there was a Chapter headed “Validation of Climate Models” I commented that this word was inappropriate as no model had ever been “validated’, and there seemed to be no attempt to do so. They agreed, and not only changed the word in the title to “evaluation”, but they did so no less than fifty times throughout the next draft. They have rigidly kept to this practice ever since.
“Validation”, as understood by computer engineers, involves an elaborate testing procedure on real data which goes beyond mere simulation of past datasets, but must include successful prediction of future behaviour to an acceptable level of accuracy. Without this process no computer model would ever be acceptable for future prediction. The absence of any form of validation still applies today to all computer models of the climate, and the IPCC wriggle out of it by outlawing the use of the word “prediction” from all its publications. It should be emphasised that the IPCC do not make “predictions”, but provide only “projections”. It is the politicians and the activists who convert these, wrongly, into “predictions.”, not the scientists. An unfortunate result of this deficiency is that without a validation process there cannot be any scientific or practical measure of accuracy. There is therefore no justified claim for the reliability of any of the “projections’
They have tried to draw attention from the undoubted fact that the models have not been shown capable of making predictions by seeking the “opinion” (or “guess”) of a panel of “experts”, all of whom have a financial stake in the outcome, and apply to these guesses levels of “likelihood” which have even been given a spurious numerical value. If the experts were employees of oil or coal companies, and their opinions were undesired there would be an outcry. As the “experts” are employees or recipients of funding from governments promoting the notion of greenhouse warming, criticism is not heard.
.
The 2007 Summary for Policymakers
The current document is really a Summary BY Policymakers, since it has been agreed lineby- line by government representatives.

http://www.pensee-unique.fr/GrayCritique.pdf

Comment on the Nature Weblog By Kevin Trenberth Entitled Predictions of climate

This is remarkable since the following statements are made

1. IN FACT THERE ARE NO PREDICTIONS BY IPCC AT ALL. AND THERE NEVER HAVE BEEN.
2.None of the models used by IPCC are initialized to the observed state and none of the climate states in the models correspond even remotely to the current observed climate.?
3.Moreover, the starting climate state in several of the models may depart significantly from the real climate owing to model errors. I postulate that regional climate change is impossible to deal with properly unless the models are initialized.
http://pielkeclimatesci.wordpress.com/2007/06/18/comment-on-the-nature-weblog-by-kevin-trenberth-entitled-predictions-of-climate/

Real Climate’s Agreement That The IPCC Multi-Decadal Projections Are Actually Sensitivity Model Runs

Now, from an unlikely source (Real Climate) have come the statements

http://pielkeclimatesci.wordpress.com/2008/04/11/real-climates-agreement-that-the-ipcc-multi-decadal-projections-are-actually-sensitivity-model-runs/

Matt:
Are you saying that “modelers” have to call their shots(predictions) on a consistent basis?

It also strikes me that models can be useful, i.e., make accurate predictions, but be seriously and knowingly incomplete and, therefore, necessarily projects greater certainty than can exist.

Briggs,

Your point about overconfidence is well made. Massimo Piattelli-Palmarini in his book Inevitable Illusions: How Mistakes of Reason Rule Our Minds, John Wiley & Sons, Inc. 1994, summarizes the classic work of Fischhoff, Slovic, & Lichtenstein from their 1977 Journal of Experimental Psychology paper.

Piatelli-Palmarini says, “This study, as well as others, shows that the discrepancy between correctness of response and overconfidence increases as the respondent is more knowledgeable…the level of accuracy increases, yes, but the level of overconfidence increases to a far greater degree. …this over-confidence is at its greatest in our own area of expertise–in short, just where it can do the most damage.”

I very much appreciate Gavin Schmidt’s willingness to address concerns on this forum. If Gavin (or others) would be so kind, I have a few questions related to the FAQ referenced in his prior post:

Re. Do models have global warming built in?

“If left to run on their own, the models will oscillate around a long-term mean that is the same regardless of what the initial conditions were”.

– Is this ocillation due to different initial conditions (as implied by the FAQ section addressing “wiggles” in the output) or is there inherent variability (chaos) in the model?
– If there is inherent volatility in the model, how many runs are typically required for a simulation?
– What is the current mean global temperature as obtained by the “base mode”model?

Re. What is tuning?

“With these approximations and empirical formulae, there is often a tunable parameter or two that can be varied in order to improve the match to whatever observations exist.”

and

“Surprisingly, there are very few of these (maybe a half dozen) that are used in adjusting the models to match the data.”

– Are these tuned parameters scalar (i.e., global) values or are they vectors and/or arrays (i.e., temporal / spatially differentiated)?

Kind Regards,

Frank

Briggs

In your reply to Gavin you write

Your points show that climate models do a respectable job of mimicking certain climatic signals. Very well; I have never disputed that either. In my “cartoon” way, I say that that means models “fit historical data.” This is a necessary but not sufficient step to prove that the theory behind the models is valid.

However, several of Gavin’s examples – 1, 3 and 4 – are not examples of fitting historical data but of predicting aspects of future climate. And, as I understand you, this is exactly what is required for validation of a model. Is this – admittedly limited – successful prediction enough to consitute validation? Be interested to hear your thoughts.

The Carl Wunsch Complaint
http://climateaudit.org/2008/07/22/the-carl-wunsch-complaint/
http://climateaudit.org/?s=ofcom

Global Warming & Greentech: Why global warming is unlikely to be a safe area for investment by Richard Lindzen, April 14, 2009

Another colleague, Carl Wunsch, professionally calls into question virtually all alarmist claims concerning sea level, ocean temperature and ocean modeling, but assiduously avoids association with skeptics; if nothing else, he has a major oceanographic program to worry about. Moreover, his politics are clearly liberal. . Perhaps, the most interesting example is Wally Broecker, whose work clearly shows that sudden climate change occurs without anthropogenic influence, and is a property of cold rather than warm climates. However, he staunchly beats the drums for alarm and is richly rewarded for doing so.
http://www.ecoworld.com/global-warming/global-warming-greentech.html

A Climate of Belief
The claim that anthropogenic CO2 is responsible for the current warming of Earth climate is scientifically insupportable because climate models are unreliable
by Patrick Frank

Acknowledgments
The author thanks Prof. Carl Wunsch, Department of Earth, Atmospheric and Planetary Sciences, Massachusetts Institute of Technology etc…. for reviewing a prior version of this manuscript
http://tinyurl.com/635bf8

The Present (Circulation) Is the Key to Understanding The Past(Circulation)
Carl Wunsch
Cambridge UK
Leverhulme Meeting
March 2008
One uses GCMs for the modern world in two distinguishable ways:
(1) Run them forward from some initial state in an “extrapolation” mode.
(2) Use them to interpolate data over finite time intervals.
All time-stepped models in extrapolation mode exhibit error growth. The only exceptions are kinematically and dynamicaly trivial (the stopped-clock analogy
Snip
It is urgent to come to some understanding of error growth in GCMs in extrapolation mode. Mere assertion that feedbacks or compensating errors occur is not science. Determining error growth is difficult, but hardly impossible
http://www.leverhulmeclimatesymposium.org/uploads/documents/cambridge_programme/Wunsch_oceancirc.ppt

brent comments below;

When I saw the original release of TGGWS, I was impressed with the openness of Carl Wunsch in discussing the modeling issues.
I had no problem in relating to what he was saying, particularly because I have a modeling background myself. I’m an old retired petroleum downstreamer who spent most of his career with a heavy involvement in optimization. I had already figured out for myself that the GCM’s would not be capable of prediction, and that the IPCC “projections” were no more than prophesy.
What Wunsch was saying in essence was we cannot even inch forward on learning anything (in a science that is in it’s infancy) without big complicated models. (I agree with this)

Politically Wunsch was embarrassed after the original TGGWS was released because of the seeming association with sceptics.
I think Steve McIntyre did an excellent job summarizing the various OFCOM complaints including Wunsch’s

Subsequently, I was certainly intrigued, when Pat Frank’s paper came out to note in the acknowledgments that he thanked none other than Carl Wunsch for reviewing an earlier draft of his paper.

Later I dredged up Carl Wunsch’s Leverhulme meeting link above, where Carl Wunsch criticizes colleagues regarding the very issue that was the central tenet of Pat Franks paper.
He even tells his colleagues that they aren’t doing “science”‘, and it’s clear that the climate establishment hasn’t a clue, or even tackled the question of propagation of error in their “non-predictions” (what IPCC calls projections).

So, for public consumption Carl Wunsch doesn’t want to be seen to be offside with the “politically correct agenda”. However he’s pretty critical when it’s supposedly just amongst “colleagues”

cheers
brent

Buy low, sell high: I have a model that predicts future share prices of companies in 5 distinct industries. It has predicted the last peaks and lows for each of these industries. My model is based on a deep understanding of the underlying dynamics and interactions among all known key economic drivers, including interest rates, Government spending, raw material prices, etc.

What would persuade you to buy my model for $250K?

Some curious points. Dr. Schmidt says:

4) Hansen et al (1988) shows substantial skill over a ‘no change’ forecast for global mean temperatures in the 1984-2009 period.

So a 1988 model is used to hindcast to 1984, and forecast 20 years forward. Yet we have been told that GCM models are only skillful in 30+ year forecasts (scenarios) and not used for hindcasting. Which is it, short range or long range, hindcast or forecast? And did the 1988 GCM correctly project the 1995 to 2009 lack of warming that has been the subject of much debate in recent months?

6) Models designed for late 20th Century conditions reproduce large parts of the orbitally driven climate changes in the mid Holocene (increased rainfall in the Sahara/Sahel, increased summer time warming etc).

Wait a second. I thought the proxies used in the IPPC reports denied the existence of earlier climatic optimums, esp. the Medieval Warm Period. And that GCM’s used atmospheric GHG’s as model forcings, not insolation. Proxies for GHG’s in pre-measurement eras are inaccurate and imprecise. That measurement error is not accounted for in the GCM’s. Instead, assumptions are based on unwarranted levels of certainty about paleo conditions.

7) Models designed for the late 20th Century show good matches first time out for the climate impacts of a slow down in the North Atlantic ocean circulation 8200 years ago driven by the final collapse of Lake Agassiz (LeGrande et al, 2006).

That statement is too generous. First much of the “match” is based on the aforementioned highly uncertain paleo proxy record. Second, other paleoclimatic events, such as the Younger Dryas, are poorly understood and poorly measured, even by the paucity of proxies that do exist. To say that GCM’s model paleoclimatic events that are not well known is giving credit where none has been established.

I know Pat Frank frequents some other sites but if someone knows more, it would be very useful. Amongst other things, the guy can write.

Briggs–

Incidentally, these kinds of models are sensitive to initial conditions. This is exploited—well, acknowledged and lived with—in the operational sense with ensemble forecasting.

I know the weather models are. Climate models use long “spin ups” that, in principle, should reduce or eliminate the sensitivity to initial conditions. (I couldn’t say if it’s eliminated. We’d have to grill Gavin on how many variations of ‘spin up’ are done for each model.)

SteveF–

Can’t you, however, develop your model parameterizations using the 1910 to 2009 data set and then run the model (understanding that the base conditions e.g. CO2 concentration will need to be adjusted)

Not really, because it’s not done that way. I’m going to make up a wildly oversimplified example that might make Gavin cringe. Suppose you were a modeler back in 1980, and you were creating a climate something you could run on your radio shack whatever. You recognize that you are going to transport water vapor around. You also recognize that if air rises, the tempeature drops, and the water condenses into droplets.

You need to “parameterize” this– that is, create a codable formula that predicts when droplets form based on other variables also computed in the code. So, you decide to take data today. You make some observations during a 3 week campaign in Illinois, send ballons up and based on your data, you conclude that coulds form when the the relative humidity hits about 80%. ±10%. You get a paper and discover some guy did a similar experiment in outer slobbobia, and got 70%±15%. Someone else got 65% ±5%. Or so everyone claims. (Maybe you find an obscure group who has an entirely different theory and thinks the level of dust in the air matters– but you think that’s too complicated.)

So, what do you pick for your model? You have a big range, you can justify. But no matter what you pick, you didn’t pick it based on 1910 to 2009 data. You picked it based on these field experiments measuring relative humidity and dectecting clouds using weather balloons.

Now, in principle, you oculd pick your critical relative humidity and all other paremterizations, then run your model and tweak them until you match the temperature series for 1910-2009 as best as possible. But, in practice, this is much too computationally intensive, and, for the most part, you can’t do that. (You can do other things– but not quite that.).

Dearieme.

Lucia, your examples happen to involve models or situations where the long run solution is a steady state. But surely climate is nothing like that? I’ve written lots of models that I could run either way in time: appeals to the Second Law wouldn’t matter a button for them.

Well, climate is nothing like that. But, to some extent climate model runs are like that. The modelers apply the forcings they think make sense for some year far back in time– say 1850. They freeze those forcings, and run year after year after year. Then after some number of years say 300) They start varying forcings as they think forcing varied during the 20th century. So, in climate model runs, the models do start from a quasi- steady state.

MikeD

Gavin and Lucia seem to think that climate models are a) thought experiments, or b) use minimal actual observations but we just don’t like to talk about them.

I don’t think climate models are thought experiments. I was merely pointing out that the way Briggs described the influence of the data describing initial conditions wasn’t quite right. It appears he agrees.

SteveE

Except maybe the Wright brothers, minus the passengers of course. And all those other crazy flight pioneers who jumped off barns, cliffs, etc.

The Wright brothers built windtunnels to test theoretical models for predicting lift!

Gavin

Why do I think climate models are useful?

I see you are answering a question no one ask. I seem to recollect you’ve answered this precise unanswered question before. Clearly, it must be the right question, because you ask it of yourself and answer it so often! 🙂

Mysteriously, you did not answer Briggs question which was

How do you know—by what specific measures—that our best climate models are accurate?

I wonder if you ever will decide to not substitute “useful” for “accurate”. Well, you do prefer rebutting strawmen, don’t you?

As for your closing statement:

Try and learn something instead of just making up stories to convince yourself that the models aren’t worth bothering with.

It’s always nice to see you wrote a post that not only opens with a strawman, but closes with one! Bravo, two snaps and around the world! ( Briggs does not appear to have claimed “convinc(ing) yourself that the models aren’t worth bothering with”. Or are you arguing with someone else? )

Bernie and Briggs,

Thanks for the heads up. Great piece (h/t Bernie) the guy can write.

It occurs to me though that we’re talking past each other. Gavin defends important work. Though as Frank points out it’s not work that we can use to make meaningful policy decisions today. Climate scientists need to continue to improve their modelling, which is only going to take time and better data.

The villain in this piece is the IPCC. It has polarized the argument because it has framed the argument with incomplete science (which isn’t fair to scientists because much of it is opinion–based on the policy guides–not science). Climate scientists have got caught up in the hype and the spotlight and are falling into the over confidence trap because they’re being cheered on by the IPCC, the NGOs and they’re being fed with taxpayer dollars.

Pat Frank provides the perfect perspective. He gives a straightforward analysis of what the models do say and what they don’t say. There’s so much we don’t know. Creating policy based on what we think we know could be more catastrophic to real, living people than the IPCCs worst case scenario will do to a future generations.

Lucia,

Thank you for your response I’ve seen how busy you are in the blogosphere, it’s appreciated. I’m not arguing with your logic. It just seems to me that you’re creating a narrower and narrower margin of error that isn’t supported by observation. When you look at Pat Frank’s projected error http://tinyurl.com/635bf8 based on your logic the error bars at the end of the century are ridiculous. I’m in business and I could never commit a capital expenditure let alone an operational expense to something like that. When I suggested the out of sample test (believe it or not) I was trying to find a way to believe that the models are more accurate than they are.

I can understand Gavin’s need to build strawmen (bless you for calling him on it). He’s convinced himself that the value of his work is dependent on a scenario he didn’t create (IPCC did) and is playing out the prisoner’s dilemma.

The modelling is important work that has made huge strides in the past 20 years. It’s just not robust enough to hog tie the world’s economy nor ensure that the third world never has a chance to make it to the first.

Cheers!

Lucia,

“The Wright brothers built windtunnels to test theoretical models for predicting lift!”

I stand corrected. It’s still pretty f***ing crazy to do what they did ;-). As Briggs said they sure weren’t going to load up the folks at Kittyhawk for a quick spin around the town based on those windtunnel tests.

Steve E:
The plaudits really go to Brent. IMO, Pat Frank writes like Montford and argues with McIntyre’s precision. That’s a tough combo.

Apologies to Brent. Thanks for the link best piece I’ve read on models.

Thanks to Bernie and Briggs for the tip about the Pat Frank article.

I have made the point about error propagation many times in arguments with people that insist that GCMs are “accurate”. They seem impervious to the actual meaning of the word accurate and use it interchangeably with “precise”.

In my physics undergraduate work if one were to submit an experimental write-up that had error estimates an order of magnitude greater than the claimed precision of the measured phenomenon it might produce laughter as a response (along with a poor grade).

If the student then insisted that the the entire world population should make sweeping changes to their lifestyles based on these results the student would have been quietly escorted to psychiatric ward of the university hospital.

Lucia, Briggs

Lucia rightly calls Gavin on refuting a straw man arguments. However, it strikes me that Briggs’ characterization of
Gavin’s argumement is also something of a straw man. Briggs responds to Gavin

Your points show that climate models do a respectable job of mimicking certain climatic signals. Very well; I have never disputed that either. In my “cartoon” way, I say that that means models “fit historical data.” This is a necessary but not sufficient step to prove that the theory behind the models is valid.

However, as I pointed out, three of Gavin’s examples – 1, 3 and 4 – are not examples of fitting historical data but of predicting aspects of future climate. Almost everyone posting here seems to have ignored this. Now maybe the predictions aren’t sufficiently numerous or of sufficiently high quality to constitute validation: I’m sure there are many here who can comment knowledgeably on this. But, nevertheless, it does appear that predictions have been made!

Here is a different cut on models – http://www.americanthinker.com/2010/03/the_granularity_of_climate_mod.html – it is not as comprehensive and addresses a somewhat different aspect of models IMHO. It seems more pertinent to the question of how exactly do modelers estimate the validity of their models.

Lucia

“…I really don’t know clouds…at all”

It seems awfully presumptuous to suggest that a modellers point-of-view is ground zero. Frank’s base position seems to carry–at least as much weight–as any modeller. To say he doesn’t see their point of view suggests that the modellers and their anomaly based view is the starting point for all debate.

You may still be right, but I’d like to see your argument for dismissing Frank. Why? For a layman like me his argument is lucid and appears to make sense.

Lucia,

I think this is where I am:

“I would not go so far as Briggs seems to and say the comparisons don’t count, but they certainly count less. I would go so far as to suggest that they count quite a bit less than a modelers who gets pleasure from seeing his models do well might wish to believe it counts.
What makes the comparisons count even less is that you get to progressively “improve” the models, and many of the choices in the improvements are driven by noting points where the models deviate from existing data, and incorporating changes you believe will remedy that. While this is a necessary part of science and entirely rational, it does make the later comparisons “count” much less than you, Gavin, might wish to believe they count.”

But where does that leave us? To me it, at least, reinforces Frank’s conclusions that models as they currently exist are inadequate to project (predict?) until 2100. Otherwise we have to concur with Gavin whose 100 year conclusions are based on changing points where the models deviate from the existing data.

To me Gavin isn’t wrong in his model development, in fact, I think that is what you have to do to improve accuracy and precision. But both are wanting in a century based time scale.

It seems to me that there is nothing wrong per se in the incremental improvement of models based upon testing them against observations. What else is a modeller to do? Clearly the conundrum is at what point and how one argues for the accuracy of one’s model while still incrementally improving them.

Steve E, Bernie,

Sadly one has to pay very close attention to nuance and semantics.

The AGW true believers have two basic teams.

The “A” team is the GCMs

The “B” team are the “Hokey Team” Studies.

All the rest is armwaving.

As noted in the Vincent Gray note I posted,
http://www.pensee-unique.fr/GrayCritique.pdf
the IPCC agreed with him that the “A” team, that is the GCM’s had never been validated in the sense that he means. Instead of admitting that GCMs are “incapable of prediction”(I.E Briggs Accuracy Issue), the IPCC outlaws the word “prediction” (relative to the GCMs, ).

Briggs tried to get at this essential central issue by pointedly asking about “accuracy”. and, Lucia followed up by pointing out how this was evaded in favor of drek (I like the characterization Lucia).

It’s never been established that the GCMs have predictive power, I.E. the accuracy issue, but the advocates don’t want the sheeple to understand this

They loudly tout their storyline (which is exactly what it is, storytelling), and when they do deign to talk about “predictions”, what they cite if considered in a limited sense as “predictions” at all, are trivial issues.(relative to the larger question, as noted by Lucia).

One of the most amusing exchanges I recall was Pielke Jr trying to delve into what “observations” would ever lead the advocates to admit that the GCMs were falsified
(note that Pielke Jr does not necessarily clue in to the particular sense in which the word “prediction” is a sensitive issue for the advocates. He tends to use the term more loosely )

Read the following closely

The Consistent-With Game: On Climate Models and the Scientific Method
http://tinyurl.com/22rhrg

Global Cooling Consistent With Global Warming
http://tinyurl.com/4thuxx

Pielke Jr concludes that there is nothing that would lead the advocates to admit that their precious GCMs are falsified. (The God hypothesis. All evil flows from CO2 🙂 )

Of course there actually is no need to falsify the GCMs . They are false, because they’ve never been established to have “predictive power”, I.E. accuracy, in the first place 🙂

Sometimes the advocacy crowd slips up and a bit of truth is revealed as per following links. Read closely

Real Climate’s Agreement That The IPCC Multi-Decadal Projections Are Actually Sensitivity Model Runs
http://tinyurl.com/yzpjg3y
Comment on the Nature Weblog By Kevin Trenberth Entitled Predictions of climate
http://tinyurl.com/ycunacr

It’s not necessary to get hung up on any issue of whether Pat Franks particular specific calculations are appropriate. What is more important is that he does describe very well certain principles.
The actual point I wanted to make with an earlier post was that none other than Carl Wunsch agrees in principle with the central tenet of Pat Franks paper.I.E. That there is a need to understand propagation of error through the GCMs. And that it is not understood now.

Wunsch even tells his colleagues that what they are doing “isn’t science”

Once again

The Present (Circulation) Is the Key to Understanding The Past(Circulation)
Carl Wunsch
Cambridge UK
Leverhulme Meeting
March 2008
One uses GCMs for the modern world in two distinguishable ways:
(1) Run them forward from some initial state in an “extrapolation” mode.
(2) Use them to interpolate data over finite time intervals.
All time-stepped models in extrapolation mode exhibit error growth. The only exceptions are kinematically and dynamicaly trivial (the stopped-clock analogy
Snip
It is urgent to come to some understanding of error growth in GCMs in extrapolation mode. Mere assertion that feedbacks or compensating errors occur is not science. Determining error growth is difficult, but hardly impossible
http://www.leverhulmeclimatesymposium.org/uploads/documents/cambridge_programme/Wunsch_oceancirc.ppt

The issue is not whether the big complex models are useful. I agree as well as others , if we are going to even inch forward in understanding in a field that is in it’s infancy, that we would have to use such tools.

I’m going to close this post with a few other tidbits:

Pielke Jr

This means that from a practical standpoint climate models are of no practical use beyond providing some intellectual authority in the promotional battle over global climate policy. I am sure that some model somewhere has foretold how the next 20 years will evolve (and please ask me in 20 years which one!). And if none get it right, it won’t mean that any were actually wrong. If there is no future over the next few decades that models rule out, then anything is possible. And of course, no one needed a model to know that.
http://sciencepolicy.colorado.edu/prometheus/archives/climate_change/001413global_cooling_consi.html

ScientistForTruth (from WUWT Ravetz thread)

Climate Change And The Death of Science

climate change models are a form of “seduction”…advocates of the models…recruit possible supporters, and then keep them on board when the inadequacy of the models becomes apparent. This is what is understood as “seduction”; but it should be observed that the process may well be directed even more to the modelers themselves, to maintain their own sense of worth in the face of disillusioning experience.
…but if they are not predictors, then what on earth are they? The models can be rescued only by being explained as having a metaphorical function, designed to teach us about ourselves and our perspectives under the guise of describing or predicting the future states of the planet…A general recognition of models as metaphors will not come easily. As metaphors, computer models are too subtle…for easy detection. And those who created them may well have been prevented…from being aware of their essential character.
http://buythetruth.wordpress.com/2009/10/31/climate-change-and-the-death-of-science/

The Social Simulation of the Public Perception of Weather Events and their Effect upon the Development of Belief in Anthropogenic Climate Change
http://www.tyndall.ac.uk/sites/default/files/wp58.pdf

Just as the GCMs are “incapable of prediction” and that’s the alarmists “A” team, I think most people on this site would be well aware of the issues with the alarmists “B” team the “Hokey Team”, from following several sources, notably ClimateAudit

The AGW “A” Team are the GCM’s.
The AGW “B” Team are the “Hockey Team” studies

The importance to the alarmists are that both are “metaphors”, as the Post-Normal Scientists like Mike Hulme explain

http://mikehulme.org/wp-content/uploads/2009/10/Hulme-Carbon-Yearbook.pdf

http://www.americanthinker.com/2009/07/what_climate_change_can_do_for.html

Lindzen has a good insight into Hulme’s PNS at link below

http://ceiondemand.org/2009/10/26/cooler-heads-event-with-dr-richard-lindzen-on-cap-and-trade/

cheers
brent

P.S. My thanks to our host,Dr Briggs. This is one of the really useful blogs that help promote discussion and understanding, and I regret that I haven’t followed it as closely as it’s merit warrants.

PPS. Excellent thread with a lot of good contributions. Special thanks to Tom Vonk for some insights

To say he doesn’t see their point of view suggests that the modellers and their anomaly based view is the starting point for all debate.

(http://www.skeptic.com/the_magazine/featured_articles/v14n01_climate_of_belief.html )

No. That’s not what I’m saying. I’m saying that giving it a quick look, it appears to me that Frank may be making a mistake implementing first law of thermo because he doesn’t understand that excess heat accumulates based on the increment of excess heat above the steady state value. This is true in all engineering problems, not just models.

It may be that I am wrong about his error, but I don’t think so.

You may still be right, but I’d like to see your argument for dismissing Frank. Why? For a layman like me his argument is lucid and appears to make sense.

Dismissing? I don’t utterly dismiss him.

If all he did was note that GCM’s did not get the absolute cloudiness of clouds correct, and this matters in some way and stopped there, I would admit that fact decreases our confidence in GCM’s ability to correctly capture the physics of climate relative to models with simulations that better predicted the absolute cloudiness of models. After all, if they are off 10% on the average cloudiness integrated over the entire planet, why should we be confident they can detect changes in the level of cloudiness? Or that even if cloudiness is mostly unaffected, they get the sensitivity correct?

But that’s a much less serious deficiency than Pat Frank goes on to discuss.

The argument developed in the paragraph just prior to figure 4 dramatically overstates the severity of the problem because– in any thermo problem– whether in a GCM or a hand calc, being off 10% on the clouds affects your prediction of the quasi-steady problem. That is: all other things being equal, getting cloudiness wrong should result in the wrong temperature for the earth at steady state. Other than that, we don’t know the consequences. But whatever they are, the aren’t what Pat Frank suggests.

Think about this:

Suppose the model is 10% on any forcing wither anthropogenic or natural. Now, freeze the model forcings at this wrong level. Now, run the model without varying the forcing, starting with some guess about the ‘earth’s’ surface temperature.

Do you think that according to the models, the earth’s temperature will rise and rise, and rise and rise (or fall, fall, fall) forever as in Pat Franks figure 4? This wouldn’t (and doesn’t) happen in remotely reasoanble model that applies the 1st law of thermo and permits heat loss to increase as the surface temperature increases. Such a model will reach a quasi-steady state, where, for the most part, the earth’s surface no longer accumulates heat.

That model steady state might get the earth’s surface temperature wrong– but it’s a steady state. If forcings are off by 10%, I can’t be sure how how far that steady state will differ from the true earth temperature– but it’s sure as heck not 120C after 20 years. It’s not going to be 120C after an infinite number of years. In reality, the computed annual average earth’s surface temperature will rise to an assymptotic state. (It will be noisy– but still, noisy around some basic level.)

The fact is, the reasoning leading to Pat Franks figure 4 is simply wrong. Noting that, I don’t plan to spend any more time discovering other things that might also be wrong.

I for one did try Gavin’s suggestion and started reading the “2006_Schmidt_etal_1.pdf” document on model E’s construction. I got about 1/3 of the way through it before my head started to hurt too much.

On page 7 of the PDF (JOC page 159) I noticed some discussion of numerical instability in the dynamical solutions and how they were corrected. I can understand that there might be problems at the polar cusp of a spherical coordinate system used for the simulation, but what bothers me more is about how these problems are corrected. A diffusion method is discussed but is there a physical or numerical basis for it? We also see that there is “velocity filtering” on the whole solution surface to remove “two-gridpoint noise.” Near the end of the page there is another parameterized correction for gravity wave instabilities with some handy tunable constants (if there is physics involved why is tuning needed?).

Problems with numerical stability at the poles creates some concern for me because the poles tend to show more warming than the lower latitudes and should require more accuracy to get right.

With so many valid scientific “tricks” applied to get the model to work correctly, I find it very hard to believe it is useful for any purpose, let alone prediction.

Lucia, you wrote,

[L] >>… it appears to me that Frank may be making a mistake implementing first law of thermo because he doesn’t understand that excess heat accumulates based on the increment of excess heat above the steady state value. This is true in all engineering problems, not just models.<>The argument developed in the paragraph just prior to figure 4 dramatically overstates the severity of the problem because– in any thermo problem– whether in a GCM or a hand calc, being off 10% on the clouds affects your prediction of the quasi-steady problem. That is: all other things being equal, getting cloudiness wrong should result in the wrong temperature for the earth at steady state. Other than that, we don’t know the consequences. But whatever they are, the aren’t what Pat Frank suggests.<<

Lucia, what you described is not what I suggested. Please look at the second paragraph under Figure 3, where Figure 4 is discussed. This paragraph says, "In terms of the actual behavior of Earth climate, this uncertainty does not mean the GCMs are predicting that the climate may possibly be 100 degrees warmer or cooler by 2100. It means that the limits of resolution of the GCMs—their pixel size—is huge compared to what they are trying to project. In each new projection year of a century-scale calculation, the growing uncertainty in the climate impact of clouds alone makes the view of a GCM become progressively fuzzier.”

The wedge around the mean line in Figure 4 represents uncertainties, not heat excursions. I followed the very straight-forward rules for propagating uncertainties, such as found here, for example. A centennial temperature anomaly trend just involves summing the set of individual annual anomalies. If each temperature anomaly has an invariable +/-10% error, the uncertainty propagates through the trend as the running sum of the individual errors.

The climate mean temperature can stay within reasonable bounds, and it would still be true that the uncertainty in the calculated temperature would increase with every calculational time step. Eventually, the uncertainty would increase to the point that the calculated temperature would cease to have any scientific meaning; which translates as having no predictive value. That’s what Figure 4 is meant to show; uncertainty, not mean. Non-predictability happens very quickly with GCMs. The “perfect model” test by Matthew Collins (Skeptic reference 28) shows that the HadCM3 became incoherent vs. the test climate within 1 year. Demetris Koutsoyiannis has shown the same lack of predictive value by GCMs, using real comparisons.

[L] >>Think about this:

[L] >>Suppose the model is 10% on any forcing wither anthropogenic or natural. Now, freeze the model forcings at this wrong level. Now, run the model without varying the forcing, starting with some guess about the ‘earth’s’ surface temperature.

[L] >>Do you think that according to the models, the earth’s temperature will rise and rise, and rise and rise (or fall, fall, fall) forever as in Pat Franks figure 4? This wouldn’t (and doesn’t) happen in remotely reasoanble model that applies the 1st law of thermo and permits heat loss to increase as the surface temperature increases. Such a model will reach a quasi-steady state, where, for the most part, the earth’s surface no longer accumulates heat.<>That model steady state might get the earth’s surface temperature wrong– but it’s a steady state. If forcings are off by 10%, I can’t be sure how how far that steady state will differ from the true earth temperature– but it’s sure as heck not 120C after 20 years.<<

It has surprised me how many people misread Figure 4 in this way. I tried very hard to be clear that I was representing uncertainties, not temperature excursions. Sorry that I led you astray, somehow, Lucia.

I see, by the way, that Gavin has derogated my skeptic article, using false arguments. No surprise there. Please do examine Figure 1 and page 2 of my article SI. Nowhere will you find me claiming that log(0) = 1, or that logs have a finite limit at zero. There is an implicit assumption in that figure, which is that the forcing of CO2 is negligible at 1 ppm, and this forcing can be safely equated to 0 ppm CO2. Regrettably, I didn’t make that assumption clear and Gavin has opportunistically beat his alleged discovery drum ever since our debate on RC.

If you read the thread succeeding, as well as “the thread preceding” as Gavin coyly suggested, you’ll find post 461, part 3b of which explicitly shows that 1 ppm CO2 has zero greenhouse forcing. Post 461 refuted all that remained of Gavin’s substantive charges, apparently leading him to fixate onto an irrelevant argument about log(0), which seemed to have played well with the RC gallery.

I need to make a correction to my post above, where I wrote that, “A centennial temperature anomaly trend just involves summing the set of individual annual anomalies.” What I should have written is that the anomaly trend involves a running set of individual annual anomalies.

The reason error propagates as it does, is because GCMs calculate future climates in a step-wise way through time. The cloud error means that every climate will have an average (+/-)2.7 Wm^-2 uncertainty in forcing, which produces an uncertainty in each mean temperature. In every step, the average cloud error in that step produces a new temperature uncertainty, must be compounded with the uncertainty of the input prior mean temperature. The forward-propagating uncertainty increases in each calculated temperature, and that increasing uncertainty trend remains with the anomalies when the baseline temperature is subtracted (actually, the uncertainties would have to be convolved with the uncertainty in the baseline temperature, too).

In short, each climatological temperature mean has an average uncertainty due to cloud error, and that temperature plus its uncertainty is used as input for the time-wise calculation of the next mean temperature. The uncertainty in each prior mean temperature must be propagated forward into the uncertainty in each new mean temperature.

There are different ways of calculating uncertainty, of course, and since no one has ever published an uncertainty propagation through a GCM climate projection, we don’t know what the structure of the uncertainty looks like in a projected climate. So, I just chose to propagate the uncertainty in the most straight-forward way, given the algebra of my equation and the validating correspondence between the calculated line and the outputs of the GCMs, shown in Figure 2.

Pat:
Many thanks for clarifying the issues. You may want to drop Lucia a line at her website and reference her back to these comments.

gavin,

For entertainment value I read some modelE code here:

http://www.giss.nasa.gov/tools/modelE/modelEsrc/

Some code even has your name on it.

There is a module “param” and there are 19 files titled INIT:

INITS, INIT_CLD, INIT_DECOMP, … , INIT_VEGETATION.

That looks like an awful lot of parameters. I would have guessed that tweaking the input parameters, I could get about any climate output I want. Perhaps some of these inputs are certain, but I bet most are at least arguable. Is your experience running the model with different parameters contrary to that? That is to say, do you think I couldn’t find a somewhat reasonable set of parameters that showed a flat line, or cooling climate?

Younger Dryas? I hope climate models can’t predict ET impacts.

Bernie, thanks, I’ve dropped Lucia a line and a link.