“J’accuse! A statistician may prove anything with his nefarious methods. He may even say a negative number is positive! You cannot trust anything he says.”
Sigh. Unfortunately, this oft-hurled charge is all too true. I and my fellow statisticians must bear its sad burden, knowing it is caused by our more zealous brethren (and sisthren). But, you know, it really isn’t their fault, for they are victims of loving not wisely but too well their own creations.
First, a fact. It is true that, based on the observed satellite data, average global temperatures since about 1998 have not continued the rough year-by-year increase that had been noticed in the decade or so before that date. The temperatures since about 1998 have increased in some years, but more often have they decreased. For example, last year was cooler than the year before last. These statements, barring unknown errors in the measurement of that data, are taken as true by everybody, even statisticians.
Th AP gave this data—concealing its source—to “several independent statisticians” who said they “found no true temperature declines over time” (link)
How can this be? Why would a statistician say that the observed cooling is not “scientifically legitimate”; and why would another state that noticing the cooling “is a case of ‘people coming at the data with preconceived notions'”?
Are these statisticians, since they are concluding the opposite of what has been observed, insane? This is impossible: statisticians are highly lucid individuals, its male members exceedingly handsome and charming. Perhaps they are rabid environmentalists who care nothing for truth? No, because none of them knew the source of the data they were analyzing. What can account for this preposterous situation!
Love. The keen pleasures of their own handiwork. That is, the adoration of lovingly crafted models.
Let me teach you to be a classical statistician. Go to your favorite climate site and download a time series picture of the satellite-derived temperature (so that we have no complications from mixing of different data sources); any will do. Here’s one from our pal Anthony Watts.
Now fetch a ruler—a straight edge—preferably one with which you have an emotional attachment. Perhaps the one your daughter used in kindergarten. The only proviso is that you must love the ruler.
Place the ruler on the temperature plot and orient it along the data so that it most pleases your eye. Grab a pencil and draw a line along its edge. Then, if you can, erase all the original temperature points so that all you are left with is the line you drew.
If a reporter calls and asks if the temperature was warmer or colder last year, do not use the original data, which of course you cannot since you erased it, but use instead your line. According to that very objective line the temperature has obviously increased. Insist on the scientificity of that line—say that according to its sophisticated inner-methodology, the pronouncement must be that the temperature has gone up! Even though, in fact, it has gone down.
Don’t laugh yet, dear ones. That analogy is too close to the truth. The only twist is that statisticians don’t use a ruler to draw their lines—some use a hockey stick. Just kidding! (Now you can laugh.) Instead, they use the mathematical equivalent of rulers and other flexible lines.
Your ruler is a model Statisticians are taught—their entire training stresses—that data isn’t data until it is modeled. Those temperatures don’t attain significance until a model can be laid over the top of them. Further, it is our credo to, in the end, ignore the data and talk solely of the model and its properties. We love models!
All this would be OK, except for one fact that is always forgotten. For any set of data, there are always an infinite number of possible models. Which is the correct one? Which indeed!
Many of these models will say the temperature has gone down, just as others will say that it has gone up. The AP statisticians used models most familiar to them; like “moving averages of about 10 years” (moving average is the most used method of replacing actual data with a model in time series); or “trend” models, which are distinct cousins to rulers.
Since we are free to choose from an infinite bag, all of our models are suspect and should not be trusted until they have proven their worth by skillfully predicting data that has not yet been seen. None of the models in the AP study have done so. Even stronger, since they said temperatures were higher when they were in fact lower, they must predict higher temperatures in the coming years, a forecast which few are making.
We are too comfortable with this old way of doing things. We really can prove anything we want with careful choice of models.
Anybody interested in flying to Chicago on an airplane that has been modeled but not tested?
Thanks for the Penn and Teller treatment of your industry. It’s not too difficult a task to design results of such data by determining the start point of any data set. If we were to see a climate model based on a “day 1” temperature of the Earth, we’d realize that we are on a sustained cooling trend. In fact, at one time in the distant past when it was substantially hotter than today, conditions existed which impelled life from nothing (so we’re told). Now we are warned that moving toward such temperatures will destroy life. Ah, the mysteries of the human condition.
Your tale reminds me of the old story of the recent accounting school graduates interviewing for a job. When the Harvard and Yale grads responded to the question, “What is 2 + 2” they each answered 4. When the grad from the no-name college responded to the same question with his answer, “What do you need it to be”, he was awarded the job.
Clinton,
Part of our grief comes from when we look at time series like they were “random processes”, and where we reify randomness. It’s that reification that gives life to our models, and is why we love them so.
Say Matt, did you notice the terminology error that the AP made in it’s headline? I think they meant (in a more accurate use of statistical vocabulary) “Statisticians fail to reject warming”.
Not as catchy, but how do you reject a hypothesis that is within the 0% confidence interval?
All,
Apparently, my piece was copied over at Watts Up With That. There are many more comments and discussion there.
Andrew,
Don’t even start me on “null” hypotheses.
This is rather wonderful. Thanks.
Matt:
Let me repeat my comment from WUWT:
“My guess is that it is Borenstein that really needs to read this clearly stated explanation. But perhaps he already knew exactly what the statisticians would have to say given his poorly articulated charge.
“I do think it would be helpful to clarify that most good looking statisticians would be loathe to undertake such a task without knowing what the data represented, since the model chosen should have some relationship to the physical, social or behavioral processes represented by the data.”
Thoughts?
Bernie,
Amen, brother. Their response should have been, “I have no idea. Could be anything.”
Actually, some groups occasionally run contests where they issue hidden-source data. But they never ask “What model generated it?” They ask, “Predict the next few values.” Those that predict best win. That is the way to do it.
Bishop,
Thanks.
This reminds me of an exercise that the financial press pulled on the Wall Street’s “technical analysts.”
A technical analyst attempts to glean future price movements from historical prices. They take their rulers and protractors and draw trends and channels and pennants, and suggest that these formations arise from the emotions of the marginal investor. The theory of technical analysis flies in the face of the “efficient market hypothesisâ€, which lies at the foundation of “modern portfolio theory.†Hence, “serious investors†consider the chartist to be charlatans.
One enterprising reporter used a random number generator (i.e. a coin) to model fictitious price histories and mailed his charts to a multitude of analysts. Some came back with buy recommendations and some came back with sell recommendations, but all saw patterns in the data. The reporter had a good laugh at the expense of the chartists.
The good news here is that the statisticians were smart enough to not stick there necks out and tell this reporter that they saw a pattern if they didn’t have a strong reason to think that one should exist.
From WUWT
aylamp (12:53:08) :
“If your experiment needs statistics, you ought to have done a better experiment.â€
Ernest Rutherford
Was Rutherford a closet Bayesian?
I was struck by the incisive Feynmanesque quality of the first Rutherford quotation so I went looking and found another Rutherford gem:
“The only possible interpretation of any research whatever in the “social sciences” is: some do, some don’t.”
Guess I have to put him on my reading list.
Bernie,
Amen, brother Rutherford. He agrees that statistics is best used for predictions.
Doug M,
If I had a secret statistical system that would make me billions on the stock market, the last person I would tell is a reporter. Come to think of it. I wouldn’t tell anybody.
I’m reminded of a screaming headline in our local paper a few years back:
“Murder Rate in XXXXXX County Doubled Last Year, Statistics Show”
Well, digging into the story, the gentle reader finds that indeed the murder rate “doubled”. It went from 1 to 2 in that year. Statistically accurate, but practically meaningless.
That would have made a nice hockey stick.
If I had a secret statistical system that would make me billions on the stock market, the last person I would tell is a reporter. Come to think of it. I wouldn’t tell anybody.
Yes you would. It can’t be done without capital, and you won’t raise capital without press.
Nah, Doug M. I’m loaded.
I love this! But have you any suggestions on how to explain to a classroom of 10-11 year olds why the output of climate models is not to be taken as gospel? Children are being frightened by these predictions but a simple exercise involving your trusty ruler might just help relieve some nightmares. Any ideas appreciated, tested on real children or not. Susan
Why do you want to disabuse the children? It is fun to scare kids.
Learning do deal with the phantom fears of childhood prepares us for the scariness of “real life.”
Susan C:
I will defer to our host, but I would say if your kids can plot data points on a graph then plotting temperature and rainfall should do the trick. You will need a dataset that includes locations at different altitudes and in different relationships to the coast and the off-shore current. A simple model is misleading. The point being that if you do not consider altitude, proximity to the coast, etc., then what appears to be true becomes much more complex and nuanced.
I remember many years ago reading an expose by a muckraking “science reporter” in the New York Times about the harm caused by a certain pesticide in rural Alabama. He dutifully reported that it caused cancer in county X, and birth defects in neighboring county Y, based on elevated occurances shown in health records for the two counties.
Now, I have no idea what the real data showed, but it occurred to me, as a lowly grad student waiting in an airport for my next dose of US Air abuse while traveling for a job interview, that perhaps some data existed on cancer in county Y and birth defects in county X. That information, of course, was not available in the story.
I could only conclude that there was one of three possibilities.
1) The data did not exist (highly doubtful).
2) The “science reporter” had the data, but it didn’t fit his/her “angle.” (I hope not)
3) It did not even occur to the “science reporter” to ask him/herself the question (my choice).
Or another of my favorites:
“Cancer Risk Halved With Reduction of Compound X in Drinking Water” (yes, halved!)
Again digging, it refers to the fact that due to a new EPA regulation and the expenditure of millions, the water supply’s content of compound X was reduced from 2 parts per 3 million to 1 part in 3 million, the new “guideline” figure.
How that got interpreted to halving the cancer risk only can only be properly explained in Media-ville.
Matt,
I’ve been having a lot of fun using an online random walk simulator and plotting all the final values it produces after 1000-2000 iterations of 100 steps each. I find that I can get negative or positive trends purely by chance. The plots look an awful lot like like 1000-2000 temperature and precipitation “reconstructions” I’ve seen, especially when I smooth the data by moving average. The program is at http://www.math.uah.edu/stat/applets/RandomWalkExperiment.xhtml
What got me interested in this subject was a paper I came across online titled “Global Warming As a Manifestation of a Random Walk,” which was published in 1991 by A.J. Gordon of Flinders University of South Australia in the Journal of Climate (AMS). I’m sorry I didn’t save the link for the paper, but you can probably find it with Google Scholar.
I’d be very interested in your thoughts about random walk (a concept that reminds me of a drunk staggering from one lamppost or other support structure to another) as a possible explanation of the “trends” in “global average temperature” etc. that have made the world crazy.
Pete Petrakis
Pete.
Thanks. Yep, I know that paper. I had the chance to write about it a while back. Link.
Susan, Bernie, Doug M,
For 10 year olds? Other than telling them that adults occasionally go insane, I don’t know. What a strange thing to be taunting kids with, anyway. Frightening them with far-fetched apocalyptic scenarios seems cruel, especially since, even if the worst forecasts are true, they can do nothing to stop. Perhaps we can charge certain people with endangering the welfare of minors?
John M, Plas,
This kind of thing is far, far too common. Journals eat this kind of thing up: the least hint that something could be bad is good enough to start the fretting.
Thanks for the link, Matt. It’s excellent. Now I wonder: have any warmologists convincingly refuted the random walk hypothesis of “climate change”?
Pete
Pete,
In their favor, it is a proposition that is impossible to refute. Unless you can discover a set of equations of motion, together with an exact set of initial conditions (observations), that predict without error future states of the atmosphere, then there is always a chance the “random” walk hypothesis is true.
Anyway, random only means “unknown”. Something is causing the climate to take the state it has, so there really isn’t a real thing as a “random walk.” It is purely a mathematical abstraction. We just haven’t figured out all the physics behind our climate.
I disagree William that random means “unknown” .
In chaos theory there is definitely a great and quantified difference between random and “unknown” .
When you look at a time series that is the result of deterministic chaos , you simply have to use the qualification “pseudo random” .
If a statistician looks at it he will say that it is random but it is not unknown because you have in front of your eyes the differential equations whose solutions produced the series .
.
After discussions with D.Koutsoyiannis and an attempt of D.Hughes to apply Hurst analysis on series that are extracted from genuine deterministic chaos (known) I am still puzzled .
There is something unpleasant happening when statistical methods are applied to chaotic systems .
I used to think , perhaps a bit naively , that randomness was a limit when the number of dimensions of the phase space tended to infinity (equivalent to saying that a system is ergodic when it can be everywhere in the phase space) .
I am no more so sure about it .
Unfortunatly I can’t dedicate much time to it but I am sure that there are more things in the triangle chaos-ergodicity-statistics than can be dreamt of in your philosophy , Horatio .
Matt,
With regard to reporting of awful things caused by so-and-so, I have a basic question about the statistics of epidemiology, and apologize if this is a bit off topic. Also, perhaps you’ve addressed it before.
Let’s say a study is conducted to look at the effects of chemical agent X on a population group, with the intent to screen for 20 forms of cancer potentially caused by that agent. The study is designed with 95% confidence limits (yeah, I know, I may have hacked up that term, but please humor me), and the authors triumphantly say “aha, we can show within 95% confidence limits that X causes cancer of the pinkie toe nail”.
Was that a stacked deck or could it still be meaningful?
In other words, does the 95% apply independently to each individual type of cancer screened for, or is such a study likely to pick up at least one cancer with “95% confidence”?
Tom,
Let me convince you that “random” does mean “unknown.” A coin flip where the initial conditions are unknown is random in the sense that we do not know the result until after the flip. A coin flip where the initial conditions are known (and, of course, the equations of motion are known) is not random and we do know what the result will be conditional on that knowledge.
I have my fedora with me today. Can you answer the question “Matt is wearing his hat”? The outcome to you is unknown, therefore random. To me, I can see, so the outcome is not random.
And, of course, there’s the old chaotic logistic equation example. If you know the constant, you know the future. If you don’t know it, you can’t predict it; it’s unknown.
John M,
It’s all over the map. There are ways to “correct” those classical confidence intervals for multiple testing, and sometimes people use them, sometimes they don’t. The “file drawer” problem always lurks. They test for dozens of maladies and only report the one that looks suspicious.
I actually talk about this a little in today’s podcast.
Briggs, Susan, Bernie, Doug M,
Regarding the schools. My daughter had a pool party with a few of her middle school friends 2 summers ago. I was talking with them and asking if they had any ideas yet about what they would like to do when they were older.
One girl said she loved photography and wanted to be a nature photographer. Then she started crying. I was really concerned and asked her why. She said that she REALLY wanted to photograph polar bears, but one of her teachers had told her that the polar bears were all dying because the north pole had melted and they were being killed off by George Bush because he was greedy and didn’t want the world to recycle (or words to that effect). There would be no polar bears in a few years for her to photograph. She also said the Pacific islands were almost all under water and people were drowning and that she was told she had better “get her parents to recycle to save the polar bears and the people”. I tried to give her some facts, but she wouldn’t listen. I guess teachers have a power over kids that we don’t. I tried to tell her that being responsible, recycling and taking care of the environment was important, but scare tactics and outright fabrications were not the right way to get people to be responsible.
This isn’t the first nonsense that’s come home . . . . .
William I understand what you say .
But it doesn’t apply to deterministic chaos .
It is irrelevant whether you know the initial conditions and the equations (you actually do) so in theory you KNOW everything . There is nothing unknown .
Yet you can’t predict anything and if you look at the data , you are doomed to make the random diagnostic (which is wrong) .
.
Let me assure you that even if you know the constant of the logistic equation , let’s take
sqrt(29) , you have no idea whatsoever about the future .
Even purely mathematically you have no idea because sqrt(29) has an infinite number of decimals .
So as you can’t make the calculation with an infinite number of decimals , you are prisoner of the number you really take .
And here as this number has necessarily a finite number of decimals , you may take 100^100 of them and stilll be completely away from what the equation will do with 100^100 + 1 decimals etc up to infinity .
Perhaps if you were God and could do actual computations with an infinite number of decimals you could find the “true” trajectory obtained with sqrt(29) .
But as you are not (unfounded assumption from my part) , any merely human attempt you would do will finish infinitely far from that “true” trajectory 🙂
.
In physics it is even worse and one doesnt need infinite numbers .
D.Hughes sent me a paper of a guy who took a real chaotic system and computed how fast the trajectories will diverge when the difference in the initial conditions is ONLY (!) due to the Heisenberg uncertainty relation .
You’ll agree that you can’t define the initial conditions more accurately at least not in the Universe we are living in .
Well even with this smallest allowed difference , the trajectories diverge damn FAST !
So these systems are genuinely known in all details yet completely unpredictable .
Treating them as “unknown” (what we already know is wrong) and trying randomness (using some pdf) fails too but in a more strange way .
F.ex D.Koutsoyiannis is considering natural hydrological systems as being fundamentally stochastical obeying to more or less sophisticated power laws .
Well it kinda “works” untill … it stops working .
.
Continuing on my last comment in the previous post , one can think that randomness (in physics) could be a kind of limit of deterministic chaos for SOME systems .
Of course I am not considering here statistical mechanics where randomness is the child of isotropy and homogeneity (both expressing isoprobability of something) and that’s why we can meaningfully talk about temperature , energy averages and such .
On the other hand trying statistics on turbulence also kinda works untill … it stops working .
Tom,
We agree on all the examples, but just not on a slight point of definition.
Take your example. If all you know is the “label” sqrt(29), then you don’t know the value of the constant: you do know its label, and you can approximate it, but you can’t claim to know it because, as you observe, it requires an infinite number of digits to represent.
But if you do know the constant—let it be a = 1—then perfect prediction is guaranteed.
Recall, we are speaking about logic here, and not about the practical impossibility in real life of ever observing initial conditions. In practice, perfect predictions of chaotic system is not going to happen. And I make no claim that the logico-mathematical logistic equation represents anything in real life.
Plus, we have those other examples I gave earlier (like my hat). No complicated physics there.
William I agree with your examples , this of hat included .
.
There is indeed apparently a fine difference in what we both call call “knowing” .
I consider that I know sqrt(29) . It is a real number , accumulation point of a Cauchy suite .
Very well defined and unique . I cannot write its decimal form but I can do mathematical operations with it .
What I cannot do is to say what will happen when the constant (or rather initial condition) in the logistical equation is this number .
I prefer initial conditions because of a technicality – the logistical equation is stable for many constants while it is always unstable for any initial conditions when the constant is greater than a given number .
In physics it’s different
It is because you can’t know the initial conditions at better than h/2Pi .
So in a sense the question about knowing the initial conditions with infinite accuracy (what is required for a chaotic system) doesn’t make physical sense .
Of course from this “unknowability” doesn’t follow that the system is random . It isn’t .
Sigh. If you cherry pick the initial conditions you can produce any result you like:
“First, a fact. It is true that, based on the observed satellite data, average global temperatures since about 1998 have not continued the rough year-by-year increase that had been noticed in the decade or so before that date. The temperatures since about 1998 have increased in some years, but more often have they decreased. For example, last year was cooler than the year before last. These statements, barring unknown errors in the measurement of that data, are taken as true by everybody, even statisticians.”
In physical science terms that is a seriously incomplete analysis. Global average temperatures peaked in 1998, for well-understood physical reasons. Some temperature measurement series suggest they reached an even higher peak in 2005. Others don’t. Also, the “rough year-by-year increase” is very rough indeed, suggesting that the trend signal is hard to disentangle from the noise.
Take any long run temperature series (the climate science noise-supressing convention is 30 years) and do something really crude to get rid of the anomalous 1998 peak – say take the 1997 value and the 1999 value, add them together and divide by 2. Guess what, the “cooling trend” vanishes (I’m sure there’s a statistical technique that I havn’t heard of which stops it vanishing, but didn’t someone here suggest there’s an infinite number of models). That’s true of the five, six, seven, eight, nine, ten, and eleven year trends starting in 1998 too (data from 2009 isn’t in yet, so too soon to tell if it applies to the twelve year trend). In physical science, we are taught to be very wary of drawing conclusions from data that we know contains an outlier, and see what the analysis shows if you leave it out. Statisticians, on the other hand, as you so rightly point out, are taught to disregard the data and stick to the model.
Oh, and don’t worry about what kids hear at school. Lots of Americans don’t believe in evolution by natural selection, and the teenage pregnancy rate is the worst in the rich world. Neither would be true if teachers could get their students to understand some physical science.
Tom, thanks for recalling our discussions. I am glad to hear that you are “no more so sure” that “randomness was a limit when the number of dimensions of the phase space tended to infinity”. I agree with William that “random” means “unknown”–although I would prefer to use “unpredictable” or “uncertain” instead of “unknown”.
I hope you will find my paper “A random walk on water” (just published for discussion in Hydrology and Earth System Science Discussions/HESSD–was my talk a few months ago in EGU) very relevant to this discussion. I will appreciate any comment on the paper. Note, HESSD is an open access and public review journal, i.e. enables posting comments online which appear in association with the paper immediately (no censoring…).
If you read the paper, you will notice that I use a 2D system–apparently low dimensional deterministic–and explain why it is (at the same time) random (aka unpredictable) and when it is wiser to neglect its deterministic dynamics in producing future predictions. Also, I am discussing the issue of the long-term (un)predictability (i.e. at climatic time scales).
Missed to include links to the paper:
http://www.hydrol-earth-syst-sci-discuss.net/6/6611/2009/hessd-6-6611-2009.html
http://www.itia.ntua.gr/en/docinfo/923/
Great paper Demetris .
I have just overflown it and will take now more time to read it carefully .
It’s indeed in the middle of themes that I am interested in .
The specific point that interests me is ergodicity in chaotic systems .
Indeed ergodicity is necessary to obtain a (meaningful) probability density function in the phase space .
However it is not known if all chaotic systems (deterministic) are ergodic and they probably are not .
It is suggested that very high dimensionnal may be – hence my idea that stochastical model (e.g randomness) could be useful in the high dimension limit .
This doesn’t seem to be the case for low dimensional deterministic chaos (as shown by D.Hughes by trying Hurst analysis on the 3 and 5 dimensional Lorenz system) .
But more after I have read your paper .
Demetris,
Thanks for the paper! I look forward to reading them.
Plas,
Yes, exactly. I am scheduled to give a few talks over the next couple of months to school kids about the amazing arctic adaptations of polar bears, to counter-act some of these scare stories. Polar bears have ways of taking care of themselves that kids probably have not heard about. In addition, bears are doing just fine right now, they had a very good year. I am thinking ahead as to how I might respond to the kinds of comments you heard from your daughter’s friends. I’m thinking perhaps describing a climate model as a grown-up version of a crystal ball might do the trick. I might even see if I can find myself a crystal ball to take with me.
Matt – if I can work a ruler into the story, I will. A ruler IN the crystal ball??
NickS. For some reason, the climate people love to fit straight lines to everything. I look at the data and see a step function, essentially flat prior to 1998 with lots of wiggles due to ENSO (each peak is an El Nino). This is followed by a second step 0.3C higher post 1998 with far fewer ENSO events. This is exactly the kind of behavior one sees in nonlinear systems.
Demetris
Your paper uses a specific 2 dimensional map in chaotic regime .
This corresponds to the family of hamiltonian systems (energy conserving) where the systems live on isoenergy surfaces .
This corresponds generally to the behaviour of hard balls , billiards or many bodies systems .
Indeed it has been proven that the position of the Earth on its orbit was chaotic and unpredictable . Of course one should not confuse the POSITION on an orbit with the orbit itself .
While one doesn’t know where the Earth will be on the orbit , one is pretty sure that the orbit will still be the same and stay stable for a long time .
The “randomness” in such systems is always linked to the measure of the unpredictability or to the decay of correlations what is the same thing .
This question is even at the center of debates lead in the ergodic theory since a rather long time .
.
One knows that the unpredictability (or “randomness”) is not an on and off feature in a system .
It is rather a gradual evolution going in the ergodic hierarchy from merely ergodic systems at the bottom to Bernouilly systems at the top .
Bernouilly systems are formally equivalent to a roulette and destroy so completely every correlation that only “randomness” stays .
Now what is very important to notice is that the weakest condition to find a pdf in a chaotic system is ergodicity .
And ergodicity can never be assumed – it must be proven .
So for example it is proven that multiple harmonic oscilators are NOT ergodic (KAM theorem) .
On the other extreme is the Baker’s transformation (map) which represents a Bernouilly system and so is not only ergodic but behaves like a roulette wheel .
The hard ball system being a Kolmogorov system (more random than only ergodic but less than Bernouilly) explains why the statistical mechanics work so well .
.
Follows that both the dose of randomness and the ability to describe it by a stochastical method (finding a pdf) vitally depends on the map you choose .
The minimum being a measure preserving ergodic map .
You could probably prove that the map you choose (Eq 2) is ergodic (I suspect that it is even more – strongly mixing) .
But it would be an error to generalise and say that ALL maps (e.g all chaotic deterministic systems) can find a statistical interpretation because non ergodic systems can’t .
Going in some finer details it can be speculated that even non ergodic systems may admit some subsets of the phase space where ergodicity i still true – kind of restrictive stochastical interpretation valid only for certain states of the system .
For a comprehensive discussion of the randomness interpretation and ergodic hierarchy see f.ex Berkovitz , Frigg , Kronz (2006) (http://www.romanfrigg.org/writings/Ergodic_Hierarchy_SHPMP.pdf)
.
Last it is very necessary to stress that all these concepts and results apply ONLY to local time dependent , generally Hamiltonian systems .
When one talks about climate , hydrology or more generally about diffusive systems the situation changes completely .
Here space matters and instead of the abstract temporal chaotique structures only “visible” in curves in the phase space on gets SPATIO-temporal chaos where the chaotic structures are visible in the real space (clouds , waves , vortexes) .
We don’t deal with (coupled) ODE anymore but with PDE .
These systems are already non Hamiltonian to begin with because they are dissipative .
The notion of ergodicity is not easy to transport here from the merely temporal case .
The problem of finding and describing invariant measures on attractors of PDE is basically open .
Some progress is being done with toy models of coupled map lattices .
Here one takes N only temporally chaotic maps (f.ex the logistical equation) and couples them .
This coupling of N maps simulates the spatial extension .
It is fair to say that problems of dynamical properties (ergodicity , mixing etc) is far , VERY far to be solved .
The little that is known shows what the common sense would tell us too – the spatio temporal chaos (and the question of “randomness” in it) is different and much more complicated than the “simple” temporal chaos in hamiltonian systems of the statistical mechanics .
Dear Tom,
Thanks for the thoughtful comments, which go beyond the scope of my article. I have tried to keep the article as simple and self contained as possible and not to use terms which I do not define or explain. Thanks also for the link to an interesting paper.
I was curious to read in your comment “one is pretty sure that the orbit [of the Earth] will still be the same and stay stable for a long time”. I must tell you, I can live on an Earth in which at least the position is not stable (I understand you agreed on that) and perhaps I could live even if the orbit was somewhat unstable, despite that “one is pretty sure” about the opposite. I must confess that I am fan of randomness, aka unpredictability, and perhaps I would have difficulties to adapt myself in a world that would be predictable (and thus fully controllable). Fortunately for me, this is not the case. It strikes me that people think that instability, unpredictability, randomness, uncertainty, and eventually change, are negative qualities which we have to fight to make things stable and predictable. As I write in the paper, only dead systems are predictable. The causes of change are the same causes of unpredictability.
I do not say that “ALL maps (e.g. all chaotic deterministic systems) can find a statistical interpretation [including] non ergodic systems …”. On the other hand I do say that uncertainty should be studied in PROBABILISTIC terms. Please note that that the replacement of the deterministic dynamics (1) and (2) in the paper with the stochastic dynamics (3) or (4) does not demand the system to be ergodic. And I hope you will agree that the stochastic formulation, which gives the evolution of probability densities (the stream-tube representation) rather than “exact” but inaccurate trajectories (the thread representation), is more powerful. Ergodicity is needed for other cases, particularly if we try to replace knowledge of dynamics (or bypass the absence of this knowledge) with information contained in the data.
I find you too optimistic when you say “ergodicity can never be assumed — it must be proven”. Proving is a very heavy duty when speaking of real world. We should not confuse the real world with our toy models, for which (the simplest of them) sometimes we can provide proofs. Even in the abstract world of mathematics, we now know (thanks to Goedel, Turing, Chaitin and others) that not everything can be provable. So we should exploit alternative ways of inference. We could lower our ambitions from proving (deducing) to making induction. And induction’s theory is the theory of probability and statistics, or more generally stochastics. That is my interpretation of Maxwell’s quotation I include in the paper, “the true logic for this world is the calculus of Probabilities” as well as the title of Jaynes’s book “Probability Theory: The Logic of Science”, whose reading I strongly recommend.
Demetris
Demetris
In order not to abuse William’s blog I registered on Copernicus via your link in order to continue this discussion .
Unfortunately despite registering I can’t log in because I am caught in some infinite loop where everytime I log in I am redirected to the administrator page which asks me again to … log in .
.
To avoid misunderstandings , I am not a “fan” of descartian determinism (which I consider dead for at least 1 century anyway) .
If I have been interested for a long time in quantum mechanics and chaos theory it is precisely because they are both uncertain AND deterministic .
So I do not consider uncertainty as a negative quality but on the contrary the very foundation of our Universe .
Isn’t it that one says that the Universe itself is born from a “quantum fluctuation of the void” ?
.
To be fair it is not only dead systems that are predictable but it is true that 95% of the 19th century physics deals with equilibrium systems which are indeed “dead” .
I think that it is not correct when you say that “Please note that that the replacement of the deterministic dynamics (1) and (2) in the paper with the stochastic dynamics (3) or (4) does not demand the system to be ergodic. ” .
First reason is that your system IS ergodic or more so there is circularity .
Second reason is that switching from the temporal dynamical representation to the statistical representation , you switch to the phase space and use terms like
[integral over S^-1 (A) of PDF(u) du .]
Doing that you implicitely assume that a PDF is defined and integrable over any S^-1 (A)
Now non ergodic systems mean that there are “holes” and “concentrations” in the phase space so you can’t know what the As are that admit a S^-1(A) .
Even worse would be fractal attractors .
In this case the boundary (in the phase space) between P=0 (system can’t be there) and P>0 (system can be there) is not computable .
The very possibility to have the “stream tube” representation depends on the fact that the trajectories fill the whole tube and there are no holes or places where the system gets trapped .
This is another way to say that the system is ergodic and the tube of your example is well behaved because , precisely , the map you choose is ergodic or better .
.
As for the quote that “ergodicity may never be assumed” , I forgot who said it .
The ergodic theory is actually mathematics (theory of measurable sets) .
So the the ergodic property is a mathematical well defined statement about measure preserving transformations operating on measurable sets .
That’s why it can and must be proven .
The real world can then be more or less well represented by measurable sets and behave more or less ergodically – that’s why ergodic theory often uses idealised models like hard spheres , geodesic flows or billiards .
Actually the real world behaves remarkably in agreement with the theory as the success of statistical mechanics and thermodynamics shows .
In any case one thing seems very sure to me , if the equations describing a given system and validated by experience lead to the (mathematical) proof that the system is NOT ergodic then any model or description that would need ergodicity will be wrong .
This is what is meant when is said “ergodicity can never be assumed” .
.
The KAM theorem is an excellent example proving (mathematically) non ergodicity in Hamiltonian systems what automatically eliminates any model/theory that would have assumed it .
Tom, Demetris,
Have no worries. I welcome this discussion and enjoy it very much. It’s perfectly appropriate.
Thanks William . I feared that debates between Demetris and me that generally turn quite technical might be out of what you wish .
.
Demetris
I forgot to add one thing .
While I don’t agree with SOME statements you made in your paper , I fully agree with the philosophy of it .
To begin with I have answered no on all your questions in the end of the paper like you , so that shows that we are obviously on the same wavelength .
I have only a reservation about the number 3 which is connected to the meaning I give to “physical explanation” .
I also understand what you wanted to do – show that uncertainty and deterministic law are not in contradiction and that in many (most) natural systems uncertainty can’t be eliminated .
And that in some cases the statistical description is not only possible but complementary .
The problem being that your toy model is so particular (ergodic sure and strongly mixing probable) that it reduces much its pedagogical reach .
William, thank you for your hospitality and kindness. I will try not to abuse them being overly technical.
Tom I am sorry that you had difficulties with the EGU/HESSD system. In the first occasion (when replying to a comment submitted to the journal) I will try to cite this discussion here and insert a link to it, so that interested HESSD readers could see it. It is really very interesting to me.
Also, I am sorry if I misinterpreted some of your initial statements. It is nice that we have some disagreements and it is great that we are converging, particularly in the philosophical implications. I find it amazing, especially given our very different origins, yours as a quantum theorist and dynamical systems physicist and mine as a civil/hydrological engineer. Apparently, the difference of origin creates some difference in views. For example I do not care too much to check whether my toy model dynamics is “integrable over any S^-1 (A)”. In the paper I do not need such check, because the Monte Carlo method I use “bypasses” the calculation of these integrals. With this I do not mean to devalue the importance of such checks. Rather, I admit that it is not my strong point and I trust that other scientists can do them, if it is worth and necessary doing. I even allow myself to make an error on this, trusting that someone else will correct it. But for the time being I do not see any.
I even do not worry too much for the case you mention where “any model or description that would need ergodicity will be wrong”. I am familiar with wrongness: in the complex real-world systems I am studying, every model is wrong in absolute/precise terms. I am concerned about the degree of approximation of reality by wrong models, which again brings me to a notion of a measure of confidence.
I did not understand why you find my toy model “so particular”. You say it is ergodic and earlier you said “The real world … behave more or less ergodically”. So what is the “pedagogical” problem? I see the “pedagogical” attitude in a different way. I wanted the toy model to be implementable in a few minutes in a spreadsheet–not to need software packages for solving partial differential equations to use it. I also wanted to produce antipersistent and persistent behaviours, to demonstrate their properties and their importance, as well as their distinction from, respectively, periodic and nonstationary processes. I think all these are well demonstrated with this very simple toy, which satisfies me.
I am indeed grateful for reading the paper and for your comments and discussions.
Sure Demetris you reach the goals you have set up for yourself .
I was actually mostly focussed on generalisations .
Basically what I say is that statistical models of chaotic systems may work more or less depending on what stage of the ergodic hierarchy the system finds itself in (that’s why I linked the paper) .
The risk being that an uncareful reader could generalise and conclude that it is ALWAYS possible .
All my remarks were leading to 2 caveats :
1) The toy model is particular , namely at least ergodic . You probably didn’t do so on purpose but it happens to be so . This has of course for consequence that the toy model would indeed pass the tests and for instance be integrable over all S^-1(A) . That’s why Monte Carlo works too . But if your model happened not to be ergodic or created fractal attractors then you would see that there are many problems with statistical interpretation and for instance the integrals in Eq 3 couldn’t be calculated .
This is a mathematical caveat due to the fact that the toy system doesn’t represent all chaotic systems , it is particular .
2) The second is a physical caveat . Most physical systems of interest which present a chaotic behaviour are extended system by opposition to local systems only varying with time .
The properties of purely temporal chaotic systems like the one you study do not transport to the spatio-temporal domain . Or rather very little is known about spatio-temporal chaos .
I believe (and it is really just that , a belief) that the spatio-temporal chaos theory will eventually achieve a developpement where equivalents of ergodicity , attractors and orbit stability would be defined . Science that made big progresses in temporal chaos since Lorez 40 years ago , is just beginning to look at spatio-temporal extended systems .
Actually I have often noticed that there are only few scientists who have insights in temporal chaos (you are one of them) and many don’t even do the difference between spatio-temporal chaos and temporal chaos .
Climate science is an astounding example where spatio-temporal chaos (weather) is transformed in a purely temporal non extended system by just … space averaging !
The (in)famous global mean temperature is an example how an extended temperature field is transformed in a single variable by just space averaging which is then studied like if it represented a 1 dimensional purely temporal system .
As for how this can be justified it is anybody’s guess .
I believe that it is because many climate scientists still live in the ancient equilibrium paradigm and didn’t yet fully integrated the fact that the system we are dealing with is a non equilibrium dynamical system that has nothing to do with black bodies in radiative equilibriums . What you do is important because you take non equilibrium dynamics seriously .
But enough of that , it has lead me too far from what I wanted to say about the caveat which is that even if something works in the temporal domain , it is not sure that it will work in the spatio-temporal domain and actually mostly will not .
I forgot .
Believe me , a model that would assume ergodicity there where is none would be UTTERLY wrong . Ergodicity is not a property that can be “approximated” , it is really as fundamental for dynamical systems as the conservation of energy .
The difference in behaviour is really big (if you have time look at the KAM theorem to see what non ergodic systems do) .
However you are safe from that in your paper because your map can surely be proven as being at least ergodic . If it was not you would have noticed yourself that there is something very unpleasant going on 🙂
Is the majority of natural local purely temporal chaotic systems ergodic ?
I do not know but from the fact that most (known) attractors are not fractal or KAM like , I would say yes . But majority is not all and one wouldn’t know before one checks .
Thanks Tom for ascertaining that I reached the goals I have set up.
I see your point about the importance of mathematical generalization, but it was my intention the toy model to be ergodic, and thus representative of natural behaviours. I did not have an ambition for mathematical generalization, but I wished to demonstrate, using simple means, a few simple things that are widely misrepresented (in my opinion) in geophysical sciences including hydrology.
And you are right, if the model was utterly wrong, perhaps I would have noticed it. I have some basic knowledge of probability, which, according to Laplace’s quotation I mention in the paper, is “nothing but common sense reduced to calculus”–and I think common sense is a good adviser to notice utterly wrong situations.
OK then we agree Demetris .
Only to remove the last uncertainty after our small discussion .
.
When you write “but it was my intention the toy model to be ergodic, and thus representative of natural behaviours.” you are aware that the converse is not true , right ?
Namely that there are natural behaviours which are NOT ergodic .
So if somebody picked up a toy model representing a natural NON ergodic system (that’s what the KAM thorem does) , he would arrive at the opposite demonstration than you did .
Namely that a statistical representation of this system doesn’t work well and that equations like your equation 3 can’t be solved .
Of course he would still answer no or no with reservations on the questions at the end of your paper .
But he would have demonstrated that a statistical representation of this specific natural system can’t be done in any useful way .
.
So the question about usefulness is really a question about how representative ergodicity is in natural systems .
Nobody really knows .
It seems to be rather common in systems that look like a great number of hard spheres or like geodesic flows and depend only on time and it doesn’t seem to be common if it exists at all in extended spatio-temporal systems .
Want an example ?
Put a heavy colored fluid on top of a lighter one (Rayleigh flow) with both fluids not mixing and observe what happens .
This is an extended spatio-temporal chaotic system .
You will see mushroomlike branching structures coming down in some places and not in others .
The phase space is full of holes and you can neither describe statistically the structures when they are formed nor give probabilities where and when they will form .
The cigarette smoke structure is a similar phenomenon but in the other direction from down to up .
To be complete in this last case after a sufficient distance when the smoke becomes fully mixed with air and invisible , the classical statistical thermodynamics of course applies (=many hard spheres) .
Tom, “representative” does not mean it covers everything. It is good for the rule — and there may be exceptions.
It is not good –and perhaps not possible– to remove all uncertainties. So if you think you removed the last uncertainty, I can put another one: Perhaps there is a way to assign probabilities in the cases you mention and you think it is impossible. But this is for another paper/another discussion…
Demetris
.
“Perhaps there is a way to assign probabilities in the cases you mention and you think it is impossible. But this is for another paper/another discussion…”
.
The answer is no and I will sketch the proof . This part of chaotic systems has already been fairly studied .
We have some iterative non linear map F defined on I -> I such as Xn+1 = F(Xn)
We define the marginal PDF P(X0) for every X0 € I .
P(X0) converges to a unique P for all X0 € I if and only if F is ergodic . P is in this case called the invariant , natural measure of F . P is furthermore stationary .
This means concretely that if you make 1000 iterations of F for different initial values , every value of F(X) will be visited with the same frequency and P is an invariant property of F .
F.ex the logistical map is ergodic and has a unique invariant measure
P(x) = 1 / Pi .Sqrt [x.(1-x)]
.
Consequence .
If F is not ergodic , there is no unique invariant measure of F . The marginal PDF depend on X0 and don’t converge . It is therefore impossible to choose a PDF which would be a property of F and the frequency with which F(X) visits some neighbourhood in I is different for every X0 .
A stochastical description of F is impossible .
This translates physically in the fact that the phase space is partitionned in N regions (N may be infinity) where the dynamics of the system is different for every N .
An analogy would be a phase change – one region is solid , one region is liquid and one region is gas .
Tom, I will insist that this discussion is for another occasion and that I wish to leave some uncertainty for myself, even if you want to be yourself concretely sure. Please notice that I spoke of the possibility of assigning probabilities (or even necessity of assigning probabilities; cf. what I call premise of incomplete precision in the paper) and I did not speak of convergence, invariance and stationarity.
Ok Demetris . So untill next time .
However please notice that I have proven that the possibility to do as you wish IMPLIES necessarily ergodicity . You indeed didn’t speak of convergence , invariance and stationarity but as shown in the post above , those are equivalent to ergodicity .
I basically wanted to spare you time if you eventually wanted to expand your paper beyond ergodic systems .
This issue has been settled a long time ago (since Kolmogorov) and the non stochastical behaviour of non ergodic natural systems has been already demonstrated .