Comments on: It depends what the meaning of mean means. http://wmbriggs.com/blog/2008/03/10/it-depends-what-the-meaning-of-mean-means/ "All manner of statistical analyses cheerfully undertaken." Wed, 20 Aug 2008 15:47:42 +0000 http://wordpress.org/?v=2.3.3 By: Chuck Peterson http://wmbriggs.com/blog/2008/03/10/it-depends-what-the-meaning-of-mean-means/#comment-1883 Chuck Peterson Mon, 31 Mar 2008 17:28:09 +0000 http://wmbriggs.com/blog/2008/03/10/it-depends-what-the-meaning-of-mean-means/#comment-1883 Checking temperature every hour. Perfect 100% reliable instruments in a pristine location with no biases. Day 1. 2 AM low of the day at 40. 4 PM high of the day at 80. Mean of 60. Every other hour of the day 52. Day 2. 3 AM low of the day at 50. 5 PM high of the day at 70. Mean of 60. Every other hour of the day 62. Can I even really compare these two numbers at a single location between the two days? No. Does the daily number show me anything? No. Do they even tell me anything about the conditions? No. Much less combining them all into months, comparing the months with some 30 year mean of similar measurements for that month, coming up with offsets, and then combining every location with many other locations. Checking temperature every hour. Perfect 100% reliable instruments in a pristine location with no biases.

Day 1. 2 AM low of the day at 40. 4 PM high of the day at 80. Mean of 60. Every other hour of the day 52.

Day 2. 3 AM low of the day at 50. 5 PM high of the day at 70. Mean of 60. Every other hour of the day 62.

Can I even really compare these two numbers at a single location between the two days? No. Does the daily number show me anything? No. Do they even tell me anything about the conditions? No.

Much less combining them all into months, comparing the months with some 30 year mean of similar measurements for that month, coming up with offsets, and then combining every location with many other locations.

]]> By: William M. Briggs, Statistician » Homework #1: Answer part II http://wmbriggs.com/blog/2008/03/10/it-depends-what-the-meaning-of-mean-means/#comment-1506 William M. Briggs, Statistician » Homework #1: Answer part II Fri, 21 Mar 2008 12:15:25 +0000 http://wmbriggs.com/blog/2008/03/10/it-depends-what-the-meaning-of-mean-means/#comment-1506 [...] Well, almost done. Good statisticians will give you some measure of uncertainty of the guess of θ, some plus or minus interval. (If you haven’t already, go back and read the post “It depends on what the meaning of mean means.”) The classical estimate used for θ is just the computed mean, the average of the past data. So the plus and minus interval will only be for the guess of the mean. In other words, just as it was in regression models, it will be too narrow and people will be overconfident when predicting new data. [...] […] Well, almost done. Good statisticians will give you some measure of uncertainty of the guess of θ, some plus or minus interval. (If you haven’t already, go back and read the post “It depends on what the meaning of mean means.”) The classical estimate used for θ is just the computed mean, the average of the past data. So the plus and minus interval will only be for the guess of the mean. In other words, just as it was in regression models, it will be too narrow and people will be overconfident when predicting new data. […]

]]> By: lucia http://wmbriggs.com/blog/2008/03/10/it-depends-what-the-meaning-of-mean-means/#comment-1307 lucia Fri, 14 Mar 2008 19:45:15 +0000 http://wmbriggs.com/blog/2008/03/10/it-depends-what-the-meaning-of-mean-means/#comment-1307 Lindzen's uncertainties look like the 95% confidence intervals based on measurement uncertainties for individual data points (using GISS or Hadley estimates of the measurement uncertainty.) That would mean they should be for the observed value. But he didn't say in that brief post. Unfortunately true the arguments about statistical significance in Climate Blog Wars seem to be un-connected to any stated hypothesis. Also, many seem stuck on testing the hypothesis trend?0 C/century, as if no other hypothesis can be tested. Can you suggest specific papers that compare GCM predictions to independent data? Generally, when people point me to papers they contain pretty pictures and the comparison tend to be "See. Don't they look similar"? It would be nice to see something quantitative if that exists. Lindzen’s uncertainties look like the 95% confidence intervals based on measurement uncertainties for individual data points (using GISS or Hadley estimates of the measurement uncertainty.) That would mean they should be for the observed value. But he didn’t say in that brief post.

Unfortunately true the arguments about statistical significance in Climate Blog Wars seem to be un-connected to any stated hypothesis. Also, many seem stuck on testing the hypothesis trend?0 C/century, as if no other hypothesis can be tested.

Can you suggest specific papers that compare GCM predictions to independent data? Generally, when people point me to papers they contain pretty pictures and the comparison tend to be “See. Don’t they look similar”? It would be nice to see something quantitative if that exists.

]]> By: Briggs http://wmbriggs.com/blog/2008/03/10/it-depends-what-the-meaning-of-mean-means/#comment-1254 Briggs Thu, 13 Mar 2008 09:55:06 +0000 http://wmbriggs.com/blog/2008/03/10/it-depends-what-the-meaning-of-mean-means/#comment-1254 Bruce, There is no reason why climate models cannot provide skillful forecasts. In fact, some climate models do. The Climate Prediction Center (or did they just change their name again?) routinely produces skillful predictions. However, these are on the order of one to six months into the future and the skill is very modest. Point is, there is no theoretical bar preventing skillful forecasts. So far, though, the big GCMs have not done well predicting independent data. Bruce,

There is no reason why climate models cannot provide skillful forecasts. In fact, some climate models do. The Climate Prediction Center (or did they just change their name again?) routinely produces skillful predictions. However, these are on the order of one to six months into the future and the skill is very modest.

Point is, there is no theoretical bar preventing skillful forecasts. So far, though, the big GCMs have not done well predicting independent data.

]]> By: Briggs http://wmbriggs.com/blog/2008/03/10/it-depends-what-the-meaning-of-mean-means/#comment-1253 Briggs Thu, 13 Mar 2008 09:52:25 +0000 http://wmbriggs.com/blog/2008/03/10/it-depends-what-the-meaning-of-mean-means/#comment-1253 JM, Had a look at Lindzen's pic. The term "statistical significance" is one I would like to see forever banned, but that's a story for another time. First, you cannot have SS without reference to some model or some formalized hypothesis about the data. Second, the SS makes some probability statement about some function of the data given some belief about a model's parameters. Example: in the linear regression I used in this post, the hypothesis is usually <code>b = 0 </code>. The probability statement is "What is the probability this odd function of the data would be larger than what we got if we repeated out experiment an infinite number of times <em>given</em> our hypothesis is certainly true?" That's what a p-value is. Confused? You should be. I'll try to make it simpler: we need a model and a probability statement about that model. I don't see either in Lindzen's pic. What model did he use? Our model B starting at 1993? Why pick 1993 and not 1992 or 1994? You have to take into account the uncertainty in picking the start date, which I very much doubt he has done. Look at his fuzzy error bars, which I always say should be on plots of this kind. Are they for the expected value (the second definition of "mean") or are they for the observed data? Looks like the former to me, but I'm guessing. Best you can probably say is "The data do not appear completely consistent with the hypothesis of a strict linear increase if we assume the linear increase started in 1994." But who knows? JM,

Had a look at Lindzen’s pic. The term “statistical significance” is one I would like to see forever banned, but that’s a story for another time.

First, you cannot have SS without reference to some model or some formalized hypothesis about the data. Second, the SS makes some probability statement about some function of the data given some belief about a model’s parameters.

Example: in the linear regression I used in this post, the hypothesis is usually b = 0 . The probability statement is “What is the probability this odd function of the data would be larger than what we got if we repeated out experiment an infinite number of times given our hypothesis is certainly true?” That’s what a p-value is.

Confused? You should be. I’ll try to make it simpler: we need a model and a probability statement about that model. I don’t see either in Lindzen’s pic. What model did he use? Our model B starting at 1993? Why pick 1993 and not 1992 or 1994? You have to take into account the uncertainty in picking the start date, which I very much doubt he has done.

Look at his fuzzy error bars, which I always say should be on plots of this kind. Are they for the expected value (the second definition of “mean”) or are they for the observed data? Looks like the former to me, but I’m guessing.

Best you can probably say is “The data do not appear completely consistent with the hypothesis of a strict linear increase if we assume the linear increase started in 1994.” But who knows?

]]> By: Bruce Foutch http://wmbriggs.com/blog/2008/03/10/it-depends-what-the-meaning-of-mean-means/#comment-1245 Bruce Foutch Thu, 13 Mar 2008 00:46:47 +0000 http://wmbriggs.com/blog/2008/03/10/it-depends-what-the-meaning-of-mean-means/#comment-1245 sorry to misspell your name Dr. Briggs. Too fast on the submit button. sorry to misspell your name Dr. Briggs. Too fast on the submit button.

]]> By: Bruce Foutch http://wmbriggs.com/blog/2008/03/10/it-depends-what-the-meaning-of-mean-means/#comment-1244 Bruce Foutch Thu, 13 Mar 2008 00:44:34 +0000 http://wmbriggs.com/blog/2008/03/10/it-depends-what-the-meaning-of-mean-means/#comment-1244 Dr. Biggs, I'm another lurker. Really enjoy your blog. Wish I understood even half of it ;) JM's comment brought to mind a related question. Have you, or will you offer an essay on the capability (or not) of statistical models, in particular GCMs, to be used to forecast future events? I did read your essay "Two differences in perception between global cooling and global warming", but hope for something more specific about why these models can or can not be predictive. Thank you Dr. Biggs,
I’m another lurker. Really enjoy your blog. Wish I understood even half of it ;)

JM’s comment brought to mind a related question. Have you, or will you offer an essay on the capability (or not) of statistical models, in particular GCMs, to be used to forecast future events?

I did read your essay “Two differences in perception between global cooling and global warming”, but hope for something more specific about why these models can or can not be predictive.

Thank you

]]> By: JM http://wmbriggs.com/blog/2008/03/10/it-depends-what-the-meaning-of-mean-means/#comment-1243 JM Wed, 12 Mar 2008 23:50:23 +0000 http://wmbriggs.com/blog/2008/03/10/it-depends-what-the-meaning-of-mean-means/#comment-1243 I'm normally a lurker. Adiscussion at Anthony Watts weblog has become quite heated concerning Richard Lindzen's comment concerning no significant warming since 1997. "A note from Richard Lindzen on statistically significant warming" http://wattsupwiththat.wordpress.com/2008/03/11/a-note-from-richard-lindzen-on-statistically-significant-warming/ Dr. Briggs, have you ever addressed this issue? I'd be interested in your analysis. Thanks. I’m normally a lurker. Adiscussion at Anthony Watts weblog has become quite heated concerning Richard Lindzen’s comment concerning no significant warming since 1997.

“A note from Richard Lindzen on statistically significant warming”
http://wattsupwiththat.wordpress.com/2008/03/11/a-note-from-richard-lindzen-on-statistically-significant-warming/

Dr. Briggs, have you ever addressed this issue? I’d be interested in your analysis.

Thanks.

]]> By: Briggs http://wmbriggs.com/blog/2008/03/10/it-depends-what-the-meaning-of-mean-means/#comment-1201 Briggs Tue, 11 Mar 2008 12:42:19 +0000 http://wmbriggs.com/blog/2008/03/10/it-depends-what-the-meaning-of-mean-means/#comment-1201 Thanks, Dave. I might have to steal that. It's actually fairly interesting. If you want to get technical, it the same as saying that sentence A is true where sentence A is "The probability of event B, where B is contingent, is less than 1 but greater than 0." You know A but you never know B. Well, if that wasn't boring, I don't know what is. Thanks, Dave. I might have to steal that. It’s actually fairly interesting. If you want to get technical, it the same as saying that sentence A is true where sentence A is “The probability of event B, where B is contingent, is less than 1 but greater than 0.” You know A but you never know B.

Well, if that wasn’t boring, I don’t know what is.

]]> By: dave http://wmbriggs.com/blog/2008/03/10/it-depends-what-the-meaning-of-mean-means/#comment-1200 dave Tue, 11 Mar 2008 12:10:06 +0000 http://wmbriggs.com/blog/2008/03/10/it-depends-what-the-meaning-of-mean-means/#comment-1200 The title of this piece reminds me of a friend's tongue-in-cheek saying: "You never know..... you know?" The title of this piece reminds me of a friend’s tongue-in-cheek saying: “You never know….. you know?”

]]>