William M. Briggs

Statistician to the Stars!

Page 4 of 567

Calculating Markov Chain Stationary Distributions Is Immoral?

From the Census Bureau's  "Content Reinterview Survey: Accuracy of Data for Selected Population and Housing Characteristics as Measured by Reinterview"

From the Census Bureau’s “Content Reinterview Survey: Accuracy of Data for Selected Population and Housing Characteristics as Measured by Reinterview”

No, I don’t think so, but the Census Bureau thought (thinks?) as much.

What follows is one of the more curious emails I’ve received, describing the experiences of Juan (not his real name), who used to work at the Census. Perhaps his story proves what readers have suspected: the more time spent with statistics, the looser your grasp of reality.

I’d really like your help with this one. I’m not sure how to answer.

…I was working at the U.S. Census Bureau doing quality control on the Current Population Survey. The primary way that we checked data quality was by performing re-interviews. We would ask the same set of questions, for the same time period, from a sub-sample of the households in our monthly survey.

One day I got the bright idea that the re-interview data I had looked a lot like a Markov chain. There was possibility that a different answer was given in the re-interview than there was in the interview. Questions that had a high frequency of answers changing were considered unreliable. I had a matrix for each question showing the frequency/probability of moving from one state (answer) to another. This looked just like the transition probability matrix that I had been taught about in my first stochastics class. I remember a problem where we had to predict tomorrow’s weather based on today’s. This was exactly the same and I went about taking a limit and calculating the stationary distributions for several of the re-interview questions.

My branch chief had me run the idea by the chief statistician at the Census Bureau and his reaction was not what I was expecting. He said that calculating the stationary distribution was simulating an immoral experiment! His thought process, as best I can remember it, was that taking the limit of that matrix was simulating re-interviewing our sample households an infinite number of times which was immoral.

A couple of years later I asked a friend, who holds a PhD in Biostatistics from Harvard, about this and she agreed with the chief statistician. This seems to me like they are taking the abstract and trying to make it real which is a huge stretch for me. Is the Bayesian interpretation of this approach different? Would Bayesians have moral qualms about calculations the stationary distribution in such a situation?

I followed up with Juan and he gave me more details confirming the story. An example of how a question on race (which was mutable) heads this post. Terrific evidence that most survey data should not be taken at face value.

Markov chains are the fancy names given to certain probabilities. Juan used weather as an example. Suppose the chance of a wet day following a dry day, given some evidence, is p01, and the chance of a wet day following a wet day, given some evidence, is p11, and say, p11 > p01. Since these are probabilities, they don’t for instance tell us why wet days are more likely to follow wet than dry days; they only characterize the uncertainty.

This is a “two-state” (wet or dry) Markov chain. Of course, you can have as many states as you like (there are 6 above), and the matrix of the probability of going from one state to another, given some evidence, describes the chain. The “stationary distribution” of any chain are the calculated probabilities of how likely the system will be in any state “in the long run”. These probabilities are no longer conditional on the previous state, but they still are (obviously) on whatever evidence was used.

There is no such thing as “the long run” as in Keynes’s quip and in the directors odd idea of infinite simulations, but these stationary distributions are useful as approximations. Say we wanted to know the chance of wet day conditional only on the evidence and not on whether yesterday was wet or dry. We get that from the stationary distribution. If, for example, p11 = 0.5 and p01 = 0.1, then the stationary distribution is π(0) = 0.17 and π(1) = 0.83 (if the back of my envelope isn’t misleading me).

What Juan did was to use the evidence of questions changing answers in the sample to guess the probability of each of the answers would be given by the population as a whole, e.g. the probability of being white, etc. Understand that this final guess was based on the guess of the transition probabilities. No matter what, guessing, i.e. models, were involved.

Is modeling immoral? Those stationary distributions are deduced, i.e. they follow no matter what, from the transition probabilities. They’re there whether they’re used or not.

One possibility is that the Census is supposed to be an enumeration—no models. And thank the Lord for that. Perhaps the director thought any introduction of models breached their mandate? Doubtful, since the document which gave the table above is filled with statistical models.

There’s even a model for the table above, which attempts to do exactly what Juan did, but using another classical test (“Index of consistency”). So I’m lost. Is this yet another frequentist panic, another instance of the Deadly Sin of Reification?

What do you think?


30 Comments

St Paddy’s Day Parade Report

Not the parade, but it is St Patrick.

I sauntered early up 64th to 5th avenue next to the grandstand and the first event o’ the day was a woman weaving eastwards with enormous bare thighs thrusting out from under a tiny tutu. Green, naturally. She was giggling with her mate, a lovely, healthy young thing who wore shorts and a t-shirt that came just above her pertinents.

My thoughts at the time: Boy, must they be cold, I wonder what St Patrick thinks about this and Confirmation of earlier reports that the pubs opened with the sunrise.

From there, a tremendous lull in the action.

Anyway, since the parade has been made into yet another political event—from which there are precious few, and dwindling, respites—it’s as well to get that nonsense out of the way.

Mayor de Blasio didn’t show in yet another year of protest. He claimed there were not enough marchers who announced with whom or with what they would like to have sexual relations with. (About one such group, more below.) This being the question is today’s society, perhaps he felt everybody should wear some kind of sticker or emblem so that passersby could know whom to hit on and whom to ignore.

Some of us weren’t buying his excuse, though. De Blasio’s relations with police—they and their supporters are a major parade presence—are sour at best, and I think he didn’t want to risk boos from the crowd. He’s a sensitive creature.

A shillelagh-wielding Cardinal Dolan and retinue trotted past shortly after the Fighting 69th, who opened festivities. The prelate received a few polite applause and a wave from two sisters standing nearby.

The first pipe band was announced by a wee Irish lass who shouted to her mother, “Look! A man wearing a skirt!” Poor thing isn’t up on her politics. But she was right. Indeed, it was many men. And boy did they rouse the blood!

I stood next to an Irish gentleman and his wife. A giant, frat-boy leprechaun who was working the crowd, encouraging young ladies to have pictures taken with him. I asked if the gentleman got that sort of thing back home. “Started by the yanks,” he said.

It was yanks who invented the selfie stick, too. An annoying wispy-haired tourist elbowed to the barricade and proceeded to photograph himself in every conceivable pose, but always with the same unhappy smile and the parade at his back, after which he left. As I’ve said before, when one day I read the headline, “Obnoxious Tourist Beaten To Death With Selfie Stick” I won’t weep any tears.

Lots of cops, firemen (the Danish have helmets that look like those from the movie Fahrenheit 459), soldiers, sailors, airmen, marines, veterans, sanitation crews, teachers, Irish appreciated societies, high school marching bands from the world over, drill teams, pipe bands galore, Ancient Order of Hibernians suborders with patron saint banners and chaplains, bands of religious with various Our Ladies, and, near the end, one set of political folks.

They were dull. By the time the “I want to have sex with people of the same sex” contingent (and their allies) marched by the grandstand (about 4:20), the television cameras had been off for over an hour and most of the spectators had departed. The south stands had one person, the main bleachers about twenty, and the north exactly ten folks (I counted), mostly playing on their devices. North half of 64th to 65th on the east side was empty.

Still, the small green-sashed group eeked out some thumbs up and applause from the people remaining. The marchers were loosely surrounded by as many bored cops, some wearing light blue “Community Affairs” jackets (but still armed). The only thing approaching excitement was when the sex group got stopped at 65th to let a firetruck (with sirens blazing) pass (65th is one of Central Park’s exit, cross-town streets).

Doubtless there will be complaints that the “I’m a man/woman and like to see other men/women naked” people should have been more prominently placed (“They didn’t even let us on TV!”). Everybody wants to not only to be a victim, but to be so publicly.

But maybe organizers will realize that, for just one day a year, and for only four hours out of that day, in a parade devoted to one saint and featuring several others, we can do without the politics and just enjoy ourselves.


7 Comments

Answering A Critic On Sampling Variability

ransom51

Alfred ‘Dominant Strategy’ (ADS) is confused that “William Briggs is confused on sampling variability”.

I wrote an article highlighting misconceptions and mistakes people make when thinking about sampling variability, and ADS kindly answered (I couldn’t discover the gentleman’s full name). In the spirit of peer review done rightly—openly, and not as a blunt instrument to suppress unpopular or misunderstood views—this answer to his rebuttal.

ADS says I think sample variability “is due to the assumption that the population follows some underlying probability distribution. That is not the case.” Well, I agree, and I don’t see how ADS missed my agreement.

It isn’t the case, i.e. it is false, that anything “follows” a probability distribution. As I say in the article and in another article which I linked to (repeated here), to say “variables” “follow probability distributions” or are “distributed as” this or that probability distribution is a common mistake and an error in ascribing cause.

It is our knowledge of the value of certain propositions that is quantified by probability distributions. Epistemology not ontology.

ADS and I agree we’re discussing a “population” of a fixed size about which we want to characterize the uncertainty of some measurement on each member of the population. As is typical, ADS gives a blizzard of math in place of simple words and speaks of “random samples” with “non-zero probability” for collecting samples from the population.

ADS agrees with me that if we knew the values of the thing for each member of the population, we’d be done. I state simply we’d know the values and therefore don’t need probability models. ADS says the average of the thing (across the population) “is not a random variable!” Although I didn’t say it in the sampling piece, I often say that “random” only means unknown, and variable means can take more than one value. So to say a thing is not a random variable is to say we know its value, which is much simpler, no?

ADS is only concerned with taking the average of the measurement across the population, whereas I talked about ascertaining the values of the measure for each individual, so my view was broader. But except for the unnecessary (mystical) language about randomness, there is no real divergence thus far.

I said we start with probative evidence about the thing of interest and use it to deduce a probability (model) to characterize the uncertainty in the unmeasured values of each member of the population, which if you like (cartoon) math is written:

     [A] Pr( Measure takes these values in the 300+ million citizens | Probative Evidence),

which can be converted to the following if we’re only interested in the mean across the population of the measure:

     [A’] Pr( Mean value of the measure of the 300+ million citizens | Probative Evidence).

This puts ADS and me on the same ground. Now suppose we have take a sample of measurements, which we can and should use to give us:

     [B’] Pr( Mean value of the measure of all citizens | Observations & Probative Evidence).

And we’re done, because [B’] can be expanded to accommodate all the measurements we have (on the right hand side). Of course, [B’] doesn’t tell us the exact value of the mean (of the thing), but gives us the probability it takes whatever values we supply (e.g. Pr( Mean = 17.32 | Observations & Probative Evidence) = 0.02, etc., etc.).

ADS goes the classical route and speaks of the sample mean being an estimate of the population mean, and that we can calculate the “variance” of the sample mean, variability which he calls sampling error. Of course, the classical interpretation of the “confidence interval” which uses this variance is itself a problem (see this or the Classic Posts page).

The problem is we don’t care about the sample mean and some interval. We want [B’]. If we had to guess what the population mean was based on [B’], we could, but that’s a decision (a prediction!); the best guess would depend on what penalties we’d pay for being wrong and so forth. If we don’t need to decide, we fall back on [B’], which contains everything we know about the uncertain quantity given the totality of our evidence.

ADS says “William Briggs is confused because he mixes sampling error with statistical inference.” Rather, ADS is confused about the goal of measuring a sample. But his is a common mistake; indeed, his view is taught as the correct way to do things.


7 Comments

I Made “Climate Denial MVP”!

Mine didn't have the white stripe, and was one model year earlier.

Mine didn’t have the white stripe, and was one model year earlier.

Gollum (who’s he? see this post on the political witch hunt of scientists) wrote yesterday to announce that a group of us made the “Climate Denial MVP” List.

I admit to being just a little proud of the distinction. I replied to Gollum saying, “When the hysteria ends and Science returns to the Real World, we’ll all be able to look upon this list and be satisfied that while everybody else had lost their way, we stayed on the path of Truth.”

The List (which is published under another name) appears at a site called “Inside Climate News” by one Katherine Bagley. Who’s she? A writer whose “print and multimedia work has appeared in…YouBeauty.com…” Here is what she wrote for my mini-biography:

Briggs is a statistician at Cornell University in Ithaca, N.Y., and a consultant at New York Methodist Hospital. More than two decades ago, he spent a year as a meteorologist for the National Weather Service. He is listed as an expert on the Heartland Institute’s website, where he wrote, “Climate change is of no real interest to anyone except climatologists.” Earlier this year, he co-wrote an article in the peer-reviewed Chinese Science Bulletin with fellow climate denialists Christopher Monckton and Willie Soon arguing that the IPCC’s models are inaccurate and the world won’t warm dangerously this century.

I also for one year drove a 1964 Plymouth Barracuda with a 273 cu. in. V8 automatic, red with curved rear window, chrome gas pipe, and AM radio, which I rebuilt and repaired and which I loved. Had to sell it when I got orders to PCS to Okinawa. Ah well.

How is my owning this gorgeous muscle car relevant to the work I’ve done in climatology? It isn’t. Neither is it relevant that Bagley wrote for YouBeauty.com to her announcing my coveted status on this MVP list. Whatever I or Bagley says on any subject must be judged by the merits of our arguments, not on who we are.

Our professional qualifications are interesting only to the extent that they tweak your interest into considering what we might say, or as possible reasons why what we have said was true or false.

That’s why I’m sure Bagley won’t mind that I pulled the same trick she pulled on me and left off a few of her more pertinent accomplishments. Those include also writing for “Popular Science, OnEarth, YouBeauty.com, Audubon, The Scientist and Science Illustrated, among others” and that she “holds master’s degrees in journalism and earth and environmental sciences from Columbia University.”

Since I’m guessing Bagley won’t be available to make corrections, here’s more about me.

My ties to Cornell are looser than Bagely lets on—I’m a Adjunct there—but I am a part-time consultant at Methodist, among other places (and why haven’t you hired me yet, dear reader?). I did spend a year launching enormous hydrogen filled balloons for the Weather Service. It is also true that climate change is of no real interest to anyone except climatologists, and I do say the world won’t warm dangerously this century (Bagely’s implication is that, of course it will).

Here’s what was left out (and which was available for a click). Both my Bachelor’s and Master’s are in the atmospheric sciences; my PhD is in mathematical statistics (with dissertation angled towards the atmospheric sciences). I served for several years on the American Meteorological Society’s Probability and Statistics Committee. I was also for several years an Associate Editor at Monthly Weather Review (if you don’t know what that is, you shouldn’t be reading Inside Climate News).

I have published in the Journal of Climate, and in several other like sources. On what subject? How to measure forecast goodness. And how to quantify how useful and valuable predictions are. Mixtures of physics, phrobability, and philosophy (yes, phrobability).

This is how I know that long-term climate models aren’t of much value. Models which predict out a handful of months ahead, however, have modest usefulness, diminishing as lead time increases. But those IPCC-like models which predict years ahead aren’t any good. You’d do better with persistence, which is the forecast that next year will look like this year. If a model can’t beat persistence, it shouldn’t be used. Simple as that.

Gollum didn’t mention if there’d be a trophy or honorarium. I’m guessing not. Climate science does not pay well to those unwilling to toe the Government Consensus line.

——————————————————————————

Note I’m growing concerned about Gollum. He predicted the DOJ would enter the witch hunt, but so far it hasn’t. But he made that prediction before Senator Inhofe’s blowback; plus the DOJ is spending a lot of time trying to get their new rights-rights-rights boss installed. So stay tuned.


26 Comments
« Older posts Newer posts »

© 2015 William M. Briggs

Theme by Anders NorenUp ↑