William M. Briggs

Statistician to the Stars!

Page 151 of 685

Predicting Doom—Guest Post by Thomas Galli

Some treatments are more efficacious than others.

Some treatments are more efficacious than others.

I am not a statistics wizard; an engineer, I value the predictive power of statistics. Indeed, if one can precisely control variables in the design of an experiment, statistics-based prediction of future material properties is remarkably accurate. The joy of predicting end strength for a new carbon nanotube concrete mix design in minutes versus days melts the heart of this engineer.

This predictive power has a foreboding downside. It attaches to other projections, including those used by the medical profession to forecast life after diagnosis with late-stage cancer. Unfortunately, I have first-hand experience with this. I was granted but 6 months of remaining life nearly 11 years ago! My doom was predicted with certainty, and for a while, I believed it.

In the dwell time between treatments, I searched for methods used to generate projections of doom. Each patient’s type, stage, age, ethnicity and race were reported to the National Cancer Institute upon diagnosis. Deaths were also reported but not the cause of death. Nothing was captured on complicating health problems like cardio-pulmonary disease, diabetes or other life-threatening diseases. The predictive data set appeared slim.

My battle turned while mindlessly searching web pages of the American Cancer Society. Ammunition in the form of a powerful essay from the noted evolutionary biologist Stephen Jay Gould—“The Median Isn’t The Message”—contained the words: “…leads us to view statistical measures of central tendency wrongly, indeed opposite to the appropriate interpretation in our actual world of variation, shadings, and continua.”

The statistician seeks to aggregate and explain. I’d forgotten that I was in a “world of variation,” was but one data point in about 1.4 million Americans diagnosed in 2004. I might be “the one” on the right-shifted curve prohibiting intersection with the x-axis.

There was one benefit from my encounter with predictive doom. I found hope—something no statistician can aggregate or explain.

Gould survived 20 years beyond his late-stage, nearly always statistically fatal, abdominal cancer diagnosis. Ironically, he passed after contracting another form of unrelated cancer. A distinguished scientist, Gould eloquently described the limits of science and statistics by suggesting that “a sanguine personality” might be the best prescription for success against cancer. There is always hope, with high confidence.


Editor’s Note I have long been interested in working with physicians who routinely make end-of-life prognoses. The concepts of rating such judgments are no different than, say, judging how well climate models predict future temperatures. I mean predictions should be rated on their difficulty. I haven’t yet discovered docs willing to conduct these experiments, but if anybody happens to know somebody, let me know.

Nonpolitical Images Evoke Neural Predictors Of Political Ideology?

From the paper.

From the paper.

The study

Another day, another dreary study purporting to show that the brains of “conservatives” are different than those of “liberals.”

This one hooked up to an electrical phrenology device (fMRI) 83 people1 and had them look at disgusting pictures (still shots from The View?) and other sorts of pictures and then rate them “using a nine-point Likert scale”. I’ve asked this before, but on a scale of -2 to 52.7, how good are these faux numerical scales at quantifying things like disgust or pleasantness? Never mind.

The peer-reviewed paper is by Read Montague and a slew of others in Current Biology, and has the same name sans question mark as today’s post.

To discover “conservatives”, “liberals”, and “moderates” questions were asked about how strongly participants supported items like “Biblical truth” (do no liberals believe this?) and “Foreign aide”. These were scored, the scores separated, and the results assumed infallible. Yes, really. There is no indication—which is to say, no indication—the uncertainty from these arbitrary questions arbitrarily scored and arbitrarily busted up was carried through in any analyses. But since everybody makes this mistake, we shouldn’t question it.

Anyway, the main result is no result. The three “groups did not significantly differ in subjective ratings of disgusting, threatening, or pleasant pictures”. Also turned out that “there were no significant group differences on [other] self-report measures”.

End of story? No, sir. Scientists do not let the absence of wee p-values discourage them. Out came the “penalized regression method called the elastic net” applied to the fMRI data. The theory was that even though there were no real differences in behavior, maybe the brains were different after all, which is a strange thing to think given there were no real differences in behavior. I hope my repeating that isn’t annoying.

Is this a good point to remind us the fMRI data are not pictures of the brain but are themselves output of models and heuristics (“Functional data were first spike-corrected to reduce the impact of artifacts using AFNI’s 3dDespike”, etc., etc.) which themselves are subject to uncertainty which should be carried forward in any analysis but which usually aren’t, and weren’t here? If not, let me know when is.

The analysis

I hesitate to describe the authors did next not because it’s difficult, but because I don’t think anybody will believe it. I will first remind us that we are to again lament that most statistical practice is designed around model fit, which tell the world how closely a model fits to the data at hand, and that the more models tried the better success of discovering one which fits.

The authors showed each person sets of neutral (whatever the hell that is), pleasant, threatening, and disgusting photos. There weren’t any reported differences in fMRI manipulated data between people seeing these images in the three different groups.

Next up was to form “contrasts”, which was to sort of difference the fMRI manipulated data from times when people looked at disgusting, threatening, and pleasant images against so-called neutral images. These same differences were applied to averages between “conservative” and “liberals.” The “moderates”, sad folks, were thereafter forgotten.

Incidentally, the types of people in the “conservative” and “liberal” groups were not the same: “liberals” averaged 33 years old, 39% female; “conservatives” 27 years old, 61% female. Might these biological differences account for differences in fMRI manipulated data? The authors admit (in supplementary material) that “religiousness”, age, and sex “were significantly correlated with political attitudes”. But they put this down to “false alarms” and carried on.

Now came generalized linear models—we still haven’t reached the elastic net—where for each individual “a temporal high-pass filter (128s) and order 1 temporal autocorrelation (AR(1)) was assumed”. And “The onsets for each picture subcondition (core/contamination disgust, animal reminder disgust, actual threat, no actual threat, social pleasure, nonsocial pleasure) and fixation crosses were convolved with a canonical hemodynamic response function…using a delta function of zero duration”, etc.

And that wasn’t all. “Six head motion parameters were also included in the first level GLM as covariates.” So were age and sex. Uh oh. Then they “separately examined the maps of [Disgusting – Neutral], [Threatening – Neutral], and [Pleasant – Neutral] contrasts”. Then some t-tests and some other things.

Result? “The contrasts with threatening or pleasant pictures revealed no regions surviving multiple corrections. However, in the [Disgusting > Neutral] contrast, the Conservative group showed greater activity than the Liberal group in several regions” (hint: amygdala! amygdala!). Yet, sadly, “No regions survived correction for multiple comparisons for the
[Liberal group > Conservative group] comparison.”

Another no result. So back to the computer and the “penalized logistic regression analysis”, a.k.a. “elastic net”.

“First, we extracted a map of the [Disgust > Neutral] contrast for each participant. Then, we applied an a priori mask, which was generated from the Neurosynth website”. Then they “obtained the union of meta-analytic (positively correlated and both forward and reverse inference) maps of ‘Emotion’ and ‘Attention'” and then finally formed up all the voxels into a matrix and submitted all to the “elastic net.”

That creature is so cumbrous I don’t dare describe it. But it was, in the end, fit to the “individual scores on a standard political ideology assay” and, mirabile dictu, the model fit was reasonable. But only for those time disgusting images were viewed (and leaving out “moderates”). Would young females dislike disgusting images more than older males? Just asking.

The true test: How well does their model predict political attitudes for people not used to fit the model? [INSERT CRICKET CHIRPS HERE]

The End

The authors conclude “Neuroscience has started to provide rich information about the neurophysiological processes underlying political behavior.” No, it hasn’t. It is true that a spate of flawed papers are appearing, each borrowing the mistakes of the other. Yet the authors don’t even blush when the say “Our results have important implications for the links between biology, emotions, political ideology, and human nature more fundamentally.”

Here’s where it gets scary, folks. They suggest “people are born with certain dispositions and traits that influence the formation of their political beliefs”. This seems trivially true; after all, some of us are men and some women, and that difference means a lot. But the differences the authors means refer to flawed ad hoc idiotically scaled questionnaires. How long until some bright academic produces “the” list of questions which separate the sheep from goats?

Next: “A wide range of brain regions contributed to the prediction of political ideology (Figure 3A), including those known from past work to be involved in the processing and interoception of disgust and other stimuli with negative affective valence, but also those involved in more basic aspects of attentive sensory processing”.

The mistake here is to assume we are our brains, slaves to them somehow, that these curious organs can make us do what they like, and that we have little to say about it. The lack of philosophical training tells again.

Nowhere do these authors (or any other that I have seen) betray any lack of confidence in their convoluted analyses. It seems as if—I’m just guessing—that all these authors think that because their analyses are complex they are therefore right. We need a name for this fallacy.


1In supplementary material the authors say 12 people were removed from the analysis, but it’s not clear if these were before or after the 83.

Thanks to Rexx Shelton, Robert, and one anonymous reader for suggesting this topic.

Nothing Is Distributed: So-Called Random Variables Do Not Follow Distributions

Wow is this wrong, but common, common.

Wow is this wrong, but common, common.

People say “random” variables “behave” in a certain way as if they have a life of their own. To behave is to act, to be caused, to react. This is reification, perhaps caused by the beauty of the mathematics where, literally, the equations undergo biogenesis. The behavior of these “random” creatures is expressed in language about “distributions.” We hear, “Many things are normally (gamma, Weibull, etc., etc.) distributed”, “Y is binomial”, “Height is normally distributed”, “Independent identically distributed random variables”.

I have seen someone write, “[Click here to] see a normal distribution being created by random chance!” Wolfram MathWorld writes, “A statistical distribution in which the variates occur with probabilities asymptotically matching their ‘true’ underlying statistical distribution is said to be random.” Examples abound.

All of this is wrong and indicates magical thinking. It is to assume murky, occult causes are at work, pushing variables this way and that so that they behave properly. To say about a proposition X that “X is normal” is to ascribe to X a hidden power to be “normal” (or “uniform” or whatever). It is to say that dark forces exist which cause X to be normal, that X somehow knows the values it can take and with what frequency.

This is false. We are only privileged to say things like this: “Give this-and-such set of premises, the probability X takes this value equals that”, where “that” is a deduced value implied by the premises. Probability is a matter of ascribable or quantifiable uncertainty, a logical relation between accepted premises and some specified proposition, and nothing more.

Let S = “Sally’s grade point average is x”. Suppose we have the premise G = “The grade point average will be some number in this set”, where the set is specified. Given our knowledge that people take only a finite number of classes and are graded on a numeric scale, this set will be some discrete collection of numbers from, say, 0 to 4; the number of members of this set will be some finite integer n. Call the numbers of this set g_1, g_2,…, g_n.

The probability of S given G does not exist. This is because x is not a number; it is a mere placeholder, an indication of where to put the number once we have one in mind. It is at this point the mistake is usually made of saying x has some “distribution”. Nearly all researchers say or assume “GPA is normal”; they will say “x is normally distributed.” Now if this is shorthand for “The uncertainty I have in the value of x is quantified by a normal distribution” the shorthand is sensible—but unwarranted. There are no premises which allow us to deduce this conclusion. This is pure subjective probability (and liable to be a rotten approximation).

When they say “x is normally distributed” they imply that x is itself “alive” in some way, that there are forces “out there” that make, i.e. cause, x to take values according to a normal distribution; that maybe even the central limit theorem lurks and causes the individual grades which comprise the GPA to take certain values.

This is all incoherent. Each and every grade Sally received was caused, almost surely by a myriad of things, probably too many for us to track. But suppose each grade was caused by one thing and the same thing. If we knew this cause, we would know the value of x; x would be deduced from our knowledge of the cause. And the same is true if each grade were caused by two known things; we could deduce x. But since each grade is almost surely the result of hundreds, maybe thousands—maybe more!—causes, we cannot deduce the GPA. The causes are unknown, but they are not random in any mystical sense, where randomness has causative powers.

What can we say in this case? Here is something we deduce: Pr(x = g_1 | G) = Pr(x = g_2 | G), where x = g_1 is shorthand for S = “Sally’s GPA is g_1” (don’t forget this!). This equation results from the so-called symmetry of individual constants, a logical principle. The probabilities are equivalent to G = “We have a device which can take any of n states, g_1, …, g_n, and which must take one state.” From the principle we deduce Pr(x = g_i | G) = 1/n.

“Briggs, you fool. That makes GPAs of 0 just as likely as 4. That isn’t possible.”

Is it not? I see you haven’t taught at a large state university. Anyway, the probabilities deduced are correct. What you are doing in your question is adding to G. You are saying to yourself something like “Pr(g_n | G & What I know about typical grades)” which I insist is not equal to Pr(g_n | G). Either way, x does not “have” a distribution.

Homework 1: discover instances of abuse. Homework 2: What’s wrong with the phrase “independent identically distributed random variables”? Hint: a lot.

Thanks, Fellow Veterans

Off we go.

It was in the first night of my military service that I learned cockroaches could fly. I am from the Northwoods where cockroaches are light on the ground, so it was somewhat of a shock to be standing at attention at two in the morning with a group of dazed men, hearing a thwwwwwwwwp, and seeing a pterodactyl-sized chitinous-armored bug fly over our heads and attach itself to the wall.

That was right before we heard the slow clic-tap, clic-tap, clic-tap of the drill sergeant drawing out his entrance from behind. I think I was more frightened of the miniature dinosaur, which was now extending its wings in a menacing fashion.

They called us a Rainbow Flight because we all wore variously colored civilian clothes and still had long hair. I had a small sack with toothbrush, some spare underwear, shaving kit, that sort of thing. I had, I think, about twenty bucks, which I then (and, given the way things are going, soon will again) regarded as a small fortune. These consisted of my worldly possessions, except for some spare clothes and some books my mom had.

There are two scents that still bring me back. Packaged rough blankets (I don’t know how else to describe it; wool blankets processed cheaply?), and Pinesol. Maybe the memories are triggered by my amygdala: every other behavior apparently is.

Only time I ever caught the attention of the TI was when I accidentally mentioned the name of another unit. He heard this and made all fifty of us rush outside, form up, then rush back upstairs, form up by our bunks, rush back outside, and so forth, about five or six times in all. Turns out our TI hated the TI that ran the other unit.

My first base was Kelly, right next door to Lackland, a major disappointment. Or at least I thought it was at the time. It did allow the Blonde Bombshell to make her way south and get hitched up (she has now served a longer term of service than I did with my Uncle Sam).

Now we had no money but at no time did we ever feel poor. And when I say “no money”, I mean no money. I think the yearly salary then was around $7,100. From which came the rent, groceries, the car, and so on. We didn’t live in the swankiest section of town. The Air Force charmingly picked up the tab for our Number One son, but this was still in the days hospitals didn’t marshal teams of experts to attend a birth.

I became expert at floor buffing, two-deck pinochle, and soldering. Not soldiering: soldering. Very different skill.

After three years of this, off to Kadena and the 1962 Communications Group. The cockroaches were bigger there than in San Antonio. Plus there were deadly slugs, deadly spiders, and a deadly snake called a habu. I never heard of anybody dying from the spiders or snakes, but every now and then a Marine would kick over after being challenged to eat a slug. Or to go swimming in the surf after a typhoon, an especially interesting experience since Okinawa is made of coral. We always thanked God for the Marines—and thanked God we weren’t one of them.

The Navy picked up the tab for Number Two son.

I tooled around Japan and Korea where I first formed the conviction that the human race is insane. I think Sister Dorothy tried to impart this valuable knowledge earlier, but I was stubborn and rebellious and didn’t realize that I was part of the problem.

Once or twice “activists” from mainland Japan came down to protest war and the military. One time they had just enough people to link hands around Kadena. We lived out by the fence and were warned not to go near them, but they seemed friendly enough. Protesting is almost always a social outing with a picnic atmosphere. Of course, this was in the late ’80s and most Japanese probably now think differently.

After we decided to get out, I typed—on a typewriter—maybe 100 letters to various companies asking for a job. Every single one of them wrote back to say No Thanks (every life has constants). Which was but proper and civilized. Those days are gone.

That was me. How many vets do we have here?

« Older posts Newer posts »

© 2016 William M. Briggs

Theme by Anders NorenUp ↑