Statistics

On The Severe (And Unrecognized) Limitations Of fMRI

r52

So this 44-year-old Frenchman—let’s call him Jacques—presented for a “mild left leg weakness“. The leg bone being connected to the hip bone, etc., it was eventually discovered that Jacques’s “skull was filled largely by fluid, leaving just a thin perimeter of actual brain tissue.”

Here’s the kicker:

And yet the man was a married father of two and a civil servant with an IQ of 75, below-average in his intelligence but not mentally disabled…

While this may seem medically miraculous, it also poses a major challenge for cognitive psychologists, says Axel Cleeremans of the Université Libre de Bruxelles.

“Any theory of consciousness has to be able to explain why a person like that, who’s missing 90% of his neurons, still exhibits normal behavior,” says Cleeremans. A theory of consciousness that depends on “specific neuroanatomical features” (the physical make-up of the brain) would have trouble explaining such cases.

To say that explaining this man via current theories of the brain is a “major challenge” is like saying Bill Clinton has a “small problem” with the ladies. According to these theories, the man should be a “vegetable”. It’s always vegetable, isn’t it? I guess neurologists didn’t see the original The Thing. Skip it.

The missing matter of Jacques’s brain was noticed by Magnetic Resonance Imaging (MRI). The MRI picture above is from the Lancet (under the bland title “Brain of a white-collar worker”).

Functional MRI is similar, except that fMRI takes pictures to discover functions of parts of the brain. That is the claim, at any rate. How do they work? People are asked to think about, say, vegetables and then squeezed into an fMRI machine, which duly takes its pictures. Loads of statistical manipulations then take place after which it will be discovered that certain areas of the brain glow statistically significantly more brightly than other areas. These glowing areas will then be declared vegetable-fantasizing areas of the brain.

But this wouldn’t work for Jacques, because poor Jacques is light in the head. Yet it’s a sure bet that Jacques knows his vegetables from his Peugeot.

Enter Kate Murphy, who wrote “Do You Believe in God, or Is That a Software Glitch?” in the New York Times (I’m guessing Murphy didn’t provide the article’s title: writers rarely do). Murphy reminds us of the “study published last month in the Proceedings of the National Academy of Sciences uncovered flaws in the software researchers rely on to analyze fM.R.I. data. The glitch can cause false positives — suggesting brain activity where there is none — up to 70 percent of the time.”

Murphy said, “This cued a chorus of ‘I told you so!‘ from critics who have long said fM.R.I. is nothing more than high-tech phrenology.”

No need to click on the link: I am the “I told you so.” I have long warned readers that the statistical methods behind fMRI rely on wee p-values and hypothesis tests, methods guaranteed to lead to over-certainty—as I document in nauseating detail in my new award-eligible book Uncertainty: The Soul of Modeling, Probability & Statistics. Another must-read book is Brainwashed: The Seductive Appeal of Mindless Neuroscience by Sally Satel (psychiatrist) and Scott Lilienfeld (psychologist).

About those statistical methods, Murphy rightly reminds us of the dead fish hooked to an fMRI which “found neural activity in its brain when it was shown photographs of humans in social situations.” This happens because the fMRI is not taking pictures of your brain. It is showing you statistical models based on estimates of which might be happening in your brain.

Other statistical problems in analyzing fM.R.I. data have been pointed out. But these kinds of finger-wagging methodological critiques aren’t easily published, much less funded. And on the rare occasions they do make it into journals, they don’t grab headlines as much as studies that show you what your brain looks like when you believe in God.

…The fM.R.I. errors added fuel to what many are calling a reproducibility crisis.

“People feel they are giving up a competitive advantage” if they share data and detail their analyses, said Jean-Baptiste Poline, senior research scientist at the University of California, Berkeley’s Brain Imaging Center…

There is also resistance because, of course, nobody likes to be proved wrong. Witness the blowback against those who ventured to point out irregularities in psychology research, dismissed by some as the “replication police” and “shameless little bullies.”

For the record, I am a often-shame-filled big bully.

MRIs are not useless. After all, they can show us nearly empty skulls à la Jacques. Shrapnel and other miscellaneous objects also show up nicely. But for telling the difference in brains between rational theists and ultimately irrational atheists? Nuh-uh.

Categories: Statistics

25 replies »

  1. There must be a joke somewhere about the French civil servant with literally no brain… lemme try:
    Q: what’s the difference between a French civil servant with a normal brain and one without 90% of its neurons?
    A: their MRI scans.

  2. Reminds me of the saying “Only the experts were surprised”. Seems everything comes as a shock to experts. There are other examples of people missing parts of their brain and functioning normally, as well as the use of “split brain” to treat epilepsy cases resistant to other means. Humans are far more adaptable than current beliefs allow.

  3. See how science self-corrects — in this case pointing out a tool’s limitations!

    Of course, that might not help all users of the given tool to adjust their technique accordingly…

    …such as noting that while the tool often gives a type of ‘false positive’ reading over-generalizing to conclude that while many study findings based on the tool are suspect and may be wrong in part or their entirety that necessarily all are wrong in their entirety (“nuh-uh”). I.E., this is exactly the same kind of error regarding total dismissal of findings as was made using the tool where findings were mindlessly accepted and extrapolated to a silly degree.

  4. “Every honest researcher I know admits he’s just a professional amateur. He’s doing whatever he’s doing for the first time. That makes him an amateur. He has enough sense to know that he’s going to have a lot of trouble, so that makes him a professional.”
    — Charles F. Kettering

  5. An article like this needs further dissemination. And this would make a good example on Prager University lecture on the unreliability of medical science sold as truth.

  6. Prof. Briggs,

    After reading some of the linked articles and comments thereon, I wonder if maybe you could address the way over-confidence in “wee p-values” intersects with people who dismiss the significance of large-scale failure to replicate published studies. Seems like there could be something there…

  7. Neuroradiologist for 30 years, grew up with MRI.

    Pt. has severe well compensated hydro, with a shunt in place. Cortical mantle is present. One man’s “thin perimeter” is another’s functional cortex. Nothing terribly unusual about the case.

    fMRI has utility in identifying motor and speech cortex to determine tumor resectability, also a host of other uses. It’s measuring deoxygenation that occurs during neuron activation. The differences are slight, and statistical modelling helps with interpretation.

    “MRIs are not useless.” Hey, thanks for letting us know. FYI, MRI, as opposed to fMRI, has, since 1984, proven itself a tremendous diagnostic advantage, first in neuro applications, then musculoskeletal, and later cardiac, abdominal and fetal medicine. To say otherwise is absurd.

  8. As a quondam physicist supervising and implementing clinical MRI and MRI research, I take exception to some of the generalizations in the article. MRI looks at the state of tissue water and can thereby detect lesions, tumors both benign and malignant and has been of tremendous value. There have been, by the way, other instances of “normal” functioning with greatly enlarged ventricles; one case that comes to mind is that of a 12 year old girl with as large a ventricle as that of the French civil servant.

    Functional MRI (fMRI) detects the presence of increased metabolic activity in brain regions by relying on relaxation time shortening due to paramagnetic blood+O2. The technique requires intense pulsed field gradients (and thereby much loud clanging) and in this respect renders judgment about higher order intellectual activity (e.g. meditation or prayer) suspect. Nevertheless, fMRI can correlate regions of brain activity with particular intellectual activities. But the insights aren’t that great in my opinion–as far as I can see, they have not delved deeper than what is known from other techniques. Knowing which brain region is associated with which kind of intellectual activity doesn’t really convey that much insight. It’s like knowing which output connector on a computer goes to a monitor, which to a keyboard, etc. But one can’t knock the technique per se because incorrect statistical analyses have been applied.

    PET and SPECT scans give much the same information as does fMRI, with better time resolution, no distracting noise, but with much poorer spatial resolution. Professor Andrew Newberg’s SPECT studies on meditation and prayer, using nuns, Buddhist monks and atheists as subjects, has yielded some interesting results. See my post (department of shameless self-promotion)
    “Are we hard-wired for faith?”
    http://rationalcatholic.blogspot.com/2014/03/are-we-hard-wired-for-faith-religious.html

  9. One other comment: it’s my experience that some radiologists very often take to using statistical tools like a neophyte does to carpentry and use tools for purposes for which they weren’t intended (e.g. hammering a screw into a board) and with no knowledge of their limitations. I’ve seen regression plots with data that looked like a shotgun fired randomly into a barn wall.
    Nevertheless, there are many signal-processing techniques, some of which have statistical foundations, used in MRI that unambiguously distinguish normal from diseased tissue, so let’s not throw the baby out with the bathwater.

  10. Jersey, I think it only goes to show how unadaptable some minds are. We have here some big-headed, cigarette-smoking drongos who’s skull is presumably stuffed with brain who cannot comprehend mere facts.

    Now, mere facts aside, I contend that the brain is only the physical organ that connects the metaphysical mind with the physical senses and physical activity.

    I, somewhere, heard about a young Pommy fella who had almost no detectable cerebellum but he’s onto his second PhD.

    Perhaps you and your ideological mates would care to explain that in terms of your assumption that mind is an accidental, or chance, product of mere chemistry.

    There is, of course, much more damning evidence against Materialism but I doubt that your monstrous brain could comprehend it.

  11. I, somewhere, heard about a young Pommy fella who had almost no detectable cerebellum but he’s onto his second PhD.

    That’s the relatively small thing in the back of the brain responsible largely for coordination. Getting a PhD with perhaps poor coordination is unsurprising. Now if he lacked a cerebrum that would be surprising or at least raise questions about the current state of education.

    assumption that mind is an accidental, or chance, product of mere chemistry.

    Accidental? Chance? Is your arm an accident of mere chemistry? You do realize that it all didn’t come together at once, yes? And that evolutionary changes proceed in small chunks, yes? Scales morph into feathers and all that?

    I contend that the brain is only the physical organ that connects the metaphysical mind

    That’s nice. Care to support it?

  12. Accidental? Chance? Is your arm an accident of mere chemistry? You do realize that it all didn’t come together at once, yes? And that evolutionary changes proceed in small chunks, yes? Scales morph into feathers and all that?

    Sick of people saying I have chunky arms and now you go say this. Arms evolved in chunks. Wow!

    And scales morph in to feathers–would these be accidents of mere morphs?

  13. It just goes to show how adaptable the brain is.

    Unless it shows how adaptable the whole organism is that uses the brain.

    Now if he lacked a cerebrum that would be surprising or at least raise questions

    See here for the story of the boy without a cerebellum:
    http://tofspot.blogspot.com/2011/02/instrumentality-of-brain.html

    Additional thoughts found here:
    http://tofspot.blogspot.com/2012/07/even-more-ethically-fraught.html
    and here
    http://tofspot.blogspot.com/2012/01/humanism-in-danger.html

  14. See here for the story of the boy without a cerebellum:

    I did. The quote was mostly opinion and when I clicked on the the contained link to the full story up popped a story about Hillary. Were you being funny?

    Not at all sure why lack of motor skills would make one a vegetable or maybe just a moron instead of a mere cripple.

    If anything, the story is about how adaptable the brain can be and how little we still understand as JMJ said. When I had some nerves cut during some dental surgery they grew back in a way that suggests rerouting. I didn’t ask who did the recruiting anymore than I ask who did that when a cut heals.

    I read an article long ago in that once great sci-mag, Scientific American, about how walking skills are mostly located in the spine (ever watch a chicken run without its head?) and apparently how many thing once thought to reside in the brain are distributed throughout the body.

  15. Murphy reminds us of the “study published last month in the Proceedings of the National Academy of Sciences uncovered flaws in the software researchers rely on to analyze fM.R.I. data. The glitch can cause false positives — suggesting brain activity where there is none — up to 70 percent of the time.”

    Yes, again, programming mistakes definitely would produce wrong results. Please note that “up to percent of the time” is a result from simulated data. It’s about when practioners blindly use a ready-made software, and not knowing the assumptions behind the models is.

    How medical techinology has improved and changed healthcare!

  16. many thing once thought to reside in the brain are distributed throughout the body.

    Exactly. It is the whole organism, not a particular organ. Without the whole body, the brain is not even a brain, but a mass of tissue.

    +++
    I don’t know why the link directed elsewhere, unless it was originally a current day’s page and the current day is now a couple years later.

  17. DAV quoted Odd:
    “I contend that the brain is only the physical organ that connects the metaphysical mind”
    Then he commented:
    “That’s nice. Care to support it?”

    There was another report that I read years ago where some psychiatrist-cum-brain surgeon got a couple of his patients to volunteer for one of his experiments.
    He opened up the skull under local anaesthetic and probed around in their brains with an electric prodder. (The brain has no pain receptors so he could poke around the brain of a fully conscious patient). With his prodder he could produce various physical reactions (muscle spasm and the like) but he could not make his specimens to “want” the reactions. Indeed, his specimens invariably tried to resist the artificial stimulus.

    On another level, what epileptic “wants” to go uncontrollably thrashing around on the floor because something in their brain is malfunctioning?

  18. what epileptic “wants” to go uncontrollably thrashing around on the floor because something in their brain is malfunctioning?

    Likely the same number that want a cancerous growth or a heart attack.
    Is there a point in this strange question?

  19. Pingback: Anonymous
  20. This is a blast from the past, thanks anonymous for pinging alert.
    C8 makes the right response to the article.
    Recall the sinking sensation on reading it, years ago.
     
    Dav is correct about the cerebellum, certainly in the absence of any other information about the patient who’s cerebellum does not function. Such patients are often mistaken for being drunk, as the symptoms look similar. The medical term being cerebella ataxia. *ataxic movement being movement without co-ordination.
     
    Re spinal reflexes and primitive spinal reflexes versus descending inhibition as it is known is the difference between a baby’s movements and that of an adult.
    Hence patients with certain types of brain injury or defect exhibit these patterns of movement which resemble an infant. Foetal postures are the default. To put it another way.
     
    Most brain function works on inhibition.
    Re C8’s comment, it is the same for most medical testing. They aren’t expected to stand alone without reference to the patient in question. Failure to do so results in poor outcomes, everybody knows that or at lest is taught that it is so. MRI’s are tools, not diagnoses. Please try and learn and stop repeating the untruthful false claim that medical tools and tests are anything but aids to diagnostics. It was the same with covid, so much dangerous ignorance

Leave a Reply

Your email address will not be published. Required fields are marked *