“Why’s that, doctor?” asked the chairman.
“Jones has a beady anterior cingulate. The mark of a clear recidivist.”
The board members peered at Jones, who sat in a chair attempting a sheepish smile. The chairman imagined he could see Jones’s whithered anterior cingulate. Satisfied, he nodded to himself.
“Our patented statistical models, based on the very most scientific functional magnetic resonance imaging, shows Jones has a 95.2% chance of committing new crimes within the next four years. Probably of a highly anti-social nature.”
“Take him away!” shouted the chairman. “Next case!”
Not fiction, ladies and gents. A foretelling of our future. Newspapers and Nature are already excited. Why?
Seems some enterprising doctors led by Eyal Aharoni (and pushed along by Michael Gazzaniga) hooked 96 crooks to their fMRI machine and claimed that the “odds that an offender with relatively low anterior cingulate activity would be rearrested were approximately double that of an offender with high activity in this region, holding constant other observed risk factors.” They published their results in PNAS with the disquieting title “Neuroprediction of future rearrest”.
Wary readers will have noted the all-important fudge words “holding constant other observed risk factors”, which means the new crime-detector is actually a complicated statistical model. How did they arrive at it?
First, they measured the crooks’ anterior cingulate cortexes (ACC) and gave each a questionnaire, sort of like you can find in women’s magazines which ask “What type of animal are you?” Now questionnaires are the basis of much modern science, so we daren’t question them. This one was “a go/no-go (GNG) impulse control task”. Results?
Wait. You would think a study which purported to claim men with beady ACCs are more likely to become recidivists would give us the number of men with beady ACCs who did and did not in fact become recidivists, so that we could compare it against the number of men without beady ACCs who did and did not became recidivists. That way all civilians could tell at a glance whether beady ACCs had anything to say about crime.
Problem was: every man in the study was re-busted (I discovered this only after examining the supplementary data)! The study was therefore also a bust. But when you have numbers, you can play. Like this:
By using hierarchical linear regression, we examined the association between ACC response and the percentage of commission errors in the GNG task. As expected, lower ACC activity entered at step 2 corresponded to a higher rate of commission errors, controlling for variance attributable to age at step 1 (R2 = 0.08, ΔR2 = 0.04, β = −0.21, P < 0.05).
If you can read this standard shorthand, it means the explanatory power of the model after adding ACC only reached the official level of, “You’re kidding me, right?” An R2 improvement of 0.04 isn’t even trivial (the measure goes from 0 to 1, which numbers closer to 1 indicting better models; numbers near 0 should be laughed at). But p-values less than 0.05 are science, so now we know beady ACCs cause one to suffer on go/no-go impulse control tasks.
Just what is a go/no-go impulse control task? Glad you asked.
[A way to present] participants with a frequently occurring target (the letter “X”; occurrence probability, 0.84) interleaved with a less-frequent distracter (the letter “K”; occurrence probability, 0.16) on a computer screen. Participants were instructed to depress a button with their right index finger as quickly and accurately as possible whenever they saw the target (“go” stimulus) and not when they saw the distractor (“no-go” stimulus).
Besides asking the crooks to press a bunch of Xs and Ks, they asked if the crooks were drunks, whether they took a toke, etc. Their race, age, IQ, and a bunch of other stuff were measured. Much of the other stuff was input into a “factor analysis” which spit out factor scores. These scores and the stuff that didn’t go into the factor analysis was all re-input into a survival analysis model, which—finally!—showed that men with beady ACCs were more likely to be re-arrested quicker. A p-value less than the magic number (0.05) confirmed the association.
Now I did this quickly, so I could be in error, but using their data I looked at the variables indicating months to rearrest (MinMonthsDV_noPVs) and beady ACC (dACC_14mm_split) and found a mean 19.5 months to rearrest for beady ACCs and 24.5 months to rearrest for non-beady ACCS. The time-honored t-test (not “controlling” for the slew of other variables) gave a p-value of 0.08, which means not significant.
Which p-value is right? Neither. There is no “right” p-value in classical statistics. You’re allowed to use any you like—including the smallest one.
————————————————————–
Thanks to Al Perrella for pointing us to this.
“As expected …”. Not embarassed to say, “We found what we were looking for” then.
Shades of Minority Report?
Phrenology of a new age.
In the Tom Cruise movie Minority Report psychics were used to indict people of “Pre-Crime” — crimes they had not yet committed. Brain scans seem much more scientific.
^ “seem” being the key.
We got a celebrity shrink in Australia that can put you in a whizzy chair and do a vestibular EEG. Then they can tell you whether you’re a schizo or just manic. It’s true, it was on the news.
As with so many things, there’s part truths intermingled with ignorance & oversight.
The paper, “Neuroprediction of Future Arrest,†is a good example of some (emphasis on “someâ€) accurate research findings extrapolated to nonsense.
We KNOW with CERTAINTY that particular brain patterns (physical and/or dynamic as measured in stimulus-response tests while observed in an MRI [e.g. what parts “light up†in response to particular stimuli or taskings) are associated with particular skills and/or behavioral traits. That much of the above research is valid in principle relative to other findings—the parts identified as indicating recidivism may be indicators of such—but for any of a variety of reasons or combinations of reasons.
However, where the research runs astray, very seriously, by ignoring how much the brain can & does change with changing stimuli.
This is a sort of “which came first, the chicken or egg†situation: is the prisoner’s brain indicating a potential for recidivism due to inherent embedded traits, or, is that physical indicator present (and let’s assume the link to recidivism is valid) because the person’s life situation encourages that sort of behavior & physical brain development? Put another way, if the person with recidivistic tendencies were given truly rehabilitative care/opportunities (& prison is likened to a training school for crooks) it’s very likely that physical indicator would subside, significantly.
Ignoring the brain’s ability to change relative to stimuli—known as neuroplasticity—is a fundamental failure of this research as it indicates an archaic & disproven view of the brain as a fixed structure known to be false.
A great book about the brain’s ability to change itself—focuses on strokes & some amazing findings (curiously, some of thi was discovered some decades/century ago but overlooked until the past decade or two): http://www.amazon.com/The-Brain-That-Changes-Itself/dp/0143113100
London Cabbies example: London’s roads are a confusing maze & cabbies spend some two+ years learning the routes – and their brains have been studied before, during & after & found to physically change in areas associated with spatial relationships: http://sharpbrains.com/blog/2011/12/13/study-brain-changes-seen-in-cabbies-who-take-the-knowledge/
The brain also changes in response to other stimuli: http://www.psychologytoday.com/blog/heal-your-brain/201107/neuroplasticity-and-depression
Research technique applied to understanding neuroplasticity can be obtained via: http://cambridge.films.com/id/27037/Maguire_Navigation_and_Brain_Structureâ€â€Core_Studies_in_Psychology.htm
WHERE IT’S EXPLOITED TO THE FULLEST (so far): “Deliberate Practice†– taps findings associated with the brain’s neuroplasticity – applied to teaching, but athletes & coaches have really exploited this:
Deliberate Practice, early theory (“why†vs. observations about “howâ€â€”what works…and quality matters as much or more than quantity…and “quality†has particular features); for examples see: http://www.missiontolearn.com/2010/04/deliberate-practice/ and http://projects.ict.usc.edu/itw/gel/EricssonDeliberatePracticePR93.pdf
More recent findings indicate “Deliberate Practice†is linked to neuroplasticity (not Erickson’s views regarding genes); for example: http://westallen.typepad.com/idealawg/2011/05/deliberate-practice-a-better-way-to-teach.html and http://www.sportsvisionmagazine.com/training/brain/neuroplasticity.html
With these rusty callipers I can deduce your tendency to witchcraft.
We truly are reliving phrenology and eugenics, when shall we start burning the heretics for real?
I guess the mob psychology cycles through predictable patterns.
Like math is hard, becoming informed enough to make informed decisions is just so old fashioned, like conservative.
Too much work, subject to too much peer disapproval, best to bleat with the herd.
These “new sciences” are mostly social studies under hyper syllable descriptors.
“Which p-value is right? Neither. There is no “right†p-value in classical statistics. You’re allowed to use any you like—including the smallest one.”
The basic problem still remains, Take a sample from a population under condition ‘A’ Change to condition ‘B’ take another sample. How do we discriminate between variability in the population and the effect of the changed condition, absent an obvious overwhelming qualitative difference?