“Why’s that, doctor?” asked the chairman.
“Jones has a beady anterior cingulate. The mark of a clear recidivist.”
The board members peered at Jones, who sat in a chair attempting a sheepish smile. The chairman imagined he could see Jones’s whithered anterior cingulate. Satisfied, he nodded to himself.
“Our patented statistical models, based on the very most scientific functional magnetic resonance imaging, shows Jones has a 95.2% chance of committing new crimes within the next four years. Probably of a highly anti-social nature.”
“Take him away!” shouted the chairman. “Next case!”
Seems some enterprising doctors led by Eyal Aharoni (and pushed along by Michael Gazzaniga) hooked 96 crooks to their fMRI machine and claimed that the “odds that an offender with relatively low anterior cingulate activity would be rearrested were approximately double that of an offender with high activity in this region, holding constant other observed risk factors.” They published their results in PNAS with the disquieting title “Neuroprediction of future rearrest”.
Wary readers will have noted the all-important fudge words “holding constant other observed risk factors”, which means the new crime-detector is actually a complicated statistical model. How did they arrive at it?
First, they measured the crooks’ anterior cingulate cortexes (ACC) and gave each a questionnaire, sort of like you can find in women’s magazines which ask “What type of animal are you?” Now questionnaires are the basis of much modern science, so we daren’t question them. This one was “a go/no-go (GNG) impulse control task”. Results?
Wait. You would think a study which purported to claim men with beady ACCs are more likely to become recidivists would give us the number of men with beady ACCs who did and did not in fact become recidivists, so that we could compare it against the number of men without beady ACCs who did and did not became recidivists. That way all civilians could tell at a glance whether beady ACCs had anything to say about crime.
Problem was: every man in the study was re-busted (I discovered this only after examining the supplementary data)! The study was therefore also a bust. But when you have numbers, you can play. Like this:
By using hierarchical linear regression, we examined the association between ACC response and the percentage of commission errors in the GNG task. As expected, lower ACC activity entered at step 2 corresponded to a higher rate of commission errors, controlling for variance attributable to age at step 1 (R2 = 0.08, ΔR2 = 0.04, β = âˆ’0.21, P < 0.05).
If you can read this standard shorthand, it means the explanatory power of the model after adding ACC only reached the official level of, “You’re kidding me, right?” An R2 improvement of 0.04 isn’t even trivial (the measure goes from 0 to 1, which numbers closer to 1 indicting better models; numbers near 0 should be laughed at). But p-values less than 0.05 are science, so now we know beady ACCs cause one to suffer on go/no-go impulse control tasks.
Just what is a go/no-go impulse control task? Glad you asked.
[A way to present] participants with a frequently occurring target (the letter “X”; occurrence probability, 0.84) interleaved with a less-frequent distracter (the letter “K”; occurrence probability, 0.16) on a computer screen. Participants were instructed to depress a button with their right index finger as quickly and accurately as possible whenever they saw the target (“go” stimulus) and not when they saw the distractor (“no-go” stimulus).
Besides asking the crooks to press a bunch of Xs and Ks, they asked if the crooks were drunks, whether they took a toke, etc. Their race, age, IQ, and a bunch of other stuff were measured. Much of the other stuff was input into a “factor analysis” which spit out factor scores. These scores and the stuff that didn’t go into the factor analysis was all re-input into a survival analysis model, which—finally!—showed that men with beady ACCs were more likely to be re-arrested quicker. A p-value less than the magic number (0.05) confirmed the association.
Now I did this quickly, so I could be in error, but using their data I looked at the variables indicating months to rearrest (MinMonthsDV_noPVs) and beady ACC (dACC_14mm_split) and found a mean 19.5 months to rearrest for beady ACCs and 24.5 months to rearrest for non-beady ACCS. The time-honored t-test (not “controlling” for the slew of other variables) gave a p-value of 0.08, which means not significant.
Which p-value is right? Neither. There is no “right” p-value in classical statistics. You’re allowed to use any you like—including the smallest one.
Thanks to Al Perrella for pointing us to this.