Yet another entry in a long and endless series of experiments which show people often see what they want. Twist in this new one is that people who aren’t too good at math stay not too good, and that people who are good sometimes forget to be good.
Paper is “Motivated Numeracy and Enlightened Self-Government” by Dan M. Kahan and some others, available at the Social Science Research Network. Kahan and pals use theory piled atop theory to explain their motivation and results, but we can skip over all of that and focus on their experiment. (The authors also created a wonderfully intricate regression model which I deemed best to draw a veil over.)
Authors found about a thousand folks who looked more or less like a sample from the US would, and then asked them to solve story problems. This provided a “numeracy” score from one to nine for each individual. Everybody was also classified on the side of angels (Conservative Republican) or the Other (Liberal Democrat) based on self report.
Then people saw these:
The first setup told folks that an experiment had been run wherein some people used a new skin cream and some didn’t, and here were the results, which were either the top-left 2×2 (A) or the top-right 2×2 (B). The people were asked to consider these results and then say which was better: skin cream or not, under (A) or (B).
Regardless of (A) or (B), only about half the people could figure it out. But three-quarters (not 100%) of those with “numeracy” scores seven or better nailed it. Any professor who has ever administered a Statistics 101 exam will be familiar with results like this. Nothing exciting so far.
The next experiment showed people the same tables, except now supposedly derived from data in cities which did or did not ban carrying handguns. Same idea as before, with two different outcomes (C) or (D). Similar question, too: was the ban of handguns associated with an increase or decrease in crime?
Again, only about half could figure it out. The small “Aha” came from splitting conservatives and liberals. The numerate (story-problem scores 7-9) lefties did well on the quiz when crime decreased, but faired just as poorly as innumerates when crime increased. And, as you saw coming, the numerate righties did well on the quiz when crime increased, but faired just as poorly as innumerates when it decreased.
Interpretation? Given the authors did not present raw numbers but only (unnecessarily and overly) massaged curves and my analysis is derived from those, it’s safe to say people roughly saw what they expected to see even when they had the facility to work out the right answer. And when they didn’t (have the facility) they didn’t use what they didn’t have.
Like I said, no surprise. But it still bears repeating that confirmation bias is prevalent and that even you can suffer from it. Yes, even you, even me, and even Chris Mooney who wrote about this paper at Mother Jones under the headline Science Confirms: Politics Wrecks Your Ability to Do Math.
Science (who could not be reached for comment) confirms no such thing. Although it is possible politics wrecks people’s mathematical ability, far more likely is that people, especially smart people, skip the math when they think they already know the answer. Or that they see what they want to see, as is common.
Chris Mooney, for your edification, wrote the books The Republican War On Science and The Republican Brain: The Science of Why They Deny Science. He also wrote the articles “Diagnosing the Republican Brain“, “The Republican Brain on the Republican Brain“, and “Why the GOP distrusts science“.
One suspects a theme.
Thanks to Roger McDermott for finding this paper.