He gave the quiz’s score a super impressive name: “Science Comprehension,” as if top marks indicated you comprehended all of science instead of showing you answered correctly questions which wouldn’t stump the kids on Are You Smarter Than A Fifth Grader?
Kahan inflicted the quiz on a bunch of people and asked them things like whether they suffered from “religiosity”, had been awarded a “degree”, or were members of the Tea Party.
Lo, slightly more of those awarded a “degree”, a certificate indicating four or more years of expensive education in all things science, scored better than those without a degree. This shows that spending all that money doesn’t buy you much. (Alternate explanation: Kahan over-sampled “Studies” majors.)
Knowing whether a person suffered from “religiosity” meant nothing to knowing how a person would score on the quiz. This will chagrin those who hoped that belief if God precluded knowing that “Electrons are smaller than atoms.”
Perhaps worst of all, knowing whether a person was a Republican or Democrat, or whether a person joined the Tea Party or not, meant almost nothing to knowing how a person would score on the quiz. If anything, ever-so-slightly more Tea Party members knew that “the Earth [goes] around the sun” than non-Tea Party members.
Kahan was shocked by his discovery:
I’ve got to confess, though, I found this result surprising. As I pushed the button to run the analysis on my computer, I fully expected I’d be shown a modest negative correlation between identifying with the Tea Party and science comprehension.
But then again, I don’t know a single person who identifies with the Tea Party. All my impressions come from watching cable tv — & I don’t watch Fox News very often — and reading the “paper” (New York Times daily, plus a variety of politics-focused internet sites like Huffington Post & Politico).
I’m a little embarrassed, but mainly I’m just glad that I no longer hold this particular mistaken view.
This admission was manly—and extremely rare. But he (almost) spoiled it by running as quickly from it as he could, lest his colleagues think him insane, by immediately calling Tea Party members immoral brutes.
Strange that he came to that conclusion based on no direct observational evidence. Not very scientific behavior, that.
This isn’t Kahan’s first foray into “science comprehension.” He and several others were responsible for media-noted “The polarizing impact of science literacy and numeracy on perceived climate change risks.” (See notes below.)
Much the same thing: small trivia quiz matched with iffy political affiliation questions. Then an enormous and creaky statistical apparatus erected on top of these thin reeds with the purpose of teasing out infinitesimal differences. My God! Did they go on and on and on and then on some more? Did they ever. All completely worthless.
These kinds of “studies” come out with depressing regularity. By “these kinds” I mean those concocted by professors who have found ways to escape teaching by engaging in “research.” They gather in the nearest Starbucks (faculty lounges are passé), think of questions which they convince each other are profound and which support a fashionable theory, call the battery of questions an “instrument”, give it to a few students, then analyze, analyze, analyze.
Seriously, now. Just how far can you go with a question like “Lasers work by focusing sound waves [true/false]”? Could you tell how the country should best be run by gleaning insights from a survey which showed that only 69% of respondents knew the right answer? Could you tie the way a person answered that to some obscure psychological, jargon-laden theory? How plausible is it that most people’s politics and religious beliefs would cause people to answer correctly or incorrectly this particular question?
Why not require people to know what happens when a teaspoon of Seaborgium is mixed with a pinch of Livermorium as a prerequisite for marriage? How about insisting people know which has more mass, a neutrino or top quark, before allowing them to purchase soda pops larger than sixteen ounces?
Or what about ensuring a man can derive the fundamental equations of radiative transfer before he’s allowed to create a quiz which supposedly defines “science comprehension”?
It’s all so silly.
Thanks to the many, many readers who alerted me to this study.