Tea Partiers Know As Much Science As Enlightened Lefties?

Nice t-shirt, Dan.
Nice t-shirt, Dan.
So this Dan Kahan, Yale prof of something-or-other, devised himself a science quiz. “The center of the Earth is very hot [true/false].” That sort of thing.

He gave the quiz’s score a super impressive name: “Science Comprehension,” as if top marks indicated you comprehended all of science instead of showing you answered correctly questions which wouldn’t stump the kids on Are You Smarter Than A Fifth Grader?

Kahan inflicted the quiz on a bunch of people and asked them things like whether they suffered from “religiosity”, had been awarded a “degree”, or were members of the Tea Party.

Lo, slightly more of those awarded a “degree”, a certificate indicating four or more years of expensive education in all things science, scored better than those without a degree. This shows that spending all that money doesn’t buy you much. (Alternate explanation: Kahan over-sampled “Studies” majors.)

Knowing whether a person suffered from “religiosity” meant nothing to knowing how a person would score on the quiz. This will chagrin those who hoped that belief if God precluded knowing that “Electrons are smaller than atoms.”

Perhaps worst of all, knowing whether a person was a Republican or Democrat, or whether a person joined the Tea Party or not, meant almost nothing to knowing how a person would score on the quiz. If anything, ever-so-slightly more Tea Party members knew that “the Earth [goes] around the sun” than non-Tea Party members.

Kahan was shocked by his discovery:

I’ve got to confess, though, I found this result surprising. As I pushed the button to run the analysis on my computer, I fully expected I’d be shown a modest negative correlation between identifying with the Tea Party and science comprehension.

But then again, I don’t know a single person who identifies with the Tea Party. All my impressions come from watching cable tv — & I don’t watch Fox News very often — and reading the “paper” (New York Times daily, plus a variety of politics-focused internet sites like Huffington Post & Politico).

I’m a little embarrassed, but mainly I’m just glad that I no longer hold this particular mistaken view.

This admission was manly—and extremely rare. But he (almost) spoiled it by running as quickly from it as he could, lest his colleagues think him insane, by immediately calling Tea Party members immoral brutes.

Strange that he came to that conclusion based on no direct observational evidence. Not very scientific behavior, that.

This isn’t Kahan’s first foray into “science comprehension.” He and several others were responsible for media-noted “The polarizing impact of science literacy and numeracy on perceived climate change risks.” (See notes below.)

Much the same thing: small trivia quiz matched with iffy political affiliation questions. Then an enormous and creaky statistical apparatus erected on top of these thin reeds with the purpose of teasing out infinitesimal differences. My God! Did they go on and on and on and then on some more? Did they ever. All completely worthless.

These kinds of “studies” come out with depressing regularity. By “these kinds” I mean those concocted by professors who have found ways to escape teaching by engaging in “research.” They gather in the nearest Starbucks (faculty lounges are passé), think of questions which they convince each other are profound and which support a fashionable theory, call the battery of questions an “instrument”, give it to a few students, then analyze, analyze, analyze.

Seriously, now. Just how far can you go with a question like “Lasers work by focusing sound waves [true/false]”? Could you tell how the country should best be run by gleaning insights from a survey which showed that only 69% of respondents knew the right answer? Could you tie the way a person answered that to some obscure psychological, jargon-laden theory? How plausible is it that most people’s politics and religious beliefs would cause people to answer correctly or incorrectly this particular question?

Why not require people to know what happens when a teaspoon of Seaborgium is mixed with a pinch of Livermorium as a prerequisite for marriage? How about insisting people know which has more mass, a neutrino or top quark, before allowing them to purchase soda pops larger than sixteen ounces?

Or what about ensuring a man can derive the fundamental equations of radiative transfer before he’s allowed to create a quiz which supposedly defines “science comprehension”?

It’s all so silly.

—————————————————————————————

Thanks to the many, many readers who alerted me to this study.

Incidentally, we met Dan Kahn before: Confirmation Bias Happens To The Other Guy, and Kahan’s Science Literacy Paper.


15 Comments

  1. Scotian

    Good one Briggs, and how many think that you just made up those elements? If you every find enough to mix them, let me know. The problem here is that Dan suffers from lawism.

  2. An Engineer

    I really enjoy this site. I read a lot about Dan Kahn; I even took the NSF Science Literacy test and got a perfect score. I conclude: there is no relationship between scientific acuity, political affinity, and faith. Science is a process of discovering new knowledge. Politics is a process of deciding what a group of people can or cannot do. Faith is a belief. I don’t see the value of attempting to predict (poorly) how the knowledge of science informs on political viewpoint as modified by a faith prospective. Professors who teach well create value. Professors who discover how to mass produce carbon nanotubes through research create value. Professors who worry about the public’s perception of science and the political affinity and religious belief are lawyers. Lots of words, no value, and on reflection a lot of politicians are lawyers….

  3. max

    Scotian,

    I was going to say that the question is wrong, the life-times of the two elements are so short that although you could add a pinch of Livermorium into a teaspoon of Seaborgium, you would wind up mixing a pinch of Flerovium into a teaspoon of a Rutherfordium/Seaborgium mixture.

  4. Nullius in Verba

    “I don’t see the value of attempting to predict (poorly) how the knowledge of science informs on political viewpoint as modified by a faith prospective.”

    That was the point. Dan is trying to stop people doing exactly that, by pointing out that there *is* no material relationship. A correlation r of 0.05 means an r^2 of 0.0025, which means about a quarter of 1% of the variance can be ‘explained’ by Tea Party membership. It’s utterly trivial.

    Unfortunately, in trying to counter the expected reaction of liberals to finding there was such a small but borderline significant difference between liberals and conservatives, he said in effect that the Tea Party were better at science, which all the talking heads immediately picked up on. He was a bit annoyed that they got it so exactly backwards – his intended point was that they’re *not*.

    There are distinctive differences between the parties in what bits of science they believe, but this is *not* because one party or another is better educated about science. It’s because that’s what they believe *politically*. Liberals believe in global warming because they’re *liberals*, not because they’re any better (or worse) at science. In fact, the more they know about science, the more quintessentially *liberal* in their beliefs about science they become.

    Until you understand the problem, you have little chance of finding a solution. But whatever.
    He’s just doing his job.

  5. Don Jackson

    It’s the pseudo-science that rankles…

  6. Don Jackson

    But this is well worth a read!

  7. max

    M. Jackson

    Nice link, and thanks for it. A typo in it gave me a new word for certain types of social science studies, “comfortbabel”. The article in fact spends some time on how there is a strong tendency to overvalue comfortbabel which make one feel comfortable with one’s biases.

  8. Don Jackson

    I like it! :)Thanks, max.

  9. Alan D McIntire

    Maybe the test should have been on “numeracy” rather than science trivia.
    At least the tea partiers are aware that budgeting more spending than you are taking in makes you wind up like Detroit, Stockton CA, Vallejo CA, or the Weimar Republic

  10. Oh, boy! The Pauline Kael Fallacy in operation! A classic.

    JJB

  11. Willis Eschenbach

    It’s even more bizarre than that. I tried to engage with Kahan on his web site. I kept getting distracted by a commenter making odd comments that almost made sense but didn’t. Here was the first comment from this strange commenter:

    @Willis:

    Sorry for failing to introduce myself properly. I am dmk38. I operate the site and sometimes hop into the discussions. As you can see, I’m not so good at expressing myself. I am hoping another discussant who does — or thinks he or she does — understand what I was trying to say will reformulate my queries.

    Thanks, & carry on!

    I clicked on the “I am dmk38” link … and came away more confused. I asked Dan Kahan what was going on with the strange commenter. Dan Kahan said it would all become clear after while, viz:

    @Willis:

    This has certainly become a very strange, Alice-in-Wonderland comment thread. I propose that we all climb out of the rabbit hole & reconvene in the comments section for the next post. Things will all make sense there, I’m sure.

    Dan Kahan

    Imagine my surprise when, after more faffing about, I found out that dmk is a sock-puppet for Kahan himself! I was blown away.

    I’ve never heard of such a thing, either before or since. The guy is operating a sock-puppet on his own website … can you say bizarre?

    I wrote him off from that moment on. When someone finally took pity on me and told me that dmk38 was a sock-puppet for Dan Kahan, I wrote:

    Seriously? You’re telling me that “dmk38” is nothing but a sock puppet for Dan Kahan? He’s appearing as a sock puppet on his own website? That’s astonishing.

    Well, Mr. Kahan, I hate to admit it, but you’re good. It sure worked on me. I was fooled. I fell for the con. My only glimmer of the truth was I thought it might be Joshua’s sock puppet, but nooo … color me embarrassed.

    You may recall upstream that I said that the trust between the climate scientists and the public had been broken because their leading lights had lied to the public and deceived them.

    And I said that folks hate to be conned. I know I do. And that once we’ve been conned, the trust is gone.

    So, you’ve successfully destroyed my trust.

    I’ve seen nothing since to change that view.

    All the best, Matt, keep up the good work.

    w.

  12. Willis Eschenbach

    Matt? Typing too fast. I meant William of Briggs of course.

    w.

  13. Mike B.

    Kahan isn’t even the worst offender of this kind of stuff. That would be this guy:

    http://woods.stanford.edu/about/woods-faculty/jon-krosnick

    Strange bit of irony here, I actually met this guy when we were 11. And all I can say is that if you’re an insufferable 11 year old, you’re like to make an insufferable adult.

Leave a Reply

Your email address will not be published. Required fields are marked *