Vengeance Is Mine
The actual title of the Live Science press release was “Believers Leave Punishment to Powerful God,” a story which opens with the memorable words:
Believing in an involved, morally active God makes people less likely to punish others for rule-breaking, new research finds.
Which I hope you agree is equivalent to saying that atheists are less forgiving, less compassionate, less merciful, and—oh, let’s just say it: they are worse people. Now don’t get mad at me. This is research, complete with wee p-values.
But then maybe this summary is too telegraphic. Because the very same research that proves that atheists are more bloodthirsty than theists also proves “that religious belief in general makes people more likely to punish wrongdoers — probably because such punishment is a way to strengthen the community as a whole” (emphasis mine).
In other words, theists are less forgiving, less compassionate, less merciful, and just plain worse people than more enlightened atheists. Except when they aren’t and when their roles are reversed. The press release explains the conundrum thusly: “In other words, religion may introduce two conflicting impulses: Punish others for their transgressions, or leave it to the Lord.”
This, friends, is the power of statistics, a field of science which, given the routine ease with which two opposite conclusions are simultaneously proved, we may now officially dub Orwellian Analytics.
Research Shows…
The paper is “Outsourcing punishment to God: beliefs in divine control reduce earthly punishment” by Kristi Laurin, and three others, and published on-line in the Proceedings of the Royal Society B.
After a lengthy introduction arguing that all morality (except presumably the morals of the authors) can be reduced to urges induced by evolutionary “pressures,” and defining something called “altruistic punishment”, the authors describe how they gathered small pools of WEIRD people (i.e. undergraduates) and had them play games. The results from these games told the authors all they needed to know about who enjoys punishment more. Incidentally, about the punishment, they said this:
Prior to effective and reliable secular institutions for punishment, large-scale societies depended on individuals engaging in ‘altruistic punishment’—bearing the costs of punishment individually, for the benefit of society.
And did you know that “According to theory”—are you ready?—“Though administering punishment benefits society as a whole, it has immediate costs for punishers themselves.” Who knew?
Experiment one corralled “Twenty undergraduates” who “participated in exchange for course credit.” That’s one more than nineteen, friends. The supplementary data (which is mysteriously left out of the main article, but which is linked there) shows that these participants contained 8 whites and 9 Asians, with 1 black and 1 Arabic left over; 10 Christians, 1 Buddhist, 1 Hindu, 1 Muslim, 1 “Other”, and 6 Atheists. The authors claim to have “measured participants’ belief in powerful, intervening Gods, and their general religiosity.” Which makes you wonder how they classed the Buddhist and “Other.” No word on the breakdown of how participants answered the “religiosity” question.
Ah, skip it; because the next is more fascinating. “We then employed the 3PPG–an economic game commonly used to measure altruistic punishment.” The words which struck yours truly were “commonly used.” It must be common, because there isn’t word one in the paper or supplementary material of what this creation is. But I can reveal to you it is the “Third-Party Punishment Game,” a frivolity invented by academics designed to flummox undergraduate participants in studies like this. About that, more another day.
The “game” runs so (sorry for the length, but do read it):
player A receives 20 dollars, and must share that money between herself and player B in two-dollar increments, without input from player B. In the second stage, player C [who presumably knows what A did], who has received 10 dollars, can spend some or all of that money to reduce player A’s final payout: For every dollar that player C spends, player A loses three dollars. Player A’s behaviour does not affect player C, all players are anonymous and expect no further interactions, and punishing player A costs player C money. People treat sharing money evenly between players A and B as the (cooperative) norm; thus, player C’s willingness to punish player A for selfishly violating this norm can be taken as an index of altruistic punishment of non-cooperators.
In other words, Player C looks at how much A gave B. If C thinks this too low, C sacrifices some of his own money to reduce the amount A kept. But A and C got the money for free and since these are students we do not know if A actually knew B in real life, or if C knew either. For example, if I as A and Uncle Mike as B and Ye Olde Statistician as C were to play this game, I would split the money with Uncle Mike and Ye Olde Statistician would go along. This is because we were pals before the game commenced. But if we were enemies, something entirely different would occur. The authors never mention if they look for these kinds of effects in this or in any experiment. Leave finding flaws and contrary evidence for others.
But never mind, because C giving up some of his play money is scarcely the same thing as C desiring that a child rapist be tossed in jail to rot, even though C knows that the cost of the rapist’s cell will be taken from his wallet. But C in real life hardly knows even that. C knows that he pays taxes and that some of his taxes go to prison upkeep, but those taxes also go to pay for the fuel to ferry the president around on Air Force One from fund raiser to fund raiser. That is, most of us Cs don’t think that ponying up taxes is altogether altruistic.
The authors are mute on this objection, too.
Enter The P-value
We regressed participants’ levels of altruistic punishment [amounts of money] on their God beliefs and their religiosity (both centred around 0) simultaneously…participants who believed more strongly in a powerful, intervening God reported less punishment of non-cooperators, β = -0.58, t(17) = 2.22, p = 0.04; whereas more religious participants showed a trend towards reporting greater punishment, β = 0.33, t(17) = 1.67, p = 0.11.
And there it is. Theists reported less punishment and more punishment. Except that the p-value for the “more punishment” isn’t small enough to excite. (And a linear regression is at best an approximation here.) The authors also discovered “more religious people tended to believe in powerful, controlling Gods.” The correlation wasn’t perfect, but neither should it be when you mix Buddhists and Christians. Let’s don’t forget that this regression model only included 6 atheists for its contrast.
The really good news is that “Given the strong correlation between religiosity and conservatism (r = 0.52), we conducted an additional analysis including conservatism in the regression. Results are reported in table 1; we found no evidence that conservatism explains the religion–punishment association.”
Sorry, Chris Mooney.
The authors did four more studies, all similar to this one, but with increasingly complicated regression models (lots of interactions, strong hints of data snooping, etc.). The findings don’t change much. In their conclusion, however, they include these strange words: “In our research, we found it necessary to remind participants of their beliefs for these beliefs to influence their decisions.” This sounds like coaching, a way to induce results the authors expected.
The real lesson to us is how this complex mass of data is squeezed into the terse, and misleading, headline.
HT HotAir.
“Let’s don’t forget that this regression model only included 6 atheists for its contrast”
Let’s not?
Scientism @ its best. I like it when silly papers disagree with other silly papers.
Luis, do they really disagree? In what sense? Certainly they don’t absolutely disagree. But is relative disagreement really disagreement at all. To what extent is punishment really punishment? It’s all so relative! I need another drink.
PS – I see you haven’t Popperly tested the gravity thing yet. Shut up and calculate!
Yes, Mike, you absolutely need another drink ;).
Withholding (effectively) play money is punishment? Scary. I’m all for the corporal variety like whippings with a wet noodle. Far more satisfying.
The essay here misrepresents the paper’s/author’s conclusions, specifically by presenting as conclusive findings that were actually observed to be conditional AND paradoxical for reasons the authors did not understand.
Of course, such a misrepresentation is crucial to the essay…otherwise there’d be not much of anything to say.
Below are selective quotes from the LiveScience summary, where they, at least, understood the actual limits of the conclusions from the authors and the uncertainties identified are readily apparent [bracketed portions are added to emphasize the plain meaning of key words]:
In other words, religion may [or may not] introduce two conflicting impulses: Punish others for their transgressions, or leave it to the Lord.
The findings, published Tuesday (May 22) in the journal Proceedings of the Royal Society B, suggested [but were NOT conclusive] that when people believed in a powerful, involved God and when they were reminded of that belief, they were less likely to punish Player A.
In a similar study, participants reminded of their belief in a powerful God were less willing to support state-sponsored punishment for a white-collar criminal.
When researchers looked at religious beliefs alone, however, they found that people with stronger beliefs were more likely to punish others after being reminded of those beliefs.
[in trying to understand the findings the authors were at a loss, but did propose some highly speculative possibilities, which could be a basis for further research]
It may [or may not] be that…
It’s also possible [but not necessarily certain] that…
How this push-and-pull between punishment and mercy plays out on a daily basis is still unknown.
.
The problem increasingly apparent with this blog is that research of the type presented & critiqued is always cherrypicked to be of the most formative studies into some area — the results of which are always, ALWAYS, necessarily tentative/hypothetical. These studies are building blocks for a foundation of further study, but critiqued as if they are the end-all of a long-term comprehensive research effort.
Which brings to mind the old proverb/saying: ‘Any fool can complain, and that’s what fools usually do.”
Here, the blogger has nurtured a noticable penchant for complaining. Constantly (the ability to present a positive, presumably objective, statistical/mathematical precept/principle, etc. actually presented objectively is fast fading….giving way to a prominent inability to de-couple the learned transfer of knowledge from a complaining putdown of someone else/others — and those putdowns show a considered effort). And then, occassionaly, comes the gripe about having difficulty finding/keeping gainful employment…as if prospective employers don’t notice the foolish pattern.
This increasingly refined style, by the way, also does not comport with the ‘do unto others’ fundamental precept of the faith espoused by the blogger. Which just illustrates Relitavism really is valid–at least as indicated by what people actually do irrespective of what values they claim, or even believe, they actually have…but this observation fits in a preceding essay.
“Given the strong correlation between religiosity and conservatism (r = 0.52), we conducted an additional analysis including conservatism in the regression.”
Wait a minute. Doesn’t that introduce multicollinearity? What did they do about the variance inflation factors? I’ve seen multicollinearity flip a correlation from one sign to the other.
And don’t get me started on linear regression.
Ken,
Ghost stories. There are a lot of them. Most tales of spooks are necessarily tentative, searching; any one of them is not a complete description of the true science of ectoplasm. But surely the bulk of these reports, their sheer mass—nearly countless in number—means that ghosts must be real. If they were not we would not read of so many accounts of them. Right?
Also, I am very happy to report that I am gainfully employed—and this will disappoint you—I am so employed because of my writings on this blog.
(Incidentally, you’ll note that the authors did not in fact address any of the criticisms I advanced above, which I would say indicates the authors did not fully understand the limitations of their work.)
participants who believed more strongly in a powerful, intervening God reported less punishment of non-cooperators; whereas more religious participants showed a trend towards reporting greater punishment.
I am confused. How are they distinguishing “beleiving in a powerful and intervening God”, from religious?
Ken, are you seriously defending these kinds of silly speculations?
I wouldn’t mind them as such, btw. I am all for all kinds of wild speculations. But then they commit the grave mistake of binding the p-value to them and declare them scientific and thus true. Absolutism at its worst I’m afraid ;).
I hate “scientists” like this. I really do.
Stop it Matt; just STOP IT!
When I started reading your blog I knew NOTHING of statistics, but assumed that actual scientists DID know statistics and that when I read a headline that said ‘Studies show something—ANYTHING’ that some scientist had actually collected a bunch of scientific data, touched it with the magic wand of statistics, and POOF!, the something—ANYTHING (in most cases, interestingly, a something—ANYTHING near and dear to the hearts of liberals) trumpeted by the reference headline was emitted from the data.
Now? Now I STILL know nothing of statistics but after reading your reports on actual data, collected by actual scientists, who actually touched it with the magic wand of statistics and produced the actual studies trumpeted by the actual headlines, my first reaction when I see something like ‘Study shows, based on 1.825e12 data samples (5e9 years X 365 data points/year) that the sun ALWAYS rises in the east.’, is to think ‘Tomorrow morning around 5AM I need to get up, make a pot of coffee, go out on my west-facing back porch, and watch history in the making’.
This is probably not the right forum for this kind of quesiton, but I seriously don’t understand the 3PPG. Why would player A give any money at all to player B? And why would C ever punish A? If I’m A, I’m giving away the bare minimum, and if C punishes me for that then I’m flabbergasted. They gave ME the money! It’s mine!
Actually, this probably tests your political beliefs. Communists would punish A for not sharing, and intelligent people would not. (bias? what bias?)
Heck, Adam, calling C’s activity “punishment” is already a verbal stretch. And calling 3PPG an “instrument” that “measures” anything, as if it were an interferometer or a titration, is like calling the whole exercise “scientific.”
All to reach a “conclusion” that:
religion may or may not introduce two conflicting impulses: Punish others for their transgressions, or leave it to the Lord.
which is not actually a “conclusion.”
Are you sure that all this passing of money around is not a proxy for taxation?
Luis, I’m quite happy to join in the criticism of taking this stuff too seriously, for many reasons, but I think Ken’s right to identify misrepresentation here. The parts of this post that made you think the results were diagreeing with themselves were just silly. The ‘researchers’ split the participants in two different ways – why in the world would you throw the same label on the supposedly more religious and those with a supposedly stronger belief in a powerful intervening God, let alone expect that the average behaviour will be the same in the two groups?
Briggs also confuses us by telling us to read the 3PPG description, including the information that players are anonymous, and then describing a situation where he knows he’s playing with his friends. Not that that takes away from the other criticisms, particularly the complaint about the shoehorning of data into a headline that is easy to take the wrong way. No doubt it is this almost universal practice which Briggs is trying to highlight when building on the misrepresentation throughout the post.
Having looked a few times into what’s getting published in the Proceedings of the Royal Society, I’m starting to wonder if they feel they need to compete against New Scientist for attention?
They did a study with only 20 subjects and they actually wrote it up? Call me when n=2,000.
“There is something fascinating about science. One gets such wholesale returns of conjecture out of such a trifling investment of fact.” — Mark Twain
Jonathan D,
Right. They do say, “Player A’s behaviour does not affect player C, all players are anonymous and expect no further interactions, and punishing player A costs player C money.” And while it might mean that these young adults have never ever seen each other before, I took it to mean, since the authors provide no information on how they solicited the participants, that “anonymous” meant that information was not exchanged between A and B and C during the game (as they explained). Yet notice the next paragraph after the quoted sentence, which opens, “Participants all imagined being player C, and indicated…” which tells us that instead of “no further interactions” the participants played this game several times; that is, they interacted.
Of course, it is possible to imagine that they took these 6 atheists and 14 theists, first proved and had them swear that none had ever seen the others before, and then locked them in separate rooms and allowed no communication between them while the experiment progressed, but since the authors tell us nothing about how they designed their experiment, this interpretation is a stretch. And don’t let’s forget that the reward for participation was class credit. Meaning that they were in the same class (perhaps spread across sections). Which means they might have not have been pals—incidentally, Uncle Mike, Ye Olde Statistician and I have never met—but they probably knew of the others. And they also did other experiments, also offering class credit. Were the same students in more than one experiment?
About the interaction: there’s lots of evidence that when people play “tit-for-tat” the results are different if they play it once or they play repeatedly, i.e. expecting to play further iterations. Behavior of the players change. So we are right to suspect behavior change in this experiment
The best way for us to confirm this—and any—study is to use the model they created and put it into a predictive form. Predict what 6 new atheists and 14 new theists (of the same breakdown as in the experiment) will do. If that prediction matches what happens, then I’ll believe it. Take a look at the regression model and tell me your honest opinion of the likelihood of this happening.
Extrapolating from this 3-PPG instrument is what pegs the smell-test meter for me. Why not just develop a short questionnaire with several committed adherents to various religions as well as atheists? Include items that gauge a level of commitment to faith or lack thereof and items that tell stories of malfeasance and ask about degrees of appropriate punishment. Simple. Clean. Certainly as trustworthy as any economic game playing.
Oh, and conduct the survey at the mall where the WEIRD quotient is likely to be more realistic.
Pingback: Friday Highlights | Pseudo-Polymath
Pingback: Stones Cry Out - If they keep silent… » Things Heard: e222v5