Update 13 August 2019 The paper has been retracted!
Welcome Guardian readers. You should understand this poor study is one of thousands, all of which make similar mistakes. See the comment at the bottom of this article for more resources.
Heard about that scientific study which scientifically shows non-religious kids are scientifically more altruistic than unscientific religious kids? The Guardian summarized it thusly: “Religious children are meaner than their secular counterparts, study finds: Religious belief appears to have negative influence on children’s altruism and judgments of others’ actions even as parents see them as ‘more empathetic’.”
Scientifically speaking, this is crap. Here’s why.
The scientific science “study” The Guardian cites is the peer-reviewed article “The Negative Association between Religiousness and Children’s Altruism across the World” in the journal Current Biology by Jean Decety and a bunch of others. Biology? Never mind.
Authors gathered kids, 5 to 12, from the US, Canada, China, Jordan, Turkey and South Africa and asked them a bunch of scientific questions, scientifically quantified those questions, produced scientific statistics, and then made scientific propositions about the whole of the human race. Say, are there differences in behavior between 5- and 12-year-olds? That doesn’t sound like a scientific question, so never mind.
Here’s how to you can replicate their study at home. First, define altruism. Go on, I’ll wait.
Have a definition in mind? I’m sure it’s correct and matches everybody else’s definition in precise detail, details like no-greater-love, supreme sacrifice, kindness, patience, love, and so on, right? Well, maybe not, but never mind. Instead, think about how you would quantify your definition. Quantification makes your definition scientific. Science means unquestionable truth.
Was your answer about quantification the “Dictator game”? Like this (from the Supplementary description)?:
[C]hildren were shown a set of 30 stickers and told to choose their 10 favorite. They were then told “these stickers are yours to keep.” Children were instructed that the experimenter did not have the time to play this game with all of the children in the school, so not everyone would be able to receive stickers. Children were finally shown a set of envelopes and informed that they could give some of their stickers to another child who would not be able to play this game by putting them in one envelope and they could put the stickers they wanted to keep in the other envelope. Experimenters turned around during the child’s choice and children were instructed to inform the experimenter when they were finished. Altruism was calculated as the number of stickers shared out of 10.
Yes, this scientifically captures every possible nuance of the scientific concept of altruism, doesn’t it? Science science science science. Science. It must be science! Scientists wrote this, peer scientists reviewed it, and scientists nod sagely when reading it.
Now define “religiosity” for kids. I’ll wait again.
Have it? Ha ha! That was a trick question. The authors never assessed the “religiosity” of kids; they did it for the kids’ “caregivers” instead. How? The authors asked parents to name their religion. They also asked parents questions like “How often do you experience the ‘divine’ in your everyday life?” They took pseudo-quantified answers from these and combined them scientifically with a quantification of religious attendance and derived a complete scientific quantification of “religiosity.” This was assigned to each kid in the study.
After that, “Children completed a moral sensitivity task programmed in E-prime 2.0 and presented on ASUS T101MT Touchscreen computers…” My goodness! How scientific! An ASUS T101MT! Just think how dramatically the results might have changed had they used an ASUS ROG G752! Or an ACER C910-C37P!
You know what happened next. Wee p-values through the terrible abuse of regression on the pseudo-quantified answers. A picture showing one of these is at the top. Notice the wee p-values? That makes the findings scientific.
All those dots are the answers to the pseudo-quantifications for each kid. The flat surface is the regression (expressing this and nothing else: the change in the central parameter of a normal distribution representing uncertainty in “altruism”; did you think it was something different?). Notice almost none of the dots are near this flat surface? That means this model has no real predictive value.
Which, scientifically speaking, means this study is crap. And where I use “science” in the old-fashioned, pre-government-grant way.
Finally, no paper would be complete without wild, over-reaching theorizing about cause. The authors say their findings “contradict the commonsense and popular assumption that children from religious households are more altruistic and kind towards others”. Idiots everywhere are taking this literally. I’m too tired to make a joke about science. You do it.
This study is so bad that it’s good. I mean, it stinks to high heaven; nearly everything is wrong with it, start to finish. Yet it’s good because it takes so much effort to dissect, and the effort reduces the critic to such a sputtering mess that the criticism is bound to sound like an old fart yelling at the kids to get off the lawn.
Why I am Right Update P-values are rotten evidence for anything (click here to learn), (2) Regression is deeply flawed and not what you think (click here, here, or here to learn), (3) Probability models do not prove cause (click here, (4) Asinine studies like this are common (click here) or here). And don’t forget that altruism was not measured, but that kids sticking stickers in envelopes was. How much influence did the researchers have, especially with the younger kids? I mean, did kids stick stickers because they wanted to prove to the whitecoat they were compliant or because they wanted to be liked or because they wanted to share? Altruism forsooth!
And most importantly, don’t throw a temper tantrum and (for you men) cry like a little girl or say that I am a bad person. Show—exactly—where I am wrong and the authors right. If you cannot do this, keep your mouth shut. (I am too tired and too surly this morning to put this more politely).
Thanks to the many readers who pointed me to this study. For more resources about why this paper stinks, see my comment to Doubting Thomas below, or click Classic Posts above.