Statistics

Can fMRI Predict Who Believes In God? Part III

Read Part I, Part II, Part III, Part IV, Part V, Part VI, Part VII

It is finally time to describe the experiment itself. People were stuck inside an fMRI and asked questions to which they could answer, by clicking a button, “‘true’ (belief), ‘false’ (disbelief), and ‘undecidable’ (uncertainty).” As to the questions, or “stimuli” (which sounds more impressive), they:

were drawn from two categories, religious and nonreligious. All statements were designed to be judged easily as “true” or “false” (the response of “undecidable,” while available to subjects, was not expected).

Within each category, we attempted to balance the stimuli with respect to semantic structure and content. Strict balancing across categories was not possible, however, as the two categories differ with respect to content, in principle…

Christians and Nonbelievers were expected to respond identically to nonreligious stimuli and to be discordant for all religious trials.

The time is took to answer is the key variable.

The questions, or “stimuli”, were created by Harris and his staff and “were tested to ensure that they would function appropriately in our experiment. For this purpose, we created several sets of candidate stimuli and solicited responses from the nonbelievers and Christians on the Internet.” The Internet? They designed their questions via self-selection of atheists and believers on the Internet?

What could go wrong?

It is telling that, from the Internet, “For each statement the number of respondents averaged around 5000, 80-90% of whom were nonbelievers.” This sample is the inverse of the actual population, incidentally. Bias? What bias? Harris kept only those questions that elicited the strongest response from those who self-identified as Christians or “nonbelievers” (i.e. non-Christians).

In the actual experiment, after the students came out of the machines, “subjects were asked to review their recorded responses to all statements to ensure that they reflected their actual beliefs at the time of scanning. Erroneous responses, responses of ‘undecided,’ or those statements which, upon debriefing, could not be clearly judged by subjects to be ‘true’ or ‘false’ were excluded from subsequent data analysis.”

Whoa, big fella! Let’s re-read that together: people were “debriefed” and then some of their data was tossed out. Thrown away. Not used. It was the difficult questions that weren’t used, too. Since the “neutral” questions were matched with a religious question, if the neutral was mistakenly answered or was undecided, was the paired religious questions tossed, too? Or were the paired neutrals tossed when the religious question was disqualified? Just what was thrown away? How many questions? We never learn. Could this extraordinary behavior have introduced a bias? Yes.

It is well to examine some (not all) of the “stimuli” (the full list of sixty-one question blocks is here). Number 61:

The existence of God is the best explanation for the beauty of the universe.
The existence of God is an inadequate explanation for the beauty of the universe.
The existence of the mythical fountain of youth has not been established by science.
The existence of the mythical fountain of youth has been established beyond any doubt.

Harris assumes that all Christians would answer “true” to the first and “false” to the second, while all non-believers would answer “false” for the first and “true” to the second. He also assumes that everybody would answer “true” for the third and “false” for the fourth question.

Deists might answer “false” for the first question, yet still consider themselves Christians. We expect that non-believers would answer “true” to the second question, but a non-believer might answer “true” for the first, especially if he is agnostic or answered the questions conditionally (“Even though I don’t believe it, if God existed, that could account for the beauty of the universe”).

It is also more than conceivable that college students might answer “true” or “false” to the latter questions, regardless of their religious belief. Finally, to call the universe “beautiful” reveals a bias towards scientism. Would Harris say that the recent tsunamis in Indonesia and Japan were beautiful? Is it “beautiful” that humans age and die, in many cases miserably? Those students with a matching bias towards scientism might answer at different rates than those who viewed nature red in tooth and claw.

Number 59:

The gift of immortality has been revealed by God in the Bible.
The gift of immortality promised in the Bible is unlikely to be real.
Giving gifts to friends can be a source of joy.
Giving gifts is never appropriate between family members.

Again, Christians were expected to answer “true” and “false” to questions one and two, non-believers the reverse. Some who call themselves Christians do not believe in immortality (especially those in older Protestant sects), or they do not believe in it for all (Jehovah’s Witnesses, for example). Or if they do believe in immortality, not all would agree that this fact has been revealed in the Bible. Catholics, for instance, do not have the same history of reading and swearing by the Bible as Protestants, and so might answer this question differently.

This question is like many others, in that it poses a theological conundrum that might cause a Christian a moment or two to ponder over the best response. Recall that timeliness in answering is the outcome. There is an asymmetry, because the non-believer (which is here defined as non-Christian) would not suffer the same mental friction. It is also possible a non-believer can believe in immortality, but not the Christian version of it, and so answer “false” to the second question.

Now it is important to also look at the “neutral” questions, because these are used as the baseline from which the answers to the first two questions were taken. One can imagine dozens of scenarios where people do not answer as Harris expects—it is so easy to think of these exceptions that it is pointless to delineate any. But ignore that and think about how these “neutral” questions are correlated (in the plain English sense of that word) with Christian belief.

It is possible that Christians would answer “true” more often or more quickly on question three (“Giving gifts to friends can be a source of joy”), or “false” more often of more quickly, than non-believers on question four (“Giving gifts is never appropriate between family members”)? If so, then this question is harmful and misleading in Harris’s experiment since it will bias the results.

Read Part I, Part II, Part III, Part IV, Part V, Part VI, Part VII

Categories: Statistics

7 replies »

  1. The problem that Briggs raises when he wonders if there are differences among Christians in what they believe is huge. For some Christians the question of ‘true’ or ‘false’ misses the point about their faith.

    Religious texts can be read and understood in a number of different ways. So for example, at least some Christians could answer false to the first two questions in section 59 and others could answer true to the same two questions. One group could think to themselves that it is not question of ‘true’ or ‘false’ because they base their belief on faith. The other group could answer ‘true’ to the same two questions because they have a particular interpretation about what was revealed.

    Question 61 has a different problem. One could be a deist and argue that arguments about ‘the beauty of the world’ have nothing to do with the existence or non-existence of a god. Atheists and agnostics could also answer ‘false’ to both questions for slightly different reasons. An atheist could answer ‘false’ to the first and second question arguing that since ‘god doesn’t exist’ the concept can not explain anything about beauty. An agnostic might say both statements are false since there isn’t enough evidence one way or another to go with ‘true’ so it’s false. Or an agnostic could opt for undecided.

    Another huge issue is discarding responses that don’t fit within the supposed experimental framework. That process will always introduce a huge positive bias to the experimental results. It’s more likely that careful examination of the responses to be discarded will reveal the profound confusions in the questions.

  2. “The gift of immortality has been revealed by God in the Bible” should be true, if only implied, for anyone who has read the New Testament of the Bible. The truth of the statement has little to do with belief in the veracity of the revelation.

    If someone answers FALSE to the previous question I would think they would have to answer FALSE to “The gift of immortality promised in the Bible is unlikely to be real” as the first question asks if immortality was revealed. If it wasn’t revealed then it would seem it wasn’t promised either.

    The possibility of multiple interpretation is one of the reasons determination of mind states by direct questioning is a bad idea. It’s also a problem in opinion polls. It is very difficult to construct unambiguous questions. Looking at the question set it’s also not hard to imagine the test subjects received clues about the subject of the experiment. This in itself could have seriously bias the results. People are largely accommodating and will supply answers they think the experimenter wants.

    This experiment was based on questionable premises from the start. The authors acknowledged this when they said One cannot reliably infer the presence of a mental state on the basis of brain data alone, unless the brain regions in question are known to be truly selective for a single state of mind. and then tried to recover with Nevertheless, our results appear to make at least provisional sense of the emotional tone of belief. So, where’s the study that shows the unless part is true? The results would be worthless even if the protocol weren’t botched.

  3. This expirment assumes that all believers have an orthodox view of religion. Some churches encourage questions and doubt. Believers may have a different responses than the expirementer may have anticipated.

  4. Others have touched on it, but IMO the “stimuli” reveal a narrow and shallow grasp of “religious beliefs”. To me they say more about the experimenters’ mindsets than realized.

    The latest motto of Neo-Scientists. “Disregard unexpected test results”.

  5. If this paper got through peer review, it reveals more about the value of peer review than the subject of the paper itself.
    Mr Briggs, why are you spending time on this?

Leave a Reply

Your email address will not be published. Required fields are marked *