William M. Briggs

I have been asked to write this review by a party who wishes to remain anonymous because of the fear of reprisal. See this article for background. Also this.

Dahn yoga might not be of interest to you, but this review is larger than that. It will show you how easy it is to publish material in a well known journal that is poor at best. Civilians are often shocked to discover that peer-review is only a weak indicator of correctness. This article removes some of that mystique.

This article is long, but has to be.

Study Abstract

Sung Lee, an internist, while at the Weill Cornell Medical School, conducted an experiment of Dahn yoga. He published the results in the paper: Prospective Study of New Participants in a Community-based Mind-body Training Program. It appeared in the Journal of General Internal Medicine, 2004 July; 19(7): 760–765.

This study has been touted by Dahn sympathizers: Ilchi Lee (Dahn founder) boasts of it (pdf), TV broadcasts (Sung Lee’s picture is in the upper-right corner), as supportive research, chiropractors in Sedona, and fan sites of Dahn.

Lee went to several locations in New York City and recruited people to join a three-month Dahn (introductory) program. All of these people self-selected themselves into the program; they came seeking yoga; all were new to Dahn. All were asked, at the beginning and end of the study, a series of questions.

The main ones were from the SF-36, a standard questionnaire. The most useful SF-36 “item” is question 1: “In general, would you say your health is: Excellent (5)…Poor (1)?”¹ Another is, “Have you been a very nervous person? (6) – (1)” This “instrument” is divided into domains, such as “vitality” and “mental health,” which are simple functions of the questions. The “nervous” question is part of the mental health domain. Another is, “Have you felt downhearted and blue? (6) – (1)”

Papers which use the SF-36 rarely show the questions; they are content to report on the domains. Seeing the exact questions makes them sound far less impressive than when stated in their usual academese. For example, Lee calls the SF-36 “a validated health assessment instrument.”

Averages of the US population of each of the domains exist. The participants in the Dahn study began with scores lower than the US average: a fact which is not surprising, considering these were people who were newly arrived for exercise training.

There was no control group: all received the Dahn training. 194 started, and 171 completed the study. Three out of four were women. Five of the 171 reported an injury due to the training: it is unknown how many of the twenty-three who dropped out were injured.

From the abstract: “New participants in a community-based mind-body training program reported poor health-related quality of life at baseline and moderate improvements after 3 months of practice.” This means that several of the people had small increases in their three-month SF-36 scores.

That is, some people went from answering “A good bit of the time” to “A little of the time” on the question “Have you been a very nervous person?” And so on for some of the other questions.

From this, Lee was able to say that “Dahn worked.” Actually, the best that could be said was “Dahn didn’t cause too much harm.” Here’s why.

Specific objections

I was at Cornell at the time Lee was completing, presenting, and writing up his study. I made my objections known at that time. You must also understand that in academics “A paper is a paper”, and nearly anything can be published in some peer-reviewed journal somewhere. Because of this, the number of journals is staggering: they increase constantly.

Continue Reading »

Regular readers might recall the series of articles in which I detailed an experiment I designed and proctored to test some of Dahn Yoga’s more outlandish claims.

That series is here: I, II, III, IV, V.

Dahn Yoga leader Ilchi Lee created the Korean Institute of Brain Science, a group that claimed it could train children to “read” colors of cards inside opaque envelopes, using “Heightened Sensory Perception.” Because I had experience in these matters and because I was officemate with a physician who was also a Dahn member, I was able to run a test to see whether the extraordinary claims could be proven. They could not.

The kids knew they were failing, the audience had become mostly silent, or sat quietly talking to one another. The kids began to get up more, ate even more candy, exercised more. But no change. Most guesses were misses.

Eventually, after it became clear that nothing more was going to happen, Sung told me he was going to stop the experiment out of concern for the kids’ anxiety. The remaining trials would be marked down as misses.

Sung stepped up and announced the trial’s ending. The audience understood, and clearly felt for the kids.

One kid did 7 trials, the other two did 6 before the experiment was stopped . They were scheduled to do 12 trials each. They got 4 hits during these 19 trials, right what chance would predict: kid one got 1, kid two got 1, kid three got 2.

Recall that before the trial started, KIBS staff members were confident each kid would get at least 10 out 12 hits.

The Dahn organization has just been hit with a lawsuit claiming that recruits “are unknowingly subjected to an intensive program of psychological manipulation, indoctrination and various techniques of coercive thought reform designed to induce them to become Ilchi Lee’s disciples and devote themselves to serving him and his ‘vision.’ ”

One of the [Dahn] exercises, known as “bow training,” involved deep knee bends to the floor to a prone position and back up again, with hands raised high over their heads. Miller, who has joined the lawsuit, says once she had to do 3,000 of the exercises — “Which took about 10 hours, and we didn’t eat or drink during that time.”

“People were screaming, people were throwing up, people were running away,” Miller said. “People were rolling around, moaning, crying, wailing — there was a lot of emotional distress. We were taught that because of this bow training, we were cleaning what was blocking us, to connect to our soul.”

A video posted at YouTube shows what a Dahn training (cleansing?) session is like. (This video was posted at a Korean ExPat web site, linked here.)

Dahn followers are internet savvy. A quick survey of YouTube videos shows a throng of positive commentors accompany every pro-Dahn video. The comments are usually brief and have the feel that they are posted by the same person using different screen names, or that they were part of an organized response.

For example, this video of my old friend Sung Lee, has user “hansooryeon” simply saying “great!” This is followed by “chunjihwa4333″ saying “I like the idea!!” and by “iintress” saying “impressive.” Over 1,600 comments followed that video, a common occurrence.

I do not know enough about what a “cult” is, its precise definitions and so forth. I can say that I was concerned what the KIBS/Dahn group was doing with children. Particularly in light of the failed experiment I ran. Naturally, Dahn advocates completely dismiss that test.

But I am glad that the group is finally receiving some well deserved negative publicity.

NOW

I lost track of Sung, and had not given much thought to Dahn Yoga or KIBS in a long time. But my memory was jogged when I finally saw somebody in the window of a Dahn Yoga center, which is on my route to the dreaded F train (the center is on the second floor on the southwest corner of 66th and 3rd in Manhattan).

I did a search and found this video, which shows Dr Sung Won Lee in a conference sponsored by KIBS, held this past summer at the United Nations.

More on that conference can be found here.

Sung uses a lot of words, but says little other than that strong emotions can sometimes cause difficulties, something we have known since the men of Sumner first drank the byproducts of moistened barely. Intriguingly, he mentions that a later speaker will be Antonio Damasio, a best selling author of The Feeling of What Happens and Descartes’ Error. Damasio is a neurologist whose main interest is in consciousness, and is somewhat well known. This means that Ilchi Lee is still reaching out.

And Dahn Yoga and KIBS is going strong. Is this a good thing?

PRESSURE

I am not, in any way, an expert of cult behavior and so cannot say too much about this. But a quick search reveals stories like this one, not at all atypical. At the very least, Dahn Yoga practitioners like to put the hard sell on people to spend a lot quickly. Many more links are at Lorie Anderson’s site.

This video is typical of what you will find:

As is this video and this one.

The Rick Ross Institute is an “Internet archive of information about cults, destructive cults, controversial groups and movements.” They have a page on various Dahn Yoga activities that I highly recommend perusing.

Cult or no? I don’t know, but none of the evidence points to Dahn Yoga being an entirely benign organization.

I worry very much about the kids back in Korea who are undergoing “training” in brain respiration/education methods.

OPEN MIND

I am often told by proponents of psychic powers that “I should keep an open mind.” That if my mind was “closed” I would never be able to appreciate what they could.

I agree.

An open mind is important. This is why I design and conduct tests like the KIBS kids test. I will not dismiss somebody’s claims out of hand. If the KIBS kids test would have been a success, I would have been willing to believe that the theory behind brain respiration, now “brain education”, had validity. But the test did not work, and so, rationally, I conclude that brain respiration is yet another failed theory, that it is invalid.

I now ask those who believe in Ilchi Lee to have an open mind. To prove you have it, answer this question: What evidence will convince you that brain respiration/education is false?

To people who believe in ESP, I ask the same thing: What evidence will convince you that ESP is false?

To people who believe in any controversial theory in which the only evidence for or against it is observational: What evidence will convince you that it is false?

If you find you cannot or do not want to answer this question, then it’s your mind that is closed, it is you that is unwilling to face the truth, it is you that is stuck in old ways of thinking.

I have never yet met a True Believer who gave me an answer.

Part I, II, III, IV, V

AFTERMATH

The test was clearly a failure, the kids bombed. The only time they succeeded was when the opportunity—and temptation—for cheating was available. It was not just that they couldn’t read the colors inside the envelopes—though the KIBS people claimed they could in Korea, and all sides agreed to the experimental protocol—but they could not read the cards when all three kids were blindfolded, taking away the opportunity for cuing.

All evidence, therefore, rationally points to the simplest explanation: HSP is false and the kids cheat. They might cheat with the best of intentions, but it’s still cheating. In other words, with this evidence, it would be irrational to increase your believe that brain respiration is real. It is rational, however, to decrease your belief in it.

Randi

Before the test, I emailed James Randi, a well known expert in designing tests of psychic phenomena. Randi has for years held out the “Million Dollar Challenge”, which will pay to the individual who can demonstrate, under test conditions, any psychic ability, what is now over one million U.S.A. dollars.

My email to Randi was unsolicited (he didn’t know me, but I called him a “hero” of mine), so I only gave a sketch of my request. He did not get back to me until the test was over, at which time I emailed him again, briefly outlining what had happened. At no time did I not mean for my sketch to be a complete report of what occurred. But Randi, without my knowledge, published my email on his popular site, and added interstitial comments, taking me to task for deviating from the protocol and allowing the non-scheduled blindfold trials after the main trial had failed.

He said, “Matt, sympathetic as I am to your situation, you let the kids run away with the situation. That doesn’t happen when I get going on such a test.” Well, Randi is an old man and curmudgeonly, and certainly has had more experience than I ever will, so I accept his criticisms gracefully.

I want to stress that had Randi asked for permission to publish my email, I probably would have given it, but I would have liked the opportunity to be more explicit about what happened. I have had no further communication with Randi since, except that I mailed his educational foundation library a copy of my book (also unsolicited).

It is important also to note that Randi had written negative commentary on Ilchi Lee prior to receiving my email. I didn’t know about these posts until afterwards, but my favorite is the one explaining the $4000 cost of a small metal turtle that Dahn Yoga followers could purchase and place by the side of their bed to “increase energy flow.”

What happened next was this: some member of Dahn Yoga found Randi’s post of my email and told Dr S. Lee. Sung was furious that I could have published something without his prior knowledge. But the first time I learned that Randi had published my email was from Sung. I told him this, and also said I didn’t care that Randi had posted it, and reminded Sung of our agreement that either of us could publish what we wanted.

Sung stop talking to me and starting communicating with me via email, mostly to dispute my conclusions about the experiment’s results. I suggested to Sung that if the kids truly get as good as they say they can get, that they contact Randi to win that million bucks. This received an emphatic no.

I also suggested that, if I was right, and brain respiration was false, then KIBS was doing the kids harm by encouraging their cheating and subjecting them to stressful situations that they might not understand. This warning fell, as it’s said, on deaf ears.

Sung

A reporter, either present at the MIT trial, or who had heard about it contacted Cornell’s Public Relations department. The reporter was concerned because there were rumors that Dahn Yoga was a cult. I did not know about these accusations before the experiment.

I was subsequently contacted by Lorie Anderson, who runs this site, which compiled evidence of Dahn Yoga’s less laudatory practices.

I assured Miss Anderson, and my boss, that we ran the MIT experiment unofficially, and that we did not involve Cornell’s name except to give our affiliations for the record. Sung also assured the Public Relations department and our boss of the same
thing. This was accepted.

But it was further discovered by officials at Cornell that Sung was running and planning experimental medical trials which used Dahn Yoga as a treatment. His interactions with the Cornell hierarchy became more acrimonious and he, about two months after the MIT experiment, resigned from the Cornell faculty.

Before he left, I tried patching things up with Sung and told him I thought brain respiration was false and why didn’t he? We went back and forth, but I finally said, “Let’s boil it all down to this turtle” (mentioned above). I asked, “Do you really and truly believe it does what it says it does? If so, then you’re a true believer; if not, then you’re willing to accept negative evidence.” He started, “Well, there are things about energy…” and I stopped him. I said, “Okay, you do believe. Let’s leave it at that.”

We shook hands and he left.

I had heard he went out to Arizona to work in a new facility Ilchi Lee was building to develop brain respiration in America. As far as I know, he hasn’t contacted anybody at Cornell since he left; I haven’t heard from him either.

Next: what’s new

Part I, II, III, IV, V

THE EXPERIMENT

The test was filmed by several cameras, most from the KIBS people (I have been unable to find any videos on line). The KIBS staff itself was large, about a dozen or two. The public was not barred from filming or photographing any part of the proceedings, and many did. I did not meet many people, but I was told prominent physics and medicine professors from Harvard and MIT were invited, as were other “thought leaders”. There were about forty people not from KIBS in attendance.

There were snacks and drinks provided. I set up in a corner and tried to be be non obtrusive. From what I could gather from the chatter, the audience was anxious but expected good things. Everybody, staff included, was all smiles. Each kid was more adorable than the last, and we were all pulling for them.

Ilchi Lee gave a standard opening speech in Korean, which was simultaneously translated by Sung Lee. Sung also gave a speech explaining the history and goals of brain respiration and what was expected that night. I was introduced, as were the proctors, but none of us spoke.

Sung explained that, in practice, the kids had hit rates of 80-90% with the test envelopes during the previous two weeks. As I said before, I have no clear idea how they did these unsupervised trials. We were thus expecting each kid to guess about 10 more envelopes correctly.

Sung outlined the experimental protocol, assured the audience we would see some special that night, and the test began.

Blindfold phase

The three kids sat together at a table, but only one at a time was blindfolded. Not surprisingly, the kids got all the blindfold guesses correct. I’m pretty sure they didn’t peek down their nose, though they could have above it because they held the cards up high in the air to indicate that they were ready to announce their guess.

But they didn’t have to peek to cheat. I and two other proctors, who also had experience in magic, felt that the kids were cluing each other (the third proctor admitted the possibility but did not care to guess); which is to say they were either telling the blindfolded kid openly what color card he held, or using other signaling methods. There were certainly countless ways the kids could have communicated to each other. All in the room could see the card except the blindfolded kid. The kids were fairly close to one another and they were allowed to move about and talk when the other of them was blindfolded. I’m fairly sure on the kids and the KIBS staff spoke Korean, so there was no way for the proctors to definitively know what was said. I suggested to one Sung about the potential ways the kids might have cheated, and that there were ways that we can check this. But he was not interested in finding out!

Interestingly, a good portion of the audience cheered and applauded when the correct guesses were revealed. One woman, not affiliated with KIBS, was moved to tears.

This phase of the experiment went exactly as I had expected it to. The kids in the KIBS tape never missed when wearing blindfolds, and neither did they now in front of an audience.

Envelopes phase

After a five-minute pause, the main trial began. As mentioned, each kid had a separate proctor, and I watched from the sidelines. The kids took up to 20 minutes for each guess, announced his guess in English, the proctor repeated the guess, the kid confirmed it loud enough for all to hear, the proctor wrote down the guess then checked the envelope for holes or tears which would have allowed the kid to cheat, then the proctor opened the envelope and wrote down the actual color.

We never found any holes, though one kid’s envelopes were consistently wet as if he tried to lick them and perhaps allow a tear—but the paper was too thick to allow him to see anything. The dampness could also have been sweat as this kid occasionally held the cards to his face tightly. He might have also been trying to see through the envelope as he consistently pressed the envelope to his face and pointed his face towards the ceiling lights. But my prior tests of directly holding the envelopes on the surface of an incandescent light bulb assured that it would not be possible to see through the envelope using a ceiling-mounted fluorescent bulb.

The first set of cards took about eight minutes. The kids grunted and sweat, ate come chocolates, did some exercises, and announced their guesses—not all at once, but when each was ready individually.

There was tension in the room when the first kid spoke his guess. The proctor confirmed it and then began to open the envelope. Remember, nobody, not me, not anybody, knew what the card’s color was going to be. People held their breath. I held my breath.

It was a miss. The color did not match the kid’s guess.

No problem, 35 more cards to go.

After a few more misses—and open groans and even hand wringing—there was finally a hit, followed by applause and cheers. Had things finally turned around?

Most of the guesses were wrong, and the pressure started to mount on the kids. Each kid was to get 12 envelopes, and by chance we’d expect they’d get 2 or so correct. But the hits—the correct guesses—were slow in coming.

The kids knew they were failing, the audience had become mostly silent, or sat quietly talking to one another. The kids began to get up more, ate even more candy, exercised more. But no change. Most guesses were misses.

Eventually, after it became clear that nothing more was going to happen, Sung told me he was going to stop the experiment out of concern for the kids’ anxiety. The remaining trials would be marked down as misses.

Sung stepped up and announced the trial’s ending. The audience understood, and clearly felt for the kids.

One kid did 7 trials, the other two did 6 before the experiment was stopped . They were scheduled to do 12 trials each. They got 4 hits during these 19 trials, right what chance would predict: kid one got 1, kid two got 1, kid three got 2.

Recall that before the trial started, KIBS staff members were confident each kid would get at least 10 out 12 hits.

Immediate sequel

Because the test was a failure, the oldest kid wanted to do another blindfold demonstration. I should not have allowed it as it was not part of the official protocol agreed to before the test. But I weakened and said Okay.

This demonstration went the same as before: he got both new cards right. It was then suggested be a proctor that all three kids be blindfolded at once, and that, respectfully, no noise be made. This was because the Sung and the KIBS people finally took seriously our suggestion that the kids were cuing each other when one was blindfolded and the others were not.

I still didn’t love this idea because there were many other people in the room who could have cheated if they wanted to (the KIBS staff, audience members, and so on), but the audience insisted on it. Again, I should not have allowed this, but I was weak.

Only kids two and three attempted a reading, but all were blindfolded anyway. In the previous blindfold demonstration, each kid made their guesses in under a minute. This time—when none of the kids could see each other—it took about five to seven minutes until kid number three started to complain of a stomachache. And so, even the last blindfold demonstration was called off with no guesses made.

The end

The evening ended with both Sung Lee and Dr. Ilchi Lee giving rambling, long, impromptu speeches saying things like, “Absence of evidence is not evidence of absence”, they hoped that more trials would be done, none of us really understood what is going on, and something about “mysterious Z-rays that are absorbed or emitted by the human brain.”

None of it made any sense, and the audience began shifting their traps by the end, clearly ready to leave.

Nobody from KIBS spoke to me after the test. I walked out with Mark Glickman and said goodbye.

Except for Sung Lee, that was the last time I talked to any KIBS member.

Next: the aftermath

Part I, II, III, IV, V

STUDY DESIGN

As I mentioned last time, it is crucial to design a test that eliminates all possibility and suspicion of cheating. If the KIBS kids were going to demonstrate extraordinary powers, I had to be sure they could not use any of their ordinary powers.

Luckily, this is easy to do for this kind of alleged ability. Blindfolds were definitely out, because nobody would ever—or should ever—believe results from a blindfold test. It is just too easy to cheat.

Still, Sung said that the kids were trained using blindfolds, were comfortable with them, and would like the chance to demonstrate their talents with them. I agreed to this, for one because there was no other way—they insisted on the blindfold demonstration—but also because a contrast between a way in which it was possible to cheat and one in which it was not would strengthen the evidence if the test failed.

Here, in our original draft of the experimental protocol, is what Sung and others said about brain respiration:

Throughout history, unusual skills have been claimed to occur as a by-product of meditative training. “Brain Respiration” (BR) is a mind-body training program that has been practiced by nearly one million persons in South Korea, Japan, North and South America, and Europe since the early 1980s, as a means to maximize brain function for the purpose of peace. More recently, the curriculum of this practice has included methods for developing “heightened sensory perception” (HSP). HSP is a putative perceptual faculty that allows identification of colors, shapes, letters or other forms of information without use of vision. HSP is described as a normal but relatively neglected neural capacity present in all humans. Use of HSP is felt to facilitate emotional and moral development, as well as to assist in discriminating among competing items of mutually exclusive information. Children are said to be more easily trained in HSP because of their fewer preconceptions against the possibility of such skills, as well as their greater neuroplasticity.

A draft document was written explaining the protocol to which all members agreed. That document was authored by Sung W. Lee, M.D., M.Sc., Instructor, Weill Medical College, Cornell University; Joseph Ingelfinger, M.D., Instructor, Harvard Medical School.; William Briggs, PhD., Assistant Prof. of Biostatistics, Weill Medical College, Cornell U.; JooRi Jun, B.S., Director of Special Projects, Korea Institute of Brain Science; Ul-Soon Lee, Vice President and Chief HSP Instructor, Korea Institute of Brain Science; Chang-Su Ryu, Director of Research, Korea Institute of Brain Science; Ilchi Lee, PhD. (honorary only), President, Korea Institute of Brain Science.

Overall

The test was to be in two phases: blindfold, and opaque envelope. Three Korean-national boys, aged 10 to 15, would take part. They were flown in a week before the test to acclimate to the new time and place. None were to take part in the test unless they expressed confidence they could perform.

During the blindfold phase, the kids would be handed cards and had to guess their colors. All in the audience could see the colors of the cards. The kids could take as long as they wanted, move in any way they wanted, eat chocolates or other candy, stand up and do exercises, until they were comfortable enough to take a guess.

In the opaque envelope phase, the kids took off the blindfolds and were presented with colored cards inside sealed envelops. The kids again could move around, eat, even talk to anybody they wanted. The only stipulation was that the envelope had to remain in sight on the table. They could take as long as they wanted until they were comfortable enough to take a guess.

During both phases, a separate proctor would watch each child. I recruited fellow professor of statistics Mark Glickman from Boston University to be one proctor. Sung found two others, both physics graduate students. I was to watch the proctors, and Sung ran the experiment.

When a child announced his guess, the proctor would hold up either the card or envelop and repeat the color. For example, if the child said “blue” (they spoke English for the color names), the proctor would say, “Blue?” or “Did you say blue?” The child could either confirm or change his mind. Once the entire audience, proctor, and child were satisfied that the guess was indeed “blue”, the proctor would record the guess on a piece of paper, and then the child would be told if he was correct or the envelope would be opened and revealed to all. The true color was then recorded on paper, and the next test would begin. If the kid got the color right, it was recorded as a “hit” else it was a “miss.” By agreement, if the kid did not know the color that trial was set to a “miss”.

After all the guesses were complete, then and there we would announce the results. We did not want anybody to go away not knowing what had happened.

The KIBS trainers were in charge of fashioning and fitting the blindfolds. Neither the proctors nor I verified their opacity or integrity. The blindfold phase was explicitly meant to be a “warm up” and would not be used as the official test results. Only the opaque envelope phase of the test was to be counted towards or against proof of HSP.

Sung and I agreed that either or both of us could write or publish anything we wanted about the test. I was very clear about this because of my concern of what might happen in the test was a failure. Our agreement was a gentleman’s one—and Sung is a gentleman—so there was and is no need of him to have written out a legal document.

Envelopes phase

Each child was to be presented with 12 cards contained in opaque envelopes. The cards were various colors: yellow, red, etc. Before the test, Dr S. Lee and I went to several office supply stores in order to find adequate envelopes. I held each sample envelope up to a 60-watt bulb to ensure there was no way they could be seen through. Once we were satisfied with the choice, Dr Lee ordered a large batch from a manufacturer in New Jersey. A quantity of these were sent to Korea by Dr S. Lee for the children to practice on. I took the rest. The envelopes were 65# grey, 6×9 inches, and colored paper card stock was cut to just fit inside each envelope. The example envelopes and colored stock were sent to Korea at least three weeks prior to the experiment, and none of these samples were used for the actual experiment.

I generated a random list of cards using the R statistical software platform. It picked the numbers 1 through 6, assigned each number to a color (1 was always “red”, for example), and then listed those colors. Computer random number generators take as initial input a “seed”, a large integer that is used to commence the sequence. I wrote the code to produce the numbers but I did not know the seed.

I gave the program to my number two son and he picked a seed, ran the program and generated the list. Then he and my number one son stuffed each of the envelopes. I did not know, before or during the test, which envelope held which color. My two sons remained in New York City while I made the trip to Boston. They sealed, in a separate envelope, the randomization sheet, which matched each envelope, which were numbered, with the colors generated by the computer. I did not open this envelope until after the experiment was over.

These precautions were necessary to ensure that there was no way I could offer any visual clues, consciously or not, to the kids during the experiment. They could not look to me to see whether their “Maybe blue?” guesses were right or not. The only two people in the world who knew the contents of the envelopes were nowhere near Boston, nor did they have any communication with anybody about the contents of those envelopes.

To be clear, the randomization procedure could have produced a list of cards that was, for example, all red, or “yellow, blue, yellow, blue,…” There was no way to know in advance—or during the test—what sequence would show itself to the kids. As it turned out, the list looked “random”, that is, it presented an equal mix of all 6 colors.

Each envelope was “licked” and then liberally glue-sticked closed. My sons and I experimented with this to see if we could find a way to cheat by, say, picking the corner loose and peeking inside. Of course, we cannot guarantee that cheating was impossible, but we felt it was extremely unlikely.

I had, and have, no idea how the kids used the envelopes to practice in Korea, but I could sense surprise when the envelopes proved difficult to open, even by the proctors.

Statistics

Calculating probabilities of “hits” and “misses” in this experiment is particularly easy. Since there is a 1 in 6 chance of getting any card right just by guessing, we can then calculate the probability, over 12 trials, the probability of 0 hits, 1 hit, 2 hits, and so on up to 12. For example, the KIBS people were confident that the kids would get around 10 hits each. The probability of guessing by chance 10 or more cards right is about 8 x 10^-7, or about 1 in 1.3 million, odds sufficiently low to give evidence that brain respiration worked if the kids scored that high.

For reference, the probability of scoring at least K hits out of 12 guesses is

However, do not forget that we are testing 3 kids, not just one. Imagine we are testing 1 million such kids. Do you think it would be unusual if at least 1 of those million got 10 or more hits? No. In fact, there is a 54% chance that at least 1 will score 10 or greater. Thus, we wouldn’t be surprised to see it happen. In our case, there is probability of 2.4 x 10^-6 that at least 1 kid will score 10 or greater. Still low enough that it would be surprising enough if it happened.

(To learn how to calculate these kinds of probabilities, turn to Chapters 3 and 5 of that wonderful book Breaking the Law of Averages: Real-Life Probability and Statistics in Plain English.)

Next: the day of the test

Part I, II, III, IV, V

INTRODUCTION

On 20 September 2004, in the Physics Department at MIT, sponsored locally by the Body and Brain Student Club, I proctored a test I had designed to probe the psychic powers of three Korean children.

The kids were under the tutelage of Ilchi Lee and the Korean Institute of Brain Science (KIBS). That group claimed that kids who had training in their methods of Brain Respiration and Dahn Yoga would be able to “read” the colors of cards inside opaque envelopes while blindfolded.

KIBS called this ability “non-visual, heightened sensory perception”, or HSP. They insisted that this is not the same as extra-sensory perception, or ESP, though the differences seem largely semantic.

This, in five parts, is the story of that experiment, what events led up to it, what happened during, and the sad but predictable aftermath.

I have written about this experiment before, but this is the first time I present all details. This will take us some time, and might even be tedious at times, but it is, you will see, necessary to be explicit as I can be.

BACKGROUND

In the fall of 2004, Dr Sung Won Lee was an attending physician in the department of General Internal Medicine and Center for Complementary and Integrative Medicine at the Cornell University Medical School. I was there, too, as an Assistant Professor of Biostatistics.

Sung and I were friends. He is one of the sweetest humans on the planet. We had been to each other’s homes for meals, compared grievances about our boss, and so on. We used to argue the physics and biology and what it means to be conscious, what constitutes evidence for or against a theory, the strictures of philosophical systems, what is and isn’t “science”. Sung put a lot of thought into his beliefs and was suspicious of logic.

Dr S. Lee got involved, a couple of years earlier, with Dr Ilchi Lee (no relation), KIBS, and Dahn Yoga. As part of an official experiment he was conducting at Cornell, Dr S. Lee was teaching adults Dahn Yoga to measure their improvement in various quality of life measures. Sung had many close social associations with other members of Dahn Yoga.

Sung knew that I had written a book about testing psychic phenomena—So, You Think You’re Psychic?—and that I was interested in magic, particularly mentalism, which is the art of fooling people into thinking you have psychic powers. So he asked whether I’d be interested in designing a test for KIBS.

He explained that his friend, Dr Ilchi Lee, had developed a method called brain respiration that would allow people to develop extraordinary sensory powers. For examples of these claims, click here. Note that this is only one of the many dozens of sites run by that group. More on this later.

Sung thought that training in brain respiration methods would allow kids to be able to “feel” colors—without, of course, using their eyes. He said that KIBS-trained kids could, while blindfolded, hold colored cards in their hands and correctly identify the color of those cards by touch. Sung had a promotional video tape from KIBS showing these miracles.

He brought the tape to my apartment and we watched it. The tape was in Korean, but Sung provided the translation. It started by showing a batch of children about six to twelve years old attending a brain respiration orientation. Before they knew its secrets, they tried reading the colors of cards blindfolded, but failed. They also tried to stick spoons on their face by “magnetizing” themselves but failed that, too. The kids then underwent training and afterwards could read colors blindfolded and could stick spoons to their faces. There might have been other abilities, but I’ve forgotten them.

I was exasperated by the performance because I thought the kids were obviously cheating. Sung believed they were not. I got a spoon from the kitchen and demonstrated it not sticking. I then “magnetizing” myself and—lo!—the spoon stuck. This is a simple trick, the simplest maybe, of all magic tricks. To get a spoon to stick, simply press it hard so that as large a surface area of the spoon touches your skin as possible. Make sure the bowl sticks outwards. It helps if your skin is not dry. I put mine on my forehead and nose.

I fashioned a blindfold and Sung handed me colored cards. I could easily “feel” their colors by cheating. Blindfold tricks are the next easiest and cheating is so easy a KIBS child could do it. Try it and you’ll find how simple it is to see anything, as long as you’re able to move the object to an advantageous spot, which the KIBS kids could.

I warned Sung that there were two ways to be fooled in psychic tests: the investigator could see things he wanted to see but weren’t there, and the person under study could cheat, knowingly or not. Psychic phenomena are supposed to occur by other than normal means, but most people do not appreciate how perceptive we are, and how we can sense subtle clues from our environment that might not allow us to, for example, say exactly what object is held in an opaque bag, but will let us guess better than randomly.

The temptation to cheat is strong, especially in kids, who are anxious to please adults and who enjoy praise when successful.

The stronger an experimenter wants to believe the more likely he will miss clues that his subjects are cheating, or to dismiss the times he did catch his subjects cheating as aberrations, or even as necessary to “relieve testing stress.” This has happened countless times in psychic testing.

In short, psychic tests are difficult because you have to work hard to eliminate all possible avenues of sensory leakage, remove the subtle clues that people unconsciously use, and make it impossible for cheating to occur.

You already believe, I told Sung, but others do not. The only way to convince outsiders that brain respiration was real was to produce incontrovertible evidence of its abilities under the strictest possible conditions. Sung agreed with this.

I told him that if a controlled experiment succeeded, I would be willing to believe in brain respiration, but given my experience with situations like this, I did not think it would work.

Then came my big warning. I explained what happens when people who believe strongly in psychic powers are presented with positive evidence that the powers are false. They refuse to acknowledge that evidence. They dismiss it, explain it away, insist it is flawed, or a fluke. In no way do they lessen their belief in the power; if anything, their belief strengthens.

They also blame others for the failure.

Sung said none of this would happen. He was confident that the KIBS trained kids would pass any test, and was excited about the prospect of demonstrating to the world the amazing possibilities of Dahn Yoga and brain respiration and training.

I said, OK, I would design the experiment.

Next: The design

Part I, II, III, IV, V