Skip to content
January 6, 2009 | 19 Comments

The MIT Dahn Yoga Brain Respiration Experiment: Part III

THE EXPERIMENT

The test was filmed by several cameras, most from the KIBS people (I have been unable to find any videos on line). The KIBS staff itself was large, about a dozen or two. The public was not barred from filming or photographing any part of the proceedings, and many did. I did not meet many people, but I was told prominent physics and medicine professors from Harvard and MIT were invited, as were other “thought leaders”. There were about forty people not from KIBS in attendance.

There were snacks and drinks provided. I set up in a corner and tried to be be non obtrusive. From what I could gather from the chatter, the audience was anxious but expected good things. Everybody, staff included, was all smiles. Each kid was more adorable than the last, and we were all pulling for them.

Ilchi Lee gave a standard opening speech in Korean, which was simultaneously translated by Sung Lee. Sung also gave a speech explaining the history and goals of brain respiration and what was expected that night. I was introduced, as were the proctors, but none of us spoke.

Sung explained that, in practice, the kids had hit rates of 80-90% with the test envelopes during the previous two weeks. As I said before, I have no clear idea how they did these unsupervised trials. We were thus expecting each kid to guess about 10 more envelopes correctly.

Sung outlined the experimental protocol, assured the audience we would see some special that night, and the test began.

Blindfold phase

The three kids sat together at a table, but only one at a time was blindfolded. Not surprisingly, the kids got all the blindfold guesses correct. I’m pretty sure they didn’t peek down their nose, though they could have above it because they held the cards up high in the air to indicate that they were ready to announce their guess.

But they didn’t have to peek to cheat. I and two other proctors, who also had experience in magic, felt that the kids were cluing each other (the third proctor admitted the possibility but did not care to guess); which is to say they were either telling the blindfolded kid openly what color card he held, or using other signaling methods. There were certainly countless ways the kids could have communicated to each other. All in the room could see the card except the blindfolded kid. The kids were fairly close to one another and they were allowed to move about and talk when the other of them was blindfolded. I’m fairly sure on the kids and the KIBS staff spoke Korean, so there was no way for the proctors to definitively know what was said. I suggested to one Sung about the potential ways the kids might have cheated, and that there were ways that we can check this. But he was not interested in finding out!

Interestingly, a good portion of the audience cheered and applauded when the correct guesses were revealed. One woman, not affiliated with KIBS, was moved to tears.

This phase of the experiment went exactly as I had expected it to. The kids in the KIBS tape never missed when wearing blindfolds, and neither did they now in front of an audience.

Envelopes phase

After a five-minute pause, the main trial began. As mentioned, each kid had a separate proctor, and I watched from the sidelines. The kids took up to 20 minutes for each guess, announced his guess in English, the proctor repeated the guess, the kid confirmed it loud enough for all to hear, the proctor wrote down the guess then checked the envelope for holes or tears which would have allowed the kid to cheat, then the proctor opened the envelope and wrote down the actual color.

We never found any holes, though one kid’s envelopes were consistently wet as if he tried to lick them and perhaps allow a tear—but the paper was too thick to allow him to see anything. The dampness could also have been sweat as this kid occasionally held the cards to his face tightly. He might have also been trying to see through the envelope as he consistently pressed the envelope to his face and pointed his face towards the ceiling lights. But my prior tests of directly holding the envelopes on the surface of an incandescent light bulb assured that it would not be possible to see through the envelope using a ceiling-mounted fluorescent bulb.

The first set of cards took about eight minutes. The kids grunted and sweat, ate come chocolates, did some exercises, and announced their guesses—not all at once, but when each was ready individually.

There was tension in the room when the first kid spoke his guess. The proctor confirmed it and then began to open the envelope. Remember, nobody, not me, not anybody, knew what the card’s color was going to be. People held their breath. I held my breath.

It was a miss. The color did not match the kid’s guess.

No problem, 35 more cards to go.

After a few more misses—and open groans and even hand wringing—there was finally a hit, followed by applause and cheers. Had things finally turned around?

Most of the guesses were wrong, and the pressure started to mount on the kids. Each kid was to get 12 envelopes, and by chance we’d expect they’d get 2 or so correct. But the hits—the correct guesses—were slow in coming.

The kids knew they were failing, the audience had become mostly silent, or sat quietly talking to one another. The kids began to get up more, ate even more candy, exercised more. But no change. Most guesses were misses.

Eventually, after it became clear that nothing more was going to happen, Sung told me he was going to stop the experiment out of concern for the kids’ anxiety. The remaining trials would be marked down as misses.

Sung stepped up and announced the trial’s ending. The audience understood, and clearly felt for the kids.

One kid did 7 trials, the other two did 6 before the experiment was stopped . They were scheduled to do 12 trials each. They got 4 hits during these 19 trials, right what chance would predict: kid one got 1, kid two got 1, kid three got 2.

Recall that before the trial started, KIBS staff members were confident each kid would get at least 10 out 12 hits.

Immediate sequel

Because the test was a failure, the oldest kid wanted to do another blindfold demonstration. I should not have allowed it as it was not part of the official protocol agreed to before the test. But I weakened and said Okay.

This demonstration went the same as before: he got both new cards right. It was then suggested be a proctor that all three kids be blindfolded at once, and that, respectfully, no noise be made. This was because the Sung and the KIBS people finally took seriously our suggestion that the kids were cuing each other when one was blindfolded and the others were not.

I still didn’t love this idea because there were many other people in the room who could have cheated if they wanted to (the KIBS staff, audience members, and so on), but the audience insisted on it. Again, I should not have allowed this, but I was weak.

Only kids two and three attempted a reading, but all were blindfolded anyway. In the previous blindfold demonstration, each kid made their guesses in under a minute. This time—when none of the kids could see each other—it took about five to seven minutes until kid number three started to complain of a stomachache. And so, even the last blindfold demonstration was called off with no guesses made.

The end

The evening ended with both Sung Lee and Dr. Ilchi Lee giving rambling, long, impromptu speeches saying things like, “Absence of evidence is not evidence of absence”, they hoped that more trials would be done, none of us really understood what is going on, and something about “mysterious Z-rays that are absorbed or emitted by the human brain.”

None of it made any sense, and the audience began shifting their traps by the end, clearly ready to leave.

Nobody from KIBS spoke to me after the test. I walked out with Mark Glickman and said goodbye.

Except for Sung Lee, that was the last time I talked to any KIBS member.

Next: the aftermath

Part I, II, III, IV, V

January 5, 2009 | 16 Comments

The MIT Dahn Yoga Brain Respiration Experiment: Part II

STUDY DESIGN

As I mentioned last time, it is crucial to design a test that eliminates all possibility and suspicion of cheating. If the KIBS kids were going to demonstrate extraordinary powers, I had to be sure they could not use any of their ordinary powers.

Luckily, this is easy to do for this kind of alleged ability. Blindfolds were definitely out, because nobody would ever—or should ever—believe results from a blindfold test. It is just too easy to cheat.

Still, Sung said that the kids were trained using blindfolds, were comfortable with them, and would like the chance to demonstrate their talents with them. I agreed to this, for one because there was no other way—they insisted on the blindfold demonstration—but also because a contrast between a way in which it was possible to cheat and one in which it was not would strengthen the evidence if the test failed.

Here, in our original draft of the experimental protocol, is what Sung and others said about brain respiration:

Throughout history, unusual skills have been claimed to occur as a by-product of meditative training. “Brain Respiration” (BR) is a mind-body training program that has been practiced by nearly one million persons in South Korea, Japan, North and South America, and Europe since the early 1980s, as a means to maximize brain function for the purpose of peace. More recently, the curriculum of this practice has included methods for developing “heightened sensory perception” (HSP). HSP is a putative perceptual faculty that allows identification of colors, shapes, letters or other forms of information without use of vision. HSP is described as a normal but relatively neglected neural capacity present in all humans. Use of HSP is felt to facilitate emotional and moral development, as well as to assist in discriminating among competing items of mutually exclusive information. Children are said to be more easily trained in HSP because of their fewer preconceptions against the possibility of such skills, as well as their greater neuroplasticity.

A draft document was written explaining the protocol to which all members agreed. That document was authored by Sung W. Lee, M.D., M.Sc., Instructor, Weill Medical College, Cornell University; Joseph Ingelfinger, M.D., Instructor, Harvard Medical School.; William Briggs, PhD., Assistant Prof. of Biostatistics, Weill Medical College, Cornell U.; JooRi Jun, B.S., Director of Special Projects, Korea Institute of Brain Science; Ul-Soon Lee, Vice President and Chief HSP Instructor, Korea Institute of Brain Science; Chang-Su Ryu, Director of Research, Korea Institute of Brain Science; Ilchi Lee, PhD. (honorary only), President, Korea Institute of Brain Science.

Overall

The test was to be in two phases: blindfold, and opaque envelope. Three Korean-national boys, aged 10 to 15, would take part. They were flown in a week before the test to acclimate to the new time and place. None were to take part in the test unless they expressed confidence they could perform.

During the blindfold phase, the kids would be handed cards and had to guess their colors. All in the audience could see the colors of the cards. The kids could take as long as they wanted, move in any way they wanted, eat chocolates or other candy, stand up and do exercises, until they were comfortable enough to take a guess.

In the opaque envelope phase, the kids took off the blindfolds and were presented with colored cards inside sealed envelops. The kids again could move around, eat, even talk to anybody they wanted. The only stipulation was that the envelope had to remain in sight on the table. They could take as long as they wanted until they were comfortable enough to take a guess.

During both phases, a separate proctor would watch each child. I recruited fellow professor of statistics Mark Glickman from Boston University to be one proctor. Sung found two others, both physics graduate students. I was to watch the proctors, and Sung ran the experiment.

When a child announced his guess, the proctor would hold up either the card or envelop and repeat the color. For example, if the child said “blue” (they spoke English for the color names), the proctor would say, “Blue?” or “Did you say blue?” The child could either confirm or change his mind. Once the entire audience, proctor, and child were satisfied that the guess was indeed “blue”, the proctor would record the guess on a piece of paper, and then the child would be told if he was correct or the envelope would be opened and revealed to all. The true color was then recorded on paper, and the next test would begin. If the kid got the color right, it was recorded as a “hit” else it was a “miss.” By agreement, if the kid did not know the color that trial was set to a “miss”.

After all the guesses were complete, then and there we would announce the results. We did not want anybody to go away not knowing what had happened.

The KIBS trainers were in charge of fashioning and fitting the blindfolds. Neither the proctors nor I verified their opacity or integrity. The blindfold phase was explicitly meant to be a “warm up” and would not be used as the official test results. Only the opaque envelope phase of the test was to be counted towards or against proof of HSP.

Sung and I agreed that either or both of us could write or publish anything we wanted about the test. I was very clear about this because of my concern of what might happen in the test was a failure. Our agreement was a gentleman’s one—and Sung is a gentleman—so there was and is no need of him to have written out a legal document.

Envelopes phase

Each child was to be presented with 12 cards contained in opaque envelopes. The cards were various colors: yellow, red, etc. Before the test, Dr S. Lee and I went to several office supply stores in order to find adequate envelopes. I held each sample envelope up to a 60-watt bulb to ensure there was no way they could be seen through. Once we were satisfied with the choice, Dr Lee ordered a large batch from a manufacturer in New Jersey. A quantity of these were sent to Korea by Dr S. Lee for the children to practice on. I took the rest. The envelopes were 65# grey, 6×9 inches, and colored paper card stock was cut to just fit inside each envelope. The example envelopes and colored stock were sent to Korea at least three weeks prior to the experiment, and none of these samples were used for the actual experiment.

I generated a random list of cards using the R statistical software platform. It picked the numbers 1 through 6, assigned each number to a color (1 was always “red”, for example), and then listed those colors. Computer random number generators take as initial input a “seed”, a large integer that is used to commence the sequence. I wrote the code to produce the numbers but I did not know the seed.

I gave the program to my number two son and he picked a seed, ran the program and generated the list. Then he and my number one son stuffed each of the envelopes. I did not know, before or during the test, which envelope held which color. My two sons remained in New York City while I made the trip to Boston. They sealed, in a separate envelope, the randomization sheet, which matched each envelope, which were numbered, with the colors generated by the computer. I did not open this envelope until after the experiment was over.

These precautions were necessary to ensure that there was no way I could offer any visual clues, consciously or not, to the kids during the experiment. They could not look to me to see whether their “Maybe blue?” guesses were right or not. The only two people in the world who knew the contents of the envelopes were nowhere near Boston, nor did they have any communication with anybody about the contents of those envelopes.

To be clear, the randomization procedure could have produced a list of cards that was, for example, all red, or “yellow, blue, yellow, blue,…” There was no way to know in advance—or during the test—what sequence would show itself to the kids. As it turned out, the list looked “random”, that is, it presented an equal mix of all 6 colors.

Each envelope was “licked” and then liberally glue-sticked closed. My sons and I experimented with this to see if we could find a way to cheat by, say, picking the corner loose and peeking inside. Of course, we cannot guarantee that cheating was impossible, but we felt it was extremely unlikely.

I had, and have, no idea how the kids used the envelopes to practice in Korea, but I could sense surprise when the envelopes proved difficult to open, even by the proctors.

Statistics

Calculating probabilities of “hits” and “misses” in this experiment is particularly easy. Since there is a 1 in 6 chance of getting any card right just by guessing, we can then calculate the probability, over 12 trials, the probability of 0 hits, 1 hit, 2 hits, and so on up to 12. For example, the KIBS people were confident that the kids would get around 10 hits each. The probability of guessing by chance 10 or more cards right is about 8 x 10^-7, or about 1 in 1.3 million, odds sufficiently low to give evidence that brain respiration worked if the kids scored that high.

For reference, the probability of scoring at least K hits out of 12 guesses is

K Prob(Hits >= K | 1/6 chance)
0 1
1 0.89
2 0.62
3 0.32
4 0.13
5 0.036
6 0.0079
7 0.0013
8 0.00016
9 1.3 x 10^-5
10 7.9 x 10^-7
11 2.8 x 10^-8
12 4.6 x 10^-10

However, do not forget that we are testing 3 kids, not just one. Imagine we are testing 1 million such kids. Do you think it would be unusual if at least 1 of those million got 10 or more hits? No. In fact, there is a 54% chance that at least 1 will score 10 or greater. Thus, we wouldn’t be surprised to see it happen. In our case, there is probability of 2.4 x 10^-6 that at least 1 kid will score 10 or greater. Still low enough that it would be surprising enough if it happened.

(To learn how to calculate these kinds of probabilities, turn to Chapters 3 and 5 of that wonderful book Breaking the Law of Averages: Real-Life Probability and Statistics in Plain English.)

Next: the day of the test

Part I, II, III, IV, V

January 3, 2009 | No comments

Technical problems: cache

Hi all.

The links to the KIBS psychic experiment will work, once they actual stories appear on their scheduled dates. They won’t work for the parts of the series that have not yet been published.

There also appears to be a problem displaying comments. I usually fix this by deleting the WordPress cache. I’d do that know, but I’m in Taiwan on an extraordinarily slow internet connection and having a difficult time deleting the cache.

All the comments are in the database, though. Nothing is lost.

I’ll update more once I find a better internet cafe.

January 1, 2009 | 2 Comments

Happy New Year!

An oldie, but a goody. Better to start the new year smiling.