I used to review these papers in the hopes people would understand the old ways of doing statistics, in particular experiments on human subjects, were guaranteed to produce over-certainty.
I still want that. But now my co-equal, or even main, goal is for you to develop less respect for science. To not love any scientific announcement with the same ardor you might have done in the past. Science, though it sure is swell, is no longer the shining example of intellectual acuity it once was. There are still areas in science and technology free from the currents, but they are decreasing. Besides, too much love of science leads to scidolatry.
Anyway, our case study today is from Nature, “Effects of short-term exposure to particulate matter air pollution on cognitive performance” by Shehab and Pope.
PM 2.5, which is to say common dust, is all the rage in academia. Scientists like it because it’s a reliable paper producer; bureaucrats like it for its regulatory potential. “We must take a stand against dust! Consider the children!”
Ben Franklin, to name but one, was no intellectual slouch. He used to read books by candlelight. Many books, many candles. He read so much he had to invent bifocals. He went on to greater triumphs, as schoolchildren used to be taught. I’m sure they now learn he was racist, or whatever.
Point is that breathing in the dust created by burning all those candles didn’t seem to have any negative effects on his cognitive ability. Nor on the many others of his and earlier generations.
You would guess, if you were of the old-fashioned scientific bent, that by, say, 1850, after millennia of candle use, people would have noticed whether candles were making them stupider. Nobody did notice, and even noticed an opposite effect, such as allowing reading in dark rooms. Neither observation is definitive proof, but then, in science, most observations or experiments aren’t.
Smokers, too, inhale loads of particulate matter on purpose. They often report cognitive increases after partaking of pipe weed. Of course, these benefits come with other costs. And it’s probably not the dust, but the nicotine, that boosts synaptic flow.
Back to the paper. Shehab and Pope in one experiment gathered a group of 30 people, mostly students, and paid them to endure candle dust for an hour then gave them a questionnaire, a quiz. They did second experiment, which was much the same, but we can only endure so much, about going by a road.
An hour.
One hour.
Their findings:
The results from the MMSE test showed a statistically robust decline in cognitive function after exposure to both the candle burning and outdoor commuting compared to ambient indoor conditions…The findings from this study are potentially far reaching; they suggest that elevated PM pollution levels significantly affect short term cognition. This implies average human cognitive ability will vary from city to city and country to country as a function of PM air pollution exposure.
They gave 11 quizzes, which all gave numerical scores, once in normal air and again in air tinged with candle dust. They compared the scores from this pre and post.
It appears they report the mean of all the scores for the pre and then the mean of all the scores for the post. Which is the wrong thing to do. It should be the within-person change in score that is of interest. They did do paired t-tests to generate their wee Ps, which shows the understood this. But it appears they forget when tabulating the results. Maybe it’s men. My attention was flagging and I had just mainlined some beeswax.
Anyway, 11 quizzes, only one of which gave a wee P, and some of which showed higher (better) means for candle-dust exposure. Even if you love p-values, which shows a corruption of your soul, this is still depressingly underwhelming. If you adjust the P-values, as some do, for multiple tests, the “significance” here disappears.
But we remember that p-values grossly exaggerate evidence in favor of effects. The differences in means is so small, coupled with the noted standard deviations, allows us to guess that if these data were analyzed predictively, no effects at all would be noticed.
Did I mention the candles burned only for an hour?
This study suggests important implications for human cognitive ability and mental health and their dependence upon PM exposure. It suggests that citizens of more polluted cities and countries will have, on average, worse cognitive ability than they would have if air quality was better. Therefore, this study suggests that reductions in PM air pollution will not only result in improved human morbidity and mortality outcomes but also upon cognitive performance. Further work is now required…
Reminder: this breathless (get it? get it?) discussion was produced after just a handful of people were exposed to a candle for one hour, and where only one not-so-wee exaggerated P was discovered.
“Say, Briggs, why do you keep emphasizing the one hour?”
Glad you asked, friend. Because we go back to our millennia of evidence where people were not just exposed to one hour, but to lifetimes of candle dust. And there is no evidence in favor of the idea we are smarter than our ancestors; indeed, all indications are we are worse.
Maybe we should burn more candles.
To support this site and its wholly independent host using credit card or PayPal (in any amount) click here
The crucial question, of course, is “what is the air speed of a fully-laden African Swallow after exposure to candle dust?”
My candle burns at both ends
It will not last the night
I’ll breathe in all the candle dust
And forget how to rhyme
What scent was the candle?
How warm was the room?
What did the subjects have to do while waiting in a warm, candle-scented room?
How many cute girls were flirting with and distracting the boys (and vice-versa) while they waited in the warm, candle-scented room to take pointless quizzes?
Let’s see… Out of 11 tests, one showed P<0.05. That makes that… somewhat more than a 50% probability of happening at random. This outcome was more likely than not to show up regardless of conditions.
McChuck—the candle scent was my first thought, too. Scented candles obviously cause massive intelligence decline, as seen by people buying them at craft fairs. I’m working on a study, but the check does not seem to have been delivered by the mail. Meanwhile, I avoid craft fairs in defense of my intelligence.
Briggs: As for less respect for science, can one have less respect than zero?
My limited studies show people who lived when there were only candles, who rode horses WITHOUT wearing the person’s protective helmet, who routinely fought and got knocked out in bar fights, and who ate unrefrigerated food from their salted meat supplies (which were organic….) actually seemed to possess more intelligence. Now, it could be because stupid killed very quickly back then and we have overcome that to allow the stupid to reproduce, but without a p value, I really can’t say.
(In all seriousness, this is what makes medicine and much of the world as vague as when we had witchdoctors and medicine men. Studies are virtually useless, even assuming you can find the actual studies and not the molested, mangled news report of them. It’s often just a guessing game on what is true and what was just a reward for saying what your hateful government wants said.)
Sheri: As soon as I get funding, I am going to do a similar test to find out which is worse, scented candles or the fragrance inside a Bath and Body Works store. I know it might be easy to get distracted inside a store that people routinely mob, despite the wall of odor that extends several feet outside the opening, but I doubt that could really affect test-taking results, don’t you??
The description of the exercise sounds more like psychology than science.
Nonsense. Not just the bit about candle effects but the objective of having readers develop less respect for … science.
Why? Because some practitioners do science poorly.
Logically this is absurd. Poppycock. Barking madness. Etc…
IF that kind of sweeping generalization insensibility has any validity Briggs should be advocating the abolishment of statistics as an analytical tool. Get rid of stats as the Left would get rid of private gun ownership. After all, use of statistics is fundamentally wrongly implemented almost universally — p-value abuse and so forth. If abuse by practitioners is the benchmark then the abolishment of statistics is long overdue. By Briggs’ reasoning.
So, why persist in trying to fix use & application of statistics by those who misapply it on the one hand and demean a general discipline (science) en masse when some do it poorly on the other hand? Because science is revealing truths that undermine mindless dogmas. Better to retain a flawed belief system (that holds “truth” as a core precept!) than be exposed to scientific discoveries that disprove the cherished doctrines. Undermining “science” overall based on the abuse or ignorance of a few thus has its appeal. Stats never goes there.
Just a matter of time before a few malpractice cases lead to the conclusion that “medicine” (diagnosis, surgery, medication … you name it) is similarly deemed unworthy. Then back to the “witch doctor” in its various forms. Like blaming liberated women for much of modern social ills in the recent post here (which, somehow, stopped short of advocating the sacrifice of virgins).
How much beer money did the students get for taking the test?
I recently saw a photo of the pollution levels in Beijing. Obviously, the population of Beijing and other Chinese cities are cognitively declining and thus are no longer an economic threat to the US.
Did they even do the appropriate negative control?
https://www.picclickimg.com/d/l400/pict/283452823577_/Two-Vintage-3-Tier-Christmas-Candolier-Electric-Window-Candelabra.jpg
This study suggests important implications for human cognitive ability and mental health and their dependence upon PM exposure. It suggests that citizens of more polluted cities and countries will have, on average, worse cognitive ability than they would have if air quality was better.
Did I mention the candles burned only for an hour?
Explains why candles are burned in church services. It controls the masses by limiting their cognition. Most church services are roughly an hour. Coincidence?
This is almost funny. They did actually show improvement on many cognitive functions, especially on some stroop tests. It’s just that they didn’t know how to interpret their own results. Just one look at means and SDs is enough to see they documented noise.
However, PM 2.5 is an issue because of the size of the particles that penetrates deep into lungs, unlike common dust or PM 10, soot, etc. Measuring and controlling for the variables is another thing, as those particles don’t come isolated in their influence. Rather, they are mixed with all other chemical pollutants and what not (we had pretty bad fires last couple of years here in CA and the smoke composition consisted of everything a common US household has, including people and cattle). It’s true that simple act of cooking at home produces pretty bad PM2.5 levels on its own and that sometimes opening a car window results in healthier air than closing them, but I think it’s best to err on the side of caution. Firefighters do have more respiratory diseases and all of us are in a big experiment so it’s really hard to do a controlled study when everyone and their cousin-in-law are breathing and ingesting all sorts of industrial crap, whether we want it or not.
Regarding this poorly-controlled study, they should have used some common sense first. Long time ago when candles were used, houses weren’t insulated efficiently, replenishing the indoor air more efficiently and the outside air was probably cleaner than today (no cars, toxic pollution, etc.). Additionally, everything was scarce, including candles, so I’m not sure about the mass exposure at the population level at any time in our history, like we have today.
Best to do studies on firefighters and other people exposed to particulates (long-distance drivers, various other industries exposed to PM 2.5 and such), but I believe those occupational hazard cases and their higher rates of respiratory illnesses are well-documented, with a remaining question of how all that stuff affects general public.
Kalif – “Best to do studies on…long-distance drivers…”. Those studies have been conducted in which they report adverse effects on the drivers, but I say 50 years and 50 pounds is causing all those health issues. Today’s truck drivers are way over weight, eat poorly, and perform very little exercise compare to truck drivers of 50 years ago.
Those studies are an epidemiological farce in my estimation.
@Ye Olde Statistician
“The description of the exercise sounds more like psychology than science.”
Are you saying that comparative quantitation of degrees of “cognitive ability” or “mental health” aren’t exactly representative of the gold standards to which all scientific measurement aspires?
…even without incense, chanting and meditation?
As for every successful study – Conclusion: More study is now required!