Here are two fun fallacies, excerpted from a paper I wrote a couple of years ago in the Journal of American Physicians and Surgeons. For more on this sort of thing, buy this book before they run out: Uncertainty: The Soul of Modeling, Probability & Statistics.
The Everyone Else Said It Was True Fallacy
“Radon is one of the most serious environmental health risks that we face,” said Univeristy of Minnesota professor Bill Angell. He explains that the colorless, odorless radioactive gas forms naturally in the ground, but when it enters your home, it is a serious problem.
“The risk of dying of lung cancer because of radon in your home is one out of 50,” said Angell, “So it’s an incredibly big risk.”
Angell’s comments were based on published studies such as a Danish cohort study by Bräuner et al., whose abstract read: “We find a positive association between radon and lung cancer risk consistent with previous studies…. [T]he results of the present prospective cohort study are fully compatible with an association between residential radon and risk for lung cancer as detected in three previous meta analyses and provide important evidence at the low end of the low end of the residential dose curve.”
In that study, the authors measured actual exposure and outcomes of about 57,000 Danes and found the “adjusted [risk] for lung cancer was 1.04 (95% CI: 0.69–1.56) in association with a 100 Bq/m^3 higher radon concentration and 1.67 (95% CI: 0.69–4.04) among non-smokers.” Since the confidence intervals include 1, the classical interpretation is that radon is therefore not significantly associated with lung cancer. In fact, the authors said as much: “The role of chance cannot be excluded as these associations were not statistically significant.”
The finding of no effect was contrary to expectations, so the authors said, “In the present study, a number of risk factors for lung cancer were less prevalent among participants living at the higher radon concentrations, including…low fruit intake, risk occupation and traffic-related air pollution. This would result in an underestimation of the association between radon and lung cancer risk in our study.”
These words were necessary to suggest that radon might still cause lung cancer even in the face of great evidence it did not. The authors felt that something had to explain the non-effect, because they were unwilling to conceive that radon (at the stated levels) might be harmless to lungs. So in their explanation they discarded the massive evidence they collected and surmised that radon was just as deadly as commonly thought.
The Everyone Else Said it Was True Fallacy is: Even though your results are the exact opposite of your belief, explain them away, then state your belief.
The Statistics Aren’t What You Think They Are Fallacy
Here are two headlines from The Daily Mail, the popular English newspaper. “Bad news for chocoholics: Dark chocolate isn’t so healthy for you after all,” from a Jan 24, 2012, article explaining that chocolate doesn’t do much for the heart. Then just three months later, on Apr 24, another headline claimed: “Eating dark chocolate is good for your heart.” Both headlines drew on different peer-reviewed medical studies that concluded, using p-values as evidentiary markers, that chocolate was and wasn’t good for hearts.
Two more headlines from this newspaper read: “Ignore all that hype about antioxidant supplements: Why daily vitamin pills can INCREASE your risk of disease” (May 21, 2012), and “The vitamin pills that actually work! How some supplements can work wonders for certain ailments” (May 27). Some of the ailments were the same in both stories. These were also based on peer-reviewed studies, using p-values to “prove” their contentions.
On Apr 11, 2011, a headline announced: “Women who drink four cups of coffee a day face higher risk of incontinence.” Then from Thomson-Reuters, (the Daily Mail did not cover the follow-up study) a year later, on Apr 27, 2012, readers were told: “Caffeine not tied to worsening urinary incontinence.” The underlying story was the same.
On Jul 29, 2004, a headline on OBGYN.net read: “Pomegranates shown to be effective for menopausal symptoms.” It took eight years for the Daily Mail to report on Jan 24, 2012, that: “Pomegranate seed oil ‘no better than a placebo’ at easing hot flashes,” (a menopausal symptom). Both reports were based on peer-reviewed studies that used p-values as evidence.
The Statistics Aren’t What You Think They Are Fallacy is also known as the P-values Aren’t Proof Fallacy. Researchers want to know the probability that some theory is true given the evidence they have collected. This theory is then often used in developing medical practice guidelines, particularly when the theory fits expectations.
But p-values, the measures upon which most studies rely, and which everybody, even those who know better, take as proof of a theory when the p-values are less than the magic value of 0.05, do not give evidence that any theory is true. 8 Indeed, the actual definition of a p-value is so complicated nobody ever remembers it; all that is recalled is that p-values should be small.
The books were being printed when ordered. What is the statistical probablility of them “running out”?
We trailer park trash don’t have to worry about radon. Seems radon doesn’t build up under trailers—something about houses crushing the soil and releasing radon, which does not happen at a significant level when your home sits on cinder blocks. I always suspected trailer park trash had it right!
I worked for HUD back in the 90s, when they were pushing homeownership (which led directly to the credit crisis), including EPA brochures on the alleged dangers of radon. I soon noticed that there was not one single FACT in the EPA’s radon brochure! Guess you could say they’ve got lying down to a science…
Wee pee values?
Caffeine doesn’t cause incontinence but is an aggravating factor in some instances.
Caffeine is a bladder ‘irritant’ as it increases the excitability of stretch receptors in the bladder wall by lowering threshold for nerves to fire. The web of contractile tissue requires less liquid to detect stretch. Caffeine can be problematic for those who already have stress incontinence or as is fashionably referred to, bladder sensitivity which is not quite the same problem but both are effected. Magazines and papers just like to feed their public and there’s always an opportunity to sell an idea or a product where there’s a need. If they were genuine they’d cover the topic properly once and for all.
Physiotherapy can change lives of men and women who have incontinence and it can be treated successfully without surgery. It is a very broad spectrum and not one situation or condition only. Physiotherapy has moved on a long way in Obs and Gynae and a proper Obs and Gina physio is competent in internal examination and palpation of muscle tissue to locate focal weakness, point them out to patients and manage accordingly.
Even in men’s health now which obviously requires a different kind of exam men who have suffered much of their life with the problem can be helped but they don’t tell anyone and often their GP doesn’t refer or know a good physio. If the consultant is keen he will refer to his physio if conservative treatment is preferred, which it always should be where possible.
One shot’s better than two, cheers!
Here in Michigan many new homes come with a radon mitigation package, a system that pulls air from your basement to outside the home.
Just to be clear I meant newspapers and magazines not the above article which is instructive as always in matters of statistics.
I would like every physio library to have a copy and for it to be compulsory reading for all medical students and physiotherapists.
Perhaps there’s a case for calling the Daily Mail reporting “balanced”.
According to a study by the National Cancer Institute (NCI) , household radon gas is a threat to your health but in another study conducted by NCI household radon gas is not a threat. Now, who are you going to believe, the NCI or the NCI?
Dose and route of administration make the medicine. Dose and route of administration make the poison. For inhalants such as radon, which is a very rare ‘noble’ gas and mostly an alpha emitter (though one isotope is a gamma emitter and another is a beta emitter), damage is directly related to proximity to tissue and the amount present. Obviously, other sources of local cell damage will have a lower threshold for radon induced damage.
Since we live on a radioactive planet, with high oxygen content in the atmosphere and illumined by a variable star, we have systems built in that deal with the every day chemical damage we face, especially for organisms that have oxygen dependent bodies. Lack of knowledge of, or consideration of, the multiplicity of factors that result in pathological states, nearly everything known in medicine is both right and wrong, contingently.
NOTE these remarks by Briggs:
“The Everyone Else Said it Was True Fallacy is: EVEN THOUGH YOUR RESULTS ARE THE EXACT OPPOSITE OF YOUR BELIEF, EXPLAIN THEM AWAY, THEN STATE YOUR BELIEF.”
AND:
“The Statistics Aren’t What You Think They Are Fallacy is also known as the P-values Aren’t Proof Fallacy. Researchers want to know the probability that some theory is true given the evidence they have collected. … But p-values, … which … EVEN THOSE WHO KNOW BETTER, take as proof of a theory … do not give evidence that any theory is true.”
Both of Briggs’ above observations reflect the behavioral realities of too many researchers with a personal agenda that will sacrifice objectivity for personal gain.
Note the following:
– Those researchers, generally, will apply the math (statistics) correctly,
– but when the actual results are “the opposite of your [their] belief,” or, the desired results/findings are not supported by the research…and then… “those who know better” sometimes report a desired [not actual] finding/conclusion/answer to prove a theory, or, preserve an unsupported belief.
Doing the research and applying the math all constitute “science.”
Manipulating and misrepresenting the findings to support a particular pre-determined result or support a belief constitutes deceit/corruption — SCIENCE STOPS WHERE THE DECEPTION STARTS.
Keep that very simple, and profoundly significant, distinction in mind if/when reviewing any of Briggs’ earlier posts and subsequent posts — those posts where “science” as a discipline is maligned. For some reason Briggs has repeatedly condemned “science” as a discipline based on the deceitful/corrupt practices of researchers who abandon their objectivity and succumb to situational politics for personal benefit (including those, such as media reporters, that publish their descriptions of the findings of actual scientists, but wrongly extrapolate the significance of those findings). For some reason, bashing “science” is preferable to pointing out that particular findings reported as science aren’t really science.
Makes one wonder, what is it about “science” that is so threatening?
Radon is a type of ionizing radiation — and radiation is believed to never healthy for an organism to absorb. The WHO & CDC, etc., have published guidelines for safe limits of Radon in pico-Curies. How do those limits compare with other limits for safe ionizing radiation doses/exposure?
Some objective measurement comparisons might reveal if there’s any inconsistencies applied to ionizing radiation exposure from radon vs other sources…” Doing such calculations & making those comparisons isn’t easy, as this MIT article makes accessibly clear about the types of radiation, unit conversions, and types of exposure vs absorption: http://news.mit.edu/2011/explained-radioactivity-0328
There may be a fundamental flaw in radiation exposure limits at very low doses “…due to the health effects of low-level radiation being based on a mathematical assessment called the linear no-threshold model. The model essentially uses math to extrapolate the effects of low doses from observed effects at high doses.”
“However, evidence that’s been trickling in since the nuclear age suggests that Low Dose Radiation (LDR) could actually benefit human health. In other words, not only is there a threshold for radiation exposure – a limit below which radiation should not be harmful– but at certain low levels ionizing radiation may do more good for your cells than harm. The idea that a low dose of a bad thing can have good effects is called hormesis.”
The support for LDR being beneficial is very dubious, at best there may be some benefits but with those outweighed by other negative effects.
References:
Quote re hormesis from: http://blogs.discovermagazine.com/crux/2015/04/06/small-radiation/#.WO0Vkfnyvcs
Early article re hormesis, noting that individual variability is considerable–making application unlikely in medical treatments, and, the aggregate environmental effect may well be bad: http://discovermagazine.com/2002/dec/featradiation
Ken,
You missed what cdquarles wrote. Radon is an element, not radiation.
“For inhalants such as radon, which is a very rare ‘noble’ gas and mostly an alpha emitter (though one isotope is a gamma emitter and another is a beta emitter), damage is directly related to proximity to tissue and the amount present.”
Ken: “hormesis is never believed to be a factor when radiation is involved, because that would be wrong.”
And I would like to remind readers of a remarkable 1987 paper that Matt noticed long ago: Berger JO, Selke T. Testing a point null hypothesis: the irreconcilability of p-values and evidence. JASA 33:112-122.
In sum, (even neglecting all the other problems there are with ‘hypothesis’ ‘testing’) Berger and Selke proved that it is not accurate that if a p-value is very low, then it’s very unlikely that the given assumptions are incorrect. They proved that a ‘low’ p-value does not necessarily indicate this.
Since you can’t tell from the p-value alone whether it’s a ‘nice’ p-value, or one similar to the ones Berger and Selke found, I would conclude, purely on mathematical grounds, and disregarding every other problem with p-values, that mistrusting all p-values ever, is the only sane response to p-values.
I’m a little late to this conversation, but hopefully some of the responders above will see these comments. As an introduction, my career was in Health Physics, ergo I know a fair amount about the effects of radiation on humans or otherwise.
1) The EPA’s so-called “science” regarding Radon is more than bad science and math, it was flat-out a deception that created another bloated bureaucracy preying on folks’ ignorance of radiation. Initially, the EPA claimed 0 – 5000 deaths per year from Radon based on radon studies from strictly high doses (all the studies had serious methodological problems). This did not raise too many eyebrows. Subsequently, the EPA revised its numbers using very dubious reasoning and math raising the death toll to 0-10 thousand per year. Again, this did not inflame the public much. Shortly, the EPA came out stating that Radon was causing 20K deaths per year (notably dropping the 0 – XYZ range designation) without any justification what so ever. I searched hard for their justification and it was not to be found.
2) I attended two Health Physics meetings where the head of the EPA’s Radon program admitted that the EPA did not, and would not, take into account any contradictory studies such as Prof. Bernard Cohen’s work demonstrating that low doses of Radon seemed to have a beneficial effect.
3) To Sherri and Yawrate. Uranium is ubiquitous in the environment. Its decay path converts the Uranium molecule to Radium and then Radon eventually. How much Radon may be present is a direct function of how much Uranium/Radium is in the soil. This quantity varies widely by location. Though cinder block will likely contain uranium/radium from its fly ash constituent, Sherri is likely right about trailer homes, because the air flow in the crawl space would disperse any Radon before it could accumulate. In a home with a basement, up to 20% of the make-up air (gases) can be drawn up through the floor and basement walls. Again this is highly variable and depends on temperature, humidity and pressure. Pressure is a big factor. Heated air in a home will rise and create a negative pressure on the basement floor/walls. This pulls the gases in the soil into the home principally through cracks in the concrete (dirt floor basements have no resistance to the gases). Oxygen, nitrogen and any gas will be drawn into the home. Of course Radon is a gas. Being heavier than other gases, it tends to accumulate in basements, but this depends on airflow in the basement. Homes with Radon mitigation systems simply put piping under the basement floor connected to a fan which creates a negative pressure under the basement and also pulls the soil gases through the pipes. Homes on slabs get very little infiltration of soil gases including Radon. Therefore Yawrate, if you buy a home with a Radon mitigation system: 1) if on a slab, turn the system off – you are just wasting energy on the fan or; 2) if with basement, turn off the system and get your basement checked for Radon. The EPA’s 4 pCi/l recommendation is garbage. Anything less than 20 pCi/l is more than safe. Personally, I’m comfortable up to 100 pCi/l.
3) I can get seriously into the weeds about Radon’s effects if anyone is interested. However, I’ll just point out that Radon is a noble gas, therefore our body does not use or store it. Because we are always breathing a little Radon (its everywhere, its everywhere!), there is a finite probability that one of the Radon atoms will emits its radiation (alpha particle) while it happens to be in our body. If this event does not occur, then the Radon atom will likely just be breathed right back out of the body. (note: a common misperception is that a single radioactive atom emits radiation all the time. It does not. A radioactive atom only emits its radiation once, then it changes to a different atom.) The decay of a Radon-222 atom goes to Po-218 and this atom is quite reactive with chemicals around it and will stay in the body. There are 6 more decay steps before the atom becomes stable Pb-206 (non-radioactive). I can get into lung clearance rates and likely distribution of the Radon daughter products is anyone is interested.
The only Health Physicists I know who think Radon is a problem are the ones who work in Radon programs set up in the States by the EPA.