Statistics

Do Heat Waves Cause Birth Defects?

A popular device to prevent birth defects

What would you name a paper which purports to discover that heat waves stress pregnant women and thus cause birth defects but which never once measures the actual exposure of pregnant women to heat?

If you answered “A Population-Based Case–Control Study of Extreme Summer Temperature and Birth Defects” then good for you, because just such a work was put into the peer-reviewed journal Environmental Health Perspectives (2012 October; 120(10): 1443–1449) by Zutphen and others.

Zuthphen and her co-authors examined birth records for a number of years in New York, plugging temperatures at a “crucial” time in women’s pregnancies and presence or absence of 84 different birth defects, along with a limited set of demographics into a statistical model and concluded:

We found positive and consistent associations with congenital cataracts of multiple ambient heat exposure indicators, including 5-degree increases in the mean daily [universal apparent temperature] (minimum, mean, and maximum), a heat wave episode, the number of heat waves, and the number of days above the 90th percentile of UAT.

The temperatures were measured at airports, and women were “assigned” the temperature of the nearest airport where they resided when they gave birth. Only 18 stations for the entire state of New York were used. This of course means we have little or no idea what temperatures any of the mothers actually experienced during their pregnancies (“we were unable to incorporate air conditioner use data…”). So we have no idea if excessive temperatures would have caused any birth defects for these women.

But statistical methods are immune to these kinds of subtleties; p-values can be produced no matter what relation a hypothesis has to actual data on hand. In this case, p-values were so produced, and for a couple of the 84 birth defects, the p-values were small enough to declare a “statistically significant” relationship.

Now this relationship is only with a few birth defects and temperatures at airports “near” where women lived when they gave birth, which is only loosely (at best) connected to the actual heat the women experienced during their pregnancies. But that’s too complicated to remember, so it’s easier to say, as the authors do say, “We found positive and consistent associations between multiple heat indicators during the relevant developmental window [of mothers] and congenital cataracts”.

I gave up early on this one and leave it as an exercise for long-time readers to discover the many other flaws of this paper. I’ll just mention the second biggest—the first is the use of the ecological fallacy, as detailed above—which I’ll put in the authors’ own words:

Under the null hypothesis, we would expect 4 of the 84 effect estimates displayed in Table 3 to be statistically significant at the p = 0.05 level. Thus, significant positive and negative associations with cataracts, renal agenesis, and anophthalmia may have been chance findings. Bonferroni adjustment to the p = 0.05 level of significance (0.05/84 = 0.0006) would yield approximate adjusted CIs for congenital cataracts that include the null value (95% CI: 0.93, 2.44).

In other words, after using the proper (frequentist) measure, the results are not statistically significant, meaning the conclusion stated above is false. As in, Nothing to see here. No effects were found. The results were insignificant.

But, darn it, they could have been significant, and that’s what’s really important. Climate change is that sneaky. So much so that the authors plead, immediately after admitting there was no significance after accounting for multiple testing, “However, the associations with congenital cataracts are biologically plausible”.

There you have it: they didn’t find an effect, but they could have, and that’s what’s really important.

Thanks to Dr K.A. Rodgers for alerting me to this paper.

Categories: Statistics

17 replies »

  1. Living near airports produces birth defects. It is the noise. The unborn do not like it.
    The evidence is indisputable.

  2. I note that the “study” didn’t attempt to correlate statistics between other regions which might already (as normal) be experiencing the conditions which would be classified as “extreme” in the areas studied. Adaptation is a word missing from some researchers’ vocabularies. How is it that all species “studied” (for how long, and using what controls to provide a comparison?) seem to be currently teetering on the brink of extinction?

    I’ve just read (reports of, paywall as usual) of a study which purports to show that many mammal species are “at risk of extinction” from rising temperatures, increasing drought and more and possibly stronger cyclones. The study didn’t actually address whether those species were actually at risk from any particular degree of “extreme heat”, drought or cyclone damage, nor the timescale involved, nor the likelihood or intensity of any particular extreme.. About as useful as a study which shows you’re more likely to be run over if you step outside your front door, or more likely to drown if you swim.

  3. Hell, I’ll repost two links here as well:

    http://en.wikipedia.org/wiki/Data_dredging

    (This is a blatant example of data dredging and should never have been accepted or published except as a null result showing that there is no unexpected correlation between heat waves and any of the things studied. Other flaws I work through on the WUWT site.

    Then because it is a bit of a pain to read the entire wikipedia article on data dredging, here’s an xkcd link from the article that reduces the entire problem with it to jellybeans and acne.

    It is precisely, presciently correct, right down to the damn headline at the end.

    http://imgs.xkcd.com/comics/significant.png

    Beware green jelly beans. They are 95% likely to cause acne.

    rgb

  4. In other words, after using the proper (frequentist) measure, the results are not statistically significant, meaning the conclusion stated above is false. As in, Nothing to see here. No effects were found. The results were insignificant.

    And if we were proper Bayesians — as I am — we would use our prior knowledge of the prevalence of the birth defect in (say) south India and (say) the UK, observe that everybody living in south India spends the entire summer, every summer at temperatures that would count as a “heat wave” in the UK, note that the prevalences are the same within statistical noise in spite of what one would expect is better prenatal care and immunization in the UK, repeat the comparison a few dozen times (comparing prevalence in Florida to prevalence in Maine, for example, to better control for population wealth and health care services) and never formulate the hypothesis as written anyway because the Bayesian priors downgrade even a significantly positive result to nothing before you start. Prevalences tend to match up within 10-20% — there isn’t room for a 50% effect on something as simple as different ambient temperatures.

    The sad thing is that there is actually some reason to believe that hyperthermia causes the birth defect in question (and a number of others) based on sound research — a variety of fevers are almost certain to be causal with a much higher degree of confidence, including fevers associated with the flu or common cold. We can make that association because we have clinical data on the actual people who had the flu and how big a fever they had and how long it lasted and so forth. It is perfectly reasonable to hypothesize “environmentally induced hyperthermia in a critical window is associated with this birth defect”.

    The problem is then that to show the association, you have to have definite clinical knowledge of an episode of hyperthermia, e.g. heat exhaustion or heat stroke, per person in your study. This is because we are homeothermic creatures and under normal circumstances our internal temperature remains very close to 98.6, even when it is very hot outside, even when we are outside and it is hot outside. We sweat, we drink plenty of fluids, we seek out shade, we seek out fans or air conditioning. Symptomatic environmental hyperthermia is rare and associated with the very old, the very young, and the very sporty who participate in things like football practice in the middle of hot, muggy afternoons. Pregnant women could be vulnerable under some circumstances (playing tennis or golf on a hot, muggy afternoon without anything to drink) but this seems relatively unlikely, so unlikely that resolving signal from noise would require substantial knowledge of specific circumstances, not just “it was hot outside during the pregancy”. Nobody gets hyperthermia walking from an air conditioned house to an air conditioned car, when they are drinking plenty of fluids, when they are sitting in front of a fan in the shade, and we get uncomfortable long before we get hyperthermia and seek out air conditioning, fans, shade, ice cold drinks.

    rgb

  5. Well written, a max of statistical info with a minimum of numbers, what’s not to like. And as usual, Robert Brown adds lucid, learned commentary. Great stuff.

    Well, except for the insistence that we actually have knowledge of women suffering from hypothermia if the theory is that exposure to heat causes cataracts … that kind of focus on actual clinical evidence is so 20th century, don’tya know.

    In the brave new world of the 21st century, mere correlation at p=0.05 suffices as ironclad proof of the possibility of future calamity.

    w.

  6. I’ve been told by my University Alumni Association that cancer was diesel exhaust fumes that cause cancer.
    http://www.news.uwa.edu.au/201212115348/research/diesel-fumes-increase-risk-childhood-brain-tumours
    “The study found that fathers who worked near diesel-powered equipment including cars, trucks, other heavy machinery and generators at about the time of conception, had children with an increased risk of childhood brain tumour. There was also an increased risk for mothers exposed to diesel exhaust fumes any time before the birth of their child.”

    The abstract of the paper gives a clue as to the verity of the conclusions. (http://www.ncbi.nlm.nih.gov/pubmed/23184618)
    “Childhood brain tumors (CBT) are the leading cause of cancer death in children; their risk factors are still largely unknown. Since most CBTs are diagnosed before five years of age, prenatal exposure and early postnatal factors may be involved in their etiology. We investigated the association between CBT and parental occupational exposure to engine exhausts in an Australian population-based case-control study. Parents of 306 cases and 950 controls completed detailed occupational histories. Odds ratios (OR) and 95% confidence intervals (CI) were estimated for both maternal and paternal exposure in key time periods. Increased risks were observed for maternal exposure to diesel exhaust any time before the child’s birth (OR 2.03, 95% CI 1.09-3.81) and paternal exposure around the time of the child’s conception (OR 1.62, 95% CI 1.12-2.34). No clear associations with other engine exhausts were found. Our results suggest that parental occupational exposure to diesel exhaust may increase the risk of CBT.”

    Why use odds ratios? Use of odds ratios is a flag that the researchers have assumed distributions and that their conclusions reflect those assumptions.

  7. Long before the world becomes rational, humans will have to give up all belief in ‘experts’ or ‘authorities’. That is, they will have to become competent at epistemology. Until then, they will be at the mercy of absurd academic frauds throwing together esoteric rhetoric to support whatever mass hysteria is fashionable at any moment. So long as faith is a virtue, sense will not be.

Leave a Reply

Your email address will not be published. Required fields are marked *