Statistics

Selling Fear Is A Risky Business: Part II

Read Part I. Seriously. Read it. Not everything is easy. Today’s stuff is used to make decisions about your life, so pay attention.

Cue the organ… When we last left Tom, he was checking his albondigas for spots. He had read a breathless press report that the risk of cancer doubled by exposure to coriandrum sativum. I weep that nobody commented on this yesterday.

Now there’s doubling and there’s doubling. Moving from 1 in 10 million to 2 in 10 million is a doubling, but of an entirely different kind than in jumping from 1 in 2 to 2 in 2, i.e. 50% to 100%. Using relative opposed to absolute risk disguises this difference. There is no reason in the world to worry or fret over a relative risk of 2 when the probabilities are in the range of 1 in 10 million. But there’d be every reason in the universe to be vexed when jumping from coin flip to certainty.

Lesson one (again): never trust anybody trying to sell you anything using relative risk. Always demand the absolute numbers.

That 1 in 10 million seem low to you? It doesn’t to the EPA. In this guide (e.g. p. 5) they fret over tiny risks, and often reference 1 in a million and 1 in 10 thousand as regulation worthy. Let’s play: boost the exposed cancer chance to 2 in 10 thousand.

We now need a workable relative risk. Use 1.06, the high-water relative risk in a series of widely touted papers by Michael Jerret and others (more here, here, here, and here; EPA adores these papers). Jerret spoke of others diseases, but what matters is the size of relative risk deemed regulation worthy. My examples work with any disease. With a relative risk of 1.06, the chance of cancer in the not-exposed group is 0.000189.

Here is a picture of the probabilities for new cancer cases in the two groups in LA.

Figure 1

These are not normal densities: they are actual probability mass functions drawn with a smooth curve. These are non-zero probabilities for observable events.

There’s a 99.99% chance that from about 300 to 440 not-exposed people will develop cancer, with the most likely number (the peak of the dotted line) about 380. And there’s the same 99.99% chance that from about 340 to 460 exposed people will develop cancer, with the most likely number about 400. A difference of about 20 folks. Surprisingly, there’s only a 78% chance that more people in the exposed group than in the not-exposed group will develop cancer. That makes a 21% chance the not-exposed group will have as many or more cancerous bodies. Make sure you get this before continuing.

This not-trick question helps: how many billions would you pay to reduce the exposure of coriandrum sativum to zero? If it disappeared, there’d be a 78% chance of saving at least one life. Not a 100% chance. Pause and reflect. Even if you shrink exposure to nothing—to absolutely nothing—there is still a 21% chance (1 in 5) of spinning your wheels.

And there’s a cap. If we use the 99.99% threshold1, then the best we could save is about 160 lives. That comes from assuming 460 exposed people develop cancer and 300 not-exposed people get it (the extremes of both pictures2). The most likely scenario is a saving of about 20 lives. Out of 4 million. Meaning at best you’d affect about 0.004% of the population, and probably more like 0.0005%. How many billions did you say?3

There are strong assumptions here. The biggest is that there is no uncertainty in the probabilities of cancer in the two groups. No as in zero. Add any uncertainty, even a wee bit, and that savings in lives goes down. In real life there is plenty of uncertainty in the probabilities. We’ll see how this effects things in Part III.

Assumption number two. That everybody who gets cancer dies. That won’t be so; at least, not for most diseases. So we have to temper that “savings” some more. Probably by a lot.

Assumption number three. Exposure is perfectly measured and there is no other contributing factor in the cancer-causing chain different between the two groups. We might “control” for some differences, but recall we’ll never know—as is never know—whether we measured and controlled for the right things. It could always—as in always—be that we missed something. But even assuming we didn’t, exposure is usually measured with error. After all, how easy is it to track exposure? I’ll tell you: not easy, not easy at all.

So difficult is exposure to track that there is substantial uncertainty in any estimated environmental “dose”.4 In our example, we said this measurement error was zero. In real life, it is not. Add any error, even a little bit, and the certainty of saving lives necessarily does down, down, down.

I ask you: is it any wonder that those with something to sell not only speak in terms of relative risk, but also ignore the various uncertainties?


Part III. Boy are we unsure.

—————————————————————————————

1We could add some 9s to this and not change the fundamental conclusion.

2There’s only there’s a 0.005% chance that 460 or more exposed people get cancer, there’s a 99.985% chance at least as many as 300 in the not-exposed people get it.

3Same numbers for the state of California, which has about 38 million residents. 99.99% chance 6950 to 7400 not-exposed, and 7350 to 7850 exposed develop cancer. Most likely lives saved about 550. 99.99% cap about 900. I.e., roughly 0.0014% to 0.0024% of the population. Under perfect conditions with no uncertainty.

4It is only in rare laboratory experiments where the dose is known exactly.

Categories: Statistics

18 replies »

  1. The main problem is that all these risks are calculated in a vacuum as if the only alternative is immortality. They should be expressed as relative risks compared to the chance (there’s that word again) of dying from all causes as adjusted for age. This would put everything into perspective. At my advanced age, even compared to the venerable Briggs, this is a distressingly large number. There is also hormesis which may, and probably does, make all this low exposure nonsense irrelevant.

    This reminds me of a recent and annoying commercial that warns us that we may spend the last ten years of our lives in poor health unless we smarten up and live right. It is not clear what the alternative is supposed to be. Die young I guess. I much prefer this Guy Lombardo song that I remember from my youth.

    http://www.metrolyrics.com/enjoy-yourself-its-later-than-you-think-lyrics-guy-lombardo.html

  2. Scotian,

    Yes, one can “adjust.” But that is really beside the point here. Don’t be distracted by it.

    Don’t forget we are assuming we know, with certainty, what the real risk is. Even with certain knowledge, we are still uncertain.

    This is key. Next time we build risk/parameter/probability uncertainty into it.

  3. Well done! Finally, someone understands the hard thing about predicting outcomes from statistics is controlling the input factors. I am a 9 year lung cancer survivor and at diagnosis, the statistics of my survival (National Cancer Institute) was about 6 months. Cancer survivor statistics don’t factor variation (age, health state, and more) in the patients. They don’t even include a cause of death analysis. Have you read Dr. Stephen J. Gould’s paper “The Median Is Not The Message.” I really enjoy your site…..

  4. “That everybody who gets cancer dies. That won’t be so; at least, not for most diseases.”

    Not everybody who get’s cancer dies of the cancer, but they do all die from something and that is true for all diseases.

    The survival ratio for life is 0%.

  5. MattS, Scotian,

    Good correction. Everybody dies of something.

    So let’s leave it as “get cancer.”

    Point is, it doesn’t really matter. The example works for “get anything.”

  6. RE: “That 1 in 10 million seem low to you? It doesn’t to the EPA. In this guide (e.g. p. 5) they fret over tiny risks, and often reference 1 in a million and 1 in 10 thousand as regulation worthy.”

    REALITY (for EPA, etc. regarding things assessed to be harmful at some “dose”):

    What gets measured gets regulated.

    It’s really that simple. When new measurement capabilities are developed, new rules are concocted to take advantage of the measurement tool/method.

    ALSO:

    Note the implicit presumption/assumption:

    If something is toxic at some dose level, it is always toxic at all [measurable] dose levels.

    That ignores possible hormesis effects…or…simply ignores hormesis altogether. Hormesis defined: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2248601/

    Hormesis seems to apply to a significant proportion of substances — something like 15+% of compounds/chemicals (if my superficial recollection of the topic is anywhere close to accurate at the moment).

  7. Dr. Hans J. Eysenck wrote a paper on how many people die of cancer and concluded the number is unknown. The first problem is that death certificates are notoriously inaccurate. In one study comparing the death certificate to autopsy results, the death certificates were wrong 30% of the time. Sometimes even after the autopsy they couldn’t decide the proximate cause of death. Regard all cancer statistics with skepticism.

  8. Ray,

    “Regard all cancer statistics with skepticism.”

    Should read “Regard all statistics with skepticism.”

  9. Gary,

    Thank you for asking! It means she’s at low risk for cancer of the albondigas.

    (But she’d still be at low risk even if she liked it.)

  10. Briggs, RECALCULATE!

    http://www.epa.gov/nata2002/gloss.html

    >>1 in a Million Cancer Risk:

    A risk level of 1 in a million implies a likelihood that up to one person, out of one million equally exposed people would contract cancer if exposed continuously (24 hours per day) to the specific concentration over 70 years (an assumed lifetime). This would be IN ADDITION to those cancer cases that would normally occur in an unexposed population of one million people. <<

  11. Tom,

    >>1 in a Million Cancer Risk:

    A risk level of 1 in a million implies a likelihood that up to one person, out of one million equally exposed people would contract cancer if exposed continuously (24 hours per day) to the specific concentration over 70 years (an assumed lifetime). This would be IN ADDITION to those cancer cases that would normally occur in an unexposed population of one million people. <<

    Here is the rub, take any environmental contaminant you want to pick, what percentage of the population is actually exposed continuously for their entire lifetime?

    Hint: The answer is ~0.

    This drastically overstates real world risks.

  12. There must be a story behind the essential zero risk standard of one-in-a-million cancer risk. By its definition in Tom’s comment, now it’s clear what it means.

    For example, assume men have a 1 in 2 lifetime risk of developing cancer. Does it mean that actions should be taken if life time exposure produces a risk of 0.5 + 10^(-6) ? Or Does it mean that actions should be taken if life time exposure produces a risk of 0.5 + 10^(-6)*0.5?

  13. All,

    Quoting from the EPA document:

    For the ingestion, dermal, and inhalation pathways, toxicity criteria are used to define an acceptable level of contamination in soil, based on a one-in-a-million (10-6) individual excess cancer risk for carcinogens and a hazard quotient (HQ) of 1 for non-carcinogens.

    It’s quite right, as many are doing, to consider the time range. We should. But I’ve been ignoring it for the sake of example. Enormous difference in “risk over next year” and “risk over next thirty years.”

Leave a Reply

Your email address will not be published. Required fields are marked *