Selling Fear Is A Risky Business: Part II

Read Part I. Seriously. Read it. Not everything is easy. Today’s stuff is used to make decisions about your life, so pay attention.

Cue the organ… When we last left Tom, he was checking his albondigas for spots. He had read a breathless press report that the risk of cancer doubled by exposure to coriandrum sativum. I weep that nobody commented on this yesterday.

Now there’s doubling and there’s doubling. Moving from 1 in 10 million to 2 in 10 million is a doubling, but of an entirely different kind than in jumping from 1 in 2 to 2 in 2, i.e. 50% to 100%. Using relative opposed to absolute risk disguises this difference. There is no reason in the world to worry or fret over a relative risk of 2 when the probabilities are in the range of 1 in 10 million. But there’d be every reason in the universe to be vexed when jumping from coin flip to certainty.

Lesson one (again): never trust anybody trying to sell you anything using relative risk. Always demand the absolute numbers.

That 1 in 10 million seem low to you? It doesn’t to the EPA. In this guide (e.g. p. 5) they fret over tiny risks, and often reference 1 in a million and 1 in 10 thousand as regulation worthy. Let’s play: boost the exposed cancer chance to 2 in 10 thousand.

We now need a workable relative risk. Use 1.06, the high-water relative risk in a series of widely touted papers by Michael Jerret and others (more here, here, here, and here; EPA adores these papers). Jerret spoke of others diseases, but what matters is the size of relative risk deemed regulation worthy. My examples work with any disease. With a relative risk of 1.06, the chance of cancer in the not-exposed group is 0.000189.

Here is a picture of the probabilities for new cancer cases in the two groups in LA.

There’s a 99.99% chance that from about 300 to 440 not-exposed people will develop cancer, with the most likely number (the peak of the dotted line) about 380. And there’s the same 99.99% chance that from about 340 to 460 exposed people will develop cancer, with the most likely number about 400. A difference of about 20 folks. Surprisingly, there’s only a 78% chance that more people in the exposed group than in the not-exposed group will develop cancer. That makes a 21% chance the not-exposed group will have as many or more cancerous bodies. Make sure you get this before continuing.

This not-trick question helps: how many billions would you pay to reduce the exposure of coriandrum sativum to zero? If it disappeared, there’d be a 78% chance of saving at least one life. Not a 100% chance. Pause and reflect. Even if you shrink exposure to nothing—to absolutely nothing—there is still a 21% chance (1 in 5) of spinning your wheels.

And there’s a cap. If we use the 99.99% threshold¹, then the best we could save is about 160 lives. That comes from assuming 460 exposed people develop cancer and 300 not-exposed people get it (the extremes of both pictures²). The most likely scenario is a saving of about 20 lives. Out of 4 million. Meaning at best you’d affect about 0.004% of the population, and probably more like 0.0005%. How many billions did you say?³

There are strong assumptions here. The biggest is that there is no uncertainty in the probabilities of cancer in the two groups. No as in zero. Add any uncertainty, even a wee bit, and that savings in lives goes down. In real life there is plenty of uncertainty in the probabilities. We’ll see how this effects things in Part III.

Assumption number two. That everybody who gets cancer dies. That won’t be so; at least, not for most diseases. So we have to temper that “savings” some more. Probably by a lot.

Assumption number three. Exposure is perfectly measured and there is no other contributing factor in the cancer-causing chain different between the two groups. We might “control” for some differences, but recall we’ll never know—as is never know—whether we measured and controlled for the right things. It could always—as in always—be that we missed something. But even assuming we didn’t, exposure is usually measured with error. After all, how easy is it to track exposure? I’ll tell you: not easy, not easy at all.

So difficult is exposure to track that there is substantial uncertainty in any estimated environmental “dose”.⁴ In our example, we said this measurement error was zero. In real life, it is not. Add any error, even a little bit, and the certainty of saving lives necessarily does down, down, down.

I ask you: is it any wonder that those with something to sell not only speak in terms of relative risk, but also ignore the various uncertainties?

Follow @mattstat

Part III. Boy are we unsure.

—————————————————————————————

¹We could add some 9s to this and not change the fundamental conclusion.

²There’s only there’s a 0.005% chance that 460 or more exposed people get cancer, there’s a 99.985% chance at least as many as 300 in the not-exposed people get it.

³Same numbers for the state of California, which has about 38 million residents. 99.99% chance 6950 to 7400 not-exposed, and 7350 to 7850 exposed develop cancer. Most likely lives saved about 550. 99.99% cap about 900. I.e., roughly 0.0014% to 0.0024% of the population. Under perfect conditions with no uncertainty.

⁴It is only in rare laboratory experiments where the dose is known exactly.