Statistics

How Effective Was This Year’s Flu Vaccine?

New & Improved Infographics!

A reader asks, was this year’s flu shot effective? Short answer: Looks to be, but there is no definitive way to know.

In order to know definitively for any one man, we’d have to learn intimate details of his biology, whether he was exposed to the flu and the extent of that exposure (one measly virus or a full-sneeze-in-the-face load?), plus we’d have to know if the vaccine, if administered, took, and if so to what extent.

Since we can’t really know all that, even if we tried really hard or were very deeply concerned, we’re forced to look at statistics. Not too pleasant, that, but nobody (except the government) said life would be easy.

Our reader wondered about the vaccine after stumbling across Dr Brownstein’s Holistic Medicine blog. Holistic medicine works by waving energized (organic) crystals over (organic) pea puree before that substance is forcibly, but oh so naturally, injected into your (organic) lower alimentary canal. This procedure is guaranteed to cure any disease which would naturally clear itself.

Anyway, Brownstein read the CDC’s flu report (here) and was flummoxed. He said, “CDC’s articles are not for the faint of heart.” Yet he was able to conclude, “Maybe in the Land of Oz [the vaccine] is effective and should be expanded, but on our planet this vaccine is virtually worthless, especially for those who need it most—the elderly.” He also mentioned Oompa Loompas.

The CDC, God bless them, didn’t help themselves. Their article is as clear as any “written” by a committee of dozens of government employees. Which is to say, as clear as the tax code or a sheaf of Obamacare regulations.

They could have, but did not, just say, “M of N people who were vaccinated got the flu, and S of T people who were not vaccinated got the flu.” We could have then glanced at the difference in rates, at least for a start. Instead, the CDC told us how many in a sample of folks had or did not have the flu, and then revealed the total number of people vaccinated, summed across flu status. Useless.

In this sample we know 1,582 folks did not have the flu, and 1,115 did. We also know 1,160 of the 1,582 + 1,115 = 2,697 were vaccinated. So what? It could be that all of the 1,160 vaccinated avoided the flu, which would show the vaccine works. How many of those who were vaccinated got the flu? Doesn’t say; at least, not in the main text or first table.

Hence Browstein’s frustration. He gave up too early, though, for later on, buried in a secondary table (and remembering the first) are the required numbers. Of those who had the flu, 367 were vaccinated. Of those flu-free, 793 were vaccinated. Here is the 2 x 2 table, written in text form (this is from non-committee me, not the CDC):

     Vaccinated = Y, Flu = Y :: 367
     Vaccinated = Y, Flu = N :: 793
     Vaccinated = N, Flu = Y :: 748
     Vaccinated = N, Flu = N :: 789

Thus 793/1160 = 68% of those vaccinated did not get the flu, and 789/1537 = 51% of those not vaccinated did not get the flu. All other things equal, that improvement from 51% to 68% is the efficacy of the vaccine.

In other words, without the vaccine you had a a 50-50 shot of getting the flu; but with the vaccine you reduced this to 1 in 3. Conclusion? The vaccine is effective. Not perfectly, but somewhat.

Now the caveats, which can be ignored by the numerically squeamish. The sample came from people who serendipitously showed up to the docs seeking “care for an acute respiratory illness with cough, within 7 days of illness onset”. What kinds of people didn’t show? The healthy, the too-sick-to-come, the sick who hate doctors. Probably the healthy outnumbered the others, and since there is good reason to suspect the healthy were more likely to be vaccinated, the CDC sample underestimates efficacy.

The sample was taken across five states, and over several age groups, four races, still just two sexes , two virus strains, etc. Age meant much. The vaccine’s efficacy was about the same for everybody under 64, but it was minimal for those older.

There was about a 69% chance for vaccinated old people to get the flu, but a 72% chance for non-vaccinated elderly. Given all the vicissitudes of the sample, we can’t say with any kind of certainty whether those rates also applied to people not in the sample. (Clearly the vaccine was more efficacious than not for the people in the sample.)

So old people shouldn’t bother? Not quite. The vaccine looked not to work for Influenza A (H3N2), but it faired well against Influenza B, where it roughly halved the chance of developing flu via that strain.

Grand conclusion: the flu shot is a good bet.

Categories: Statistics

11 replies »

  1. It is hard to take these statistics seriously, since considerably less than 50% of the population gets the flue. If you are only going to look at people who contract the flue then a better measure would be the severity of the illness, or survival rate, especially in the elderly.

  2. I need more information to make a decision about vaccination. How many people got sick as result of being vaccinated, not necessarily with flu? As so often, life is a little more complicated than statistics.

  3. That’s a great article, it really shows how difficult a subject statistics is. There is a little video on TED.com where Art Benjamin, a mathematician describes how pre-university mathematics is mis-directed. He described the whole process as aiming at a pinacle and that that pinacle is calculus. But it shouldn’t be, the pinnacle should be staistics.

    I’m not ‘dissing’ calculus by the way.

  4. Vaccinations are not as effective as advertising leads you to believe. The shingles vaccine is another one that has about a 60% success rate at best.
    The vaccine is expensive (and yes, someone pays for it even if you don’t–insurance, the government, someone).
    If you are afraid of getting shingles, it’s may be worth it. However, before running out for a vaccine, you might want to check the effectiveness.
    I am not against vaccinations, I just think people have expectations from them that in reality are not possible. (My physician got the flu even when vaccinated, so he’s pretty lenient as far as need for the shot!)

  5. The vaccine works for me, I do not take it and I do not get the flu.
    Had the shot once, 2005, and had flu symptoms all winter.

  6. Thanks for the link Ray. Philip Alcabes has some good articles on obesity as well, a topic I’ve discussed here before.

  7. Even better is homeopathic “medicine” which probably doesn’t contain even a single molecule of the active substance. Another cutting-edge medicine from the 18th century.

  8. John–I had the same experience. I was doing child care and thought, well, I guess a flu vaccine would be in order. I had flu symptoms for months. Live and learn.

    Wayne-I consider homeopathic useful in the sense I know what I am Not getting in the product!

  9. Anyone have data on side effect problems in those getting the flu shot. Without these data we have no basis to make a risk-reward assessment of the shot.

Leave a Reply

Your email address will not be published. Required fields are marked *