Statistics

“Probably Fine” Isn’t A Number

What are those white spots?

Today’s headline is a true proposition. “Probably fine” isn’t a number, yet it is a perfectly reasonable way to communicate risk. Indeed most risk is, and should be, given in the form of words, or even vague thoughts.

What is to be discouraged is the relentless, brutal search for scientific-sounding precision which, unless the situation is rigorously defined, is absurd and leads to bad decisions.

The proof of this is easy. Take the context from where the quote originated, a Wall Street Journal article by a once-pregnant lady (Emily Oster) who was investigating the myriad restrictions and cautions issued to women who wish to take care of the lives growing inside them. Oster asked her doctor whether she could have “one or two glasses” a wine a week, to which the doctor replied that it was “probably fine.”

This led Oster to observe that “probably fine” was not a number, which was less than satisfying. She desired “real answers.”

In her pursuit of real answers, Oster discovered in medical journals that some deli meats had the risk of carrying listeria bacteria, which is bad.

I concluded that avoiding queso fresco and deli turkey was a good idea, but in the end I didn’t feel that it made sense even to exclude other deli meats. My best guess was that avoiding sliced ham would lower my risk of listeria from 1 in 8,333 to 1 in 8,255. I just didn’t think it was worth it. It would have made more sense to avoid cantaloupe.

Here are two “real” answers: 1 in 8,333 and 1 in 8,255. Each a nice quantification and, as promised, both give the feeling that science is happening. Both are also absurdly, unjustifiably precise.

These numbers were culled from various medical studies. Now any study is conditional (as all probability is conditional) on the kind and type of observations that were taken. For instance, in studies of deli meats containing listeria, there are the kinds of meats—all the different kind of hams, bologna, including that with and without olives or other stuffings, various types of turkey, salted beef, hard salamis, soft ones. There are different manufacturers—domestic, international, the actual plants, the carriers and methods of transport to the delis.

There are the animals used in the meats—pigs, cows, mystery meats, each of these grown on farms God knows where, each fed different foods, some genetically this way, others genetically that, some fed with antibiotics in various doses, some given hormones.

Then there are the delis—located in this and that neighborhood, kept under who knows what temperature control, selling meats fresh and past its sell-by dates, owned by fastidious shopkeepers or by corporations, managed by new hires and old. There are many more items easily added to the list which are is possible to imagine influence the chance a given piece of meat contains listeria.

In the end we have a piece of meat from a deli. Either it will have listeria or it will not (a tautology, therefore always true of any piece of meat). If it does, we look at all the characteristics listed above and put a check mark next to the ones which are true of this slice (from a pig, sold on a weekend, etc.). One of these characteristics might have been the cause of the meat having listeria, or maybe we missed measuring the cause, and that the characteristics we measured are only associated with the cause.

A second piece of meat won’t have listeria. Again we check off all the boxes, some of which will be also checked in the meat with listeria, some won’t. If boxes are checked for both pieces, it seems likely that those characteristics aren’t the one that caused or didn’t cause the bacteria to grow. But some characteristics will be checked on the bacteria-laden meat which won’t be checked on the clean meat, and vice versa. Perhaps one of these, or lack of one of these, is why the meat got infected. It is only “seems likely” and “perhaps” because we have to recall that we might not have measured all the right things.

In the end, we have to pick some characteristics and eschew others; we have to settle for summaries. For example, in one experiment we might find a greater frequency of turkey slices than ham slices contained listeria. The risk of eating turkey or ham can thus be given a precise number. But because this summary necessarily ignores many characteristics, and we are never sure we have thought of every relevant one, the quantification is only meaningful if it is certain that characteristics we ignored or didn’t measure where not involved in listeria production.

Since we are not certain, the quantification is the wrong number: it is not the real answer. It is misleading, it is too sure. Better to round to the nearest order of magnitude. Or say something like “Eating a slice of deli ham is probably fine.”

And we haven’t even added the layers of complexity which comes from eating the meats! Some people can eat the bacteria-laden meat and never develop symptoms, others need only a fragment to become ill. Oh my! It goes on and on.

And so have I. So I’ll stop.


———————————————————————–

Thanks to reader Jim Fedako for suggesting this topic.

Categories: Statistics

19 replies »

  1. avoiding sliced ham would lower my risk of listeria from 1 in 8,333 to 1 in 8,255.

    Surely 1 in 8333 is a lower risk than 1 in 8255? Not much point in turning your risk estimate into a number if you don’t which one is bigger.

  2. Quoting from your article: “owned by fastidious shopkeepers or by corporations”.

    This is a rather invidious comparison. It is the corporations that have largely driven out the greasy spoon food joints or at least compelled them to adopt high levels of hygiene and quality in order to compete. The first step in this direction was taken by Duncan Hines.

    http://en.wikipedia.org/wiki/Duncan_Hines

    Quotations of his:

    http://www.evancarmichael.com/Famous-Entrepreneurs/4139/Duncan-Hines-Quotes.html

  3. While your statitical argument is true, I wouldn’t be surprised if ham tended to be bigger problem than salami.

    Or orange cat loved lunch meat. When I made lunches he would beg for ham; I would give him a piece. He would not eat sliced ham that was more than about 3 days old even though it looked fine to me. (I cannot detect the deterioration for about.. oh… a week. But the cat could). My rule was: Ham the cat won’t eat should not be put in hubby’s sandwich. It is ‘possibly not fine’. I then developed the habit of never buying more than 3 days of sliced deli meat.

    On a less cat-oriented note: My impression is that food surfaces exposed to air tend to deteriorate more quickly than interior surfaces. Moister (as in higher water content) meats are likely to deteriorate faster than drier meats. So, the issue with sliced deli meats can be related to the convenience of using pre-sliced meats and moister deli meats. Ham is often pre-sliced and lean ham is often moister than things like salami.

    Plus, a cat who begs for fresh ham rejects it if it’s more than 3 days old. To me that’s a better indicator than statistics.

  4. Lucia,

    I bet a dog would eat it. Dogs will eat just about anything, the more disgusting the better.

  5. At some point people need to be accountable for interpreting things for themselves. To address the extent to which, or limits of, something can easily get ridiculous–and lawyers are always ready to pounce, for example who wudda thought that playing the game of football might lead to concussions:

    http://www.girardikeese.com/Articles/Helmet-Maker-Riddell-Found-Liable-For-Failure-To-Warn-Of-Concussion-Risk.shtml

    Actually, the case cited in the above link suggests the helmet manufacturer may have skimped on some required testing, or on conveying certain results…that’s a bit unclear, so the verdict might actually make sense as a punitive strike.

    The hair-splitting about statistical precision, or lack thereof, ballyhooed about in this essay is, in its way, getting rather lawyerly.

  6. Scotian: Actually, I have the same rule as lucia, only it’s my dog that refuses to eat the meat. Some of my dogs would eat anything, but the current one is more particular. Guess it depends on the critter.

    Since what we are looking at here is just whether or not to eat lunch meat, there’s nothing wrong with skipping it if you have reason to doubt that it is good. I think that was the point–probably good or bad is close enough in most cases. Not everything is easily quantifiable, even though people seem to expect it to be.

  7. When I was in the Navy we used to eat green weenies and beef with a rainbow sheen on it. It didn’t kill us so it was evidently fine.

  8. Just use sugar, salt, vinegar, or whiskey as a condiment. If you use enough you won’t have to worry about bacteria. Go ahead and try it. Statistically it may (or may not) be good enough for you.

  9. I had a cat that wouldn’t eat anything from the dinner table (or “people” food, in general). Wonder what that meant. OTOH, I didn’t want to eat what the cat liked.

    —-

    I think from now on whenever I’m asked “How are you?” I’m going to reply “Probably fine.”

  10. On the subject of listeria on deli meat – the hidden variables are the level of contamination and your susceptibility to it. Contamination is not an either/or proposition; for example, anything prepared in anyone’s kitchen is loaded with bacteria, no exceptions. Normally, stomach acid and the immune system deal with it.

    Similarly, the earth in your garden has gold in it. No doubt at all about that. But is it any use to you?

  11. VXXC – Re .22 vs. .45:

    A lot of the usual B.S. in that link.

    FBI did a proper study (they had access to “real shooting data” — “real people getting shot with real bullets” kind of “shooting data” that’s hard to replicate in pretend tests): http://www.firearmstactical.com/pdf/fbi-hwfe.pdf

    Bottom line: Something critical has got to get hit for a shot to matter; where the bullet goes matters more than the caliber of bullet that went there.

  12. VXXC initiates an interesting study that illustrates “models vs. reality” (one of my pet peeves here):

    The link provided concludes that ‘statistically the .22 is better than a larger caliber’ for self-defense.

    FBI (my input above) has all that data & more & instead looked at the physics of what it takes to actually immobilize a human target–and what does it take to get a bullet there. That approach leads to a very different answer — and the physics involved shows a .22 very often cannot possibly do the required job.

    The statistics approach suggests a .22 is safer, overall, for those confronted. BUT, the FBI analysis shows that if you are confronted your best chance of survival in that very confrontation, not overall but the very next time, is with a non-.22 caliber defensive weapon — because the laws of physics say so — and those are the only laws we always obey.

    Statistics are good to a point.

    But …. physics ALWAYS prevails.

    Any statistical analysis done in ignorance of the relevant physics is at risk of being irrelevant to what really matters–the mathematical consistency & certainty facilitates false confidence in the conclusions.

    The philosophical approach is even worse for the same fundamental reasons & more.

  13. Fletcher: Very true. When Lucia said she did not put the meat on hubby’s sandwich, my first thought was “I would”. Now, before you all think I’m some kind of sadist or black widow, what Fletcher said applies here. My hubby has rarely gotten sick from eating any food. On the other hand, I got food poisoning big time from hamburger that was not properly cooked–was sick for three days. He got heartburn for a few hours from it. 🙂
    So the person’s tolerance is a big factor.

Leave a Reply

Your email address will not be published. Required fields are marked *