Z-Scores Are Misleading & Create Panic: Or, How Not To Compare Pandemics

A number of people panicked to the point their minds have turned to quivering jelly, angry at my calmness, have been emailing and tweeting me the equivalent of “Oh yeah, smart guy? If it wasn’t so bad, then explain to me why the z-scores are so high!”

This proves propaganda works. (Did you know our State Department boasts of its propagandistic abilities?) Not one of these people touting them could define what a z-score is, even if you threatened to take away their Netflix.

What has these folks frightened are scary graphs with big bumps issued by our official propaganda organs. People are told to shiver when gazing at them, and so they do shiver. We have become a very obedient people.

Yet z-scores can cause harmful misinterpretations in judging the severity of pandemics.

Let’s go through a simplified example to show this.

First, z-scores are the results of statistical models, built with an aim of hypothesis testing, which all regular readers know leads to grief. The first cousin of z-scores are p-values, and p-values should never be used. New readers won’t understand any of that, and you don’t need to for this demonstration. But if you’d like to learn, I have a free on-line class, dozens of articles and a book proving all this.

Second, here are simulated z-scores for deaths over a period of about 11 years: 10 previous years, and our incomplete year of doom and hysteria.

Look at that sucker soar! The other wiggles are nothing next to the whopper of a z-score for our year of doom. (I could have done this so that weeks lined with up with our current frenzy, i.e. the peak coming at 2020 week 14 or so, but I am lazy, and it makes no difference. R is a pain in the keister trying to label x-axes so that week 52 is in middle of simple plots.)

Anyway, there it is. The frightening z-score. If I saw that and had no clue what a z-score is I might say “Golly!”, too. Especially if I saw z-score plots across several countries, which appear confirmatory.

Now we’re using a simplified statistical model here, unlike some epidemiological models which can grow complex. The complexity is irrelevant, however. The interpretation remains the same. Our z-score is as simple as you can get: at each week, the observation minus the mean all divided by the standard deviation.

It really does appear, and furthermore it is true (since I picked the numbers) that this period is unusually high. We don’t need to test, because we know with certainty about the unusualness.

This is where most of these plots stop, leaving the viewer suitably awed. But a few go a little way further and show the excess deaths.

This is time series plot, where week 0 was about 11 years ago. Even looking at the excess deaths, it is clear this year is different from all the rest.

Our excess deaths are calculated with reference to the same model as above. Again, other models are more complicated, but the interpretation is the same. Excess deaths are, for each week the observation minus the mean. Simple.

So, once again, it is true that excess deaths are greater than usual, at least over these past 11 years.

Now the viewer is really shaking—or is it “literally” shaking? These excess deaths look the same at many different countries, too! This justifies the panic.

Regular readers will know the answer to this question: what do we call it when we look at the model and not the data, and then act like the model is the data? Right! This is the Deadly Sin of Reification, a terrible sin. We have reified the model, and forgotten reality. This is a common blunder made by scientists everywhere.

How about we now look at the actual deaths, from which the above plots were generated? Good idea:

There it is, the whole 11 years. There is a regular 52-week period, with deaths peaking once in every year, and then falling to a low later in each year. Only to repeat the whole thing the next year.

This, then, simulates a regular year, where flu deaths start ramping up around October, and falling off by the end of April. These are meant to be total and not just flu deaths. People do die of things besides viruses, contrary to media reports.

See the arrow at the end? And the little blip under it? That little blip represents the “excess deaths” which I added (20, 60, and 40 for weeks 38-40 in the final year).

These extra deaths happened, God rest their simulated souls. We are not trying to say they did not happen. They did. But put into context they are nothing to grow hysterical about.

“But Briggs, real data is noisier than that! This is cheating.”

It’s a simulation intended to show that one should not examine z-scores, because z-scores only show departures from the model mean and do not put the number of deaths into a reality-based context.

The reason z-scores look scary for COVID-19 is the same reason they looked scary here, the “excess” deaths are happening at an unusual time. Put into context, however, they are not as scary, as this post shows. Here’s the per-capita all-cause death data for the US:

(The fall off is late reporting by the CDC: read the original post for details.)

If coronavirus happened to coincide in timing with the flu, instead of lagging it a few week, the z-scores everybody shows would be much smaller even though the number of deaths would be the same. Z-scores are a matter of timing, because time is a component of the model.

Here’s the proof. I removed the excess deaths from the end and put them at the peak of the simulated flu, then recomputed the z-scores.

The panic-inducing powers of the z-scores have been rendered impotent. Same number of deaths. Nothing has changed except the timing.

The excess death plots are also cheating, when used as propaganda. You should not compare actual data with means. This was proved in this post. Look at the actual data first.

Here are the new excess deaths plot, with the simulated deaths coming earlier.

And the actual deaths:

You can’t even see the additions. If coronavirus happened at the peak of the flu year, both in China and everywhere else, maybe no one except for the people who keep track of the yearly flu and cold viruses.

Understand, coronavirus would still have been here and still killed the people it killed, just as flu was still with us and killed the people it killed. Only maybe the press would have turned their attention to propping up Joe Biden full time instead. Maybe our dear leaders wouldn’t have reacted to the virus.

Maybe, but, if. Like my dad always says, if ifs and buts were candy and nuts, every day would be Christmas.

To support this site and its wholly independent host using credit card or PayPal (in any amount) click here