Statistics

The Tyranny Of Models

From reader Alistair Haimes comes this note about the Imperial College model, a note which will illustrate the tyranny of models.

So, the errors in the Imperial College covid-19 model. It sounds complicated, actually it’s incredibly simple.

Their model (which isn’t actually that complicated, uses standard libraries to do the heavy lifting, quite properly: [github link]) is fetching the data for cases and deaths automatically from the ECDC dataset.

The issue is, though, that for Sweden and the UK (at least), the death figures will be wrong since the ECDC figures aren’t the actual number of people who die that particular day: they are the update to the cumulative totals which are announced each day.

So for example, one day the UK might get a big batch of results back from the labs, which they then allocate back to the days those people actually died, and they then publish each day’s actual deaths on the NHS website with the updated number of people who actually died on each particular day. Sweden works the same.

But the problem is that Imperial is taking the daily updates to the cumulative number as each day’s death. This means firstly that the trajectories are wonky, and secondly also that they are working off the wrong “base” numbers – in the UK for example we had a day recently where 980 odd results came back, which obviously will have looked like a bad “death day” and which the Imperial number will fit its curve too and use as a springboard for the future, but really only 730-odd people *actually* died that day. Today we announced about 750, but only 113 actually died yesterday, the rest are deaths going back weeks – and that 113 will get pushed up over the next few days (particularly next 2 days) as post-mortems and late filings and etc. come in for people who died yesterday. Recently a lot of results have been coming back, but the people actually died quite a while ago.

So they’ve built a very slick chained-Markov Monte Carlo model that produces beautiful output but is based on totally the wrong numbers.

It’s actually not that difficult to do a better pro forma: for my version I take each day’s actual death figures (updated daily from the NHS), inflate say the last 5 days numbers by the average that those numbers tend to get pushed up (which is pretty consistent), and you have a much more reliable dataset. Not perfect, sure, but better. At the pretty simple end of data wrangling.

Nuts eh?

Averaging this across all the countries in a big dataset would be fine, but it would be nice to try and get the data right for countries that are making policy decisions based on this model (I’ve been in lockdown for 3 weeks now largely off the back of it) but I’m whistling into the wind.

Haimes illustrates the difference between modeling the reporting process, which we’ve been doing, and modeling actual deaths (and cases etc.). It just isn’t the same thing.

A reporting model is useful for political projections, by which I mean as fodder for guessing what our elites and leaders will do. And what a panicked populace does.

An actual-numbers model is useful for politicians and elites to make plans. By which I mean, in what they will do to us.

Obviously—or I hope obviously—a reporting model doesn’t have to get the actual numbers right. It doesn’t even have to know anything about the actual numbers. It does have to (or should) get the reported numbers right.

But an actual-numbers model must get the actual numbers right, or else the plans our leaders make will be based on error. And therefore it’s likely the plans will be suboptimal.

Our strategy of PANIC! PANIC! PANIC! can’t be considered by optimal by anybody who is not a politician or elite seeking greater control.

Maybe you’re skeptical about the political control bit, preferring not to be as black of heart as your host. If so, take a look-see at this headline: US may have to keep some social distancing measures until 2022: study.

Subtitle: “Harvard researchers used computer models to simulate how the COVID-19 pandemic could play out.”

Uh huh.

There are still a few people amazed about models run on “computers”, as if the act of running models on a computer guarantees success. There are more who assign the same scrupulosity to Harvard products. (Yes, really.)

But this model is so asinine it doesn’t need refuting. Do these foolish academics really believe we have never had pandemics before and got through without worldwide government control? Don’t answer.

That is, answer this instead: do you think some power hungry elite will use Hahvahd’s model as an excuse to tighten her grip?

To support this site and its wholly independent host using credit card or PayPal (in any amount) click here

Categories: Statistics

26 replies »

1. Harry G says:

So are you saying that the Australian Health Minister is correct when he said”The only numbers I trust are Australia’s”?
Here are today’s numbers for review.

In Australia up to 3:00PM 17 April 2020
Number of people tested 391,000 (we have now tested over 1.5% of the Australian population)
Number who have tested positive 6523
Number who have died 65
Infection rate of those tested 1.67%
Death rate of those tested positive 0.996%
Death rate of all tested 0.017%
Fatality rate per 100,000 of Aussie population 0.264
Infection rate of Australian Population 0.027%
Current survival rate 99.0%
Number currently in a serious or critical condition 60 (0.92%)

2. Wow, so the common cold and flu will still be with us in 2022?
Inconceivable!

This has been a test of the emergency power maintenance system. This has only been a test.
We failed.

3. Sheri says:

“Don’t answer.” You take all the fun out of this……

Models are today’s psychics trussed up in silicone and metal. The models are oracles, but “scientific” ones (read as: political propaganda). In the Dark Ages, government-controlled oracles spoke and the panicked people, who were in a continual state of panic if possible, or at least in a state of fear, had to follow the pronouncements. Leaders, and paralyzing fear, demanded it. Today, oracle-worshippers flock to Flauci and Birx oracles, pleading for a glimpse of their what must be a horrible future. We ARE the Dark Ages all over again. and this pandemic of stupid proves it beyond a doubt.

As for FAKE numbers and using statistics to lie with (I’m sure that’s why they were invented. ): Unemployment numbers. UTTER LIE. It is NOT unemployment. It’s “forced confinement by a tyrannical governors” numbers. They people are NOT unemployed, they are prisoners of their wicked governors (except South Dakota–Kristi Noem is my hero! ) That IS their job. Being a hostage and prisoner. Honest reporting would make the government look bad, so the “oracle” of models calls Kristi evil and wrong and gets her berated in order to maintain order. “Unemployment” ended when the government took over the industries in their states and dictated who had jobs. Now, sitting at home collecting \$2400 a month extra is your job.

4. James says:

The IHME does the same thing when only cumulative death data is available (which is the case for many states). There are 100+ death days that are actually quite spread out, but the model considers them as one day, which can bump up the peak of the sigmoid fit.

5. Trust Harvard’s products? Only after they’ve been verified :). I’d trust Podunk U’s products first ;p.

About the petty tyrants and their lackeys? Of course they’ll try to hold on to whatever temporary power they have and grab as much more as they can get.

6. Fredo says:

I think the next logical step is to treat the virus as a crime scene and
follow it’s genetic footprint back to the lab it was conceived
in. I’ve read that this virus is very different from the handful of corona
viruses that recycle every year throughout the world. Much as the
anthrax used in the 2001 attacks was traced back to the lab at
Fort Detrick. Perhaps someone could come up with a model for that.
We may have to look no further than the Patent Office.
Just a hunch.

7. Becky says:

Designing anything requires information and experience, and being very careful when safety is a factor. The computer is not a magic box that fixes flaws or makes everything ok because the green light flashes when you hit compute. Computers produce pretty graphics.

I think the following could be used as an example of a well regarded expert’s mistake leading to catastrophic results (he will probably lose his license, either by law or because no one will use him again):

The NTSB concluded the Florida Pedestrian Bridge collapse was due basically to poor engineering. In a list of failures, for instance one states “FIGG Bridge Engineers’ analytical modeling for the bridge design resulted in a significant
underestimation of demand at critical and highly loaded nodal regions”, and another item from the list “FIGG Bridge Engineers’ failure to adhere to the Florida Department of Transportation Plans Preparation Manual requirements for a complex category 2 bridge structure within its work proposal to MCM, calling for an independent firm to conduct a comprehensive peer review, led to the inadequate peer review performed by Louis Berger, which failed to detect the under-design of the bridge” which shows how easy it is to go through motions of verifying something, and I’m not necessarily saying this guy is unusual or evil in the big scheme of things.

In an introductory statics class in college, the professor said the math is easy, it is figuring out the conditions/ loads that is the difficult part.

The difference is in the timeline of the disaster of the model. In the bridge, the trajectory of the calculations cummulated in a catastrophic collapse within seconds where victims could not escape, but in the case of the Imperial model the collapse is slow motion, and it is doubtful that anyone will lose their livelihood.

8. Dave says:

Interestingly, the only models that have actually been falsified at this point are ones that have definitely under predicted. Like say, models that estimated “~160,000” total worldwide infections: https://wmbriggs.com/post/29734/

Surely, other models have over estimated, and we can find evidence that some are unlikely to come to fruition, or of data errors, etc. But over estimation can’t actually be proven until it’s over.

9. “There are still a few people amazed about models run on “computers”, as if the act of running models on a computer guarantees success.”

But, but, flashing lights! R2D2! Whirring and whizzing! Heresy!

10. Kalif says:

Regarding the models, they are only as good as their data input. Dr. Briggs’ model uses non-lin. logistic distribution (derived from the log. fun.) because it may have been the best fit, or resembled that distribution the most at that time.

However, numerous variables that would actually refine the prediction are not taken into account (adjusted for) simply for the lack of any decent data. Using only mortality/infection rates in a model does not reveal anything about how this virus actually behaves in Physical/Chemical world. We also don’t have information on what viral load is necessary to get someone infected, who was where, at what time and with whom, etc. etc.

Most importantly, we don’t have info on what the spread would have been like, without all the measures taken so far (counterfactual).
Although, judging by some meat-packing plants, senior homes and such, we can get a pretty good idea.

@Sheri
“Now, sitting at home collecting \$2400 a month extra is your job.”

You are barking up the wrong tree. Here’s where you need to start:
https://www.whitehouse.senate.gov/news/release/whitehouse-doggett-release-new-analysis-showing-gop-tax-provisions-in-cares-act-overwhelmingly-benefit-million-dollar-plus-earners

11. Ken says:

Cannot help but notice the cult-like quality to popular beliefs re Coronavirus/Covid-19:

That here and many other places, as the infected & death rates seemed to be stabilizing (indicating defensive measures like distancing were working and the battle was being won) the number and type of counter-Covid-19 measures imposed from our hired help leaders starting increasing.

Masks went from recommended to mandatory, for example. And more… perhaps including economic destruction as a proxy for direct infections & deaths.

As if a belief in a terrible punishing (cleansing?) plague was immanent and then failed to follow thru.

The behavioral trend seems to be the same as observed in cult members when a hard prediction fails to pass — instead of acknowledging they’ve been duped and the faith is a fraud by a charismatic leader, they double down and believe even more.

SEE: “When Prophecy Fails,” in Wikipedia , subsection Conditions. Or, similar in many sites.

Modeling errors & fallacies aside, much of society is acting as if Covid-19 is an element of faith — with that element failing to fulfill prophecy.

12. Enough with the models!

Worst U.S. Job Losses on Record (Four Week Period)
Year Description Peak Jobless claims (4-wk total) % of U.S. Population
1975 Stagflation 2.24 million 1.0%
1980 Fed tightening (Volcker) 2.52 million 1.1%
1982 Double-dip recession 2.70 million 1.2%
1991 Early 1990s recession 2.00 million 0.8%
2001 Dotcom Bust 1.96 million 0.7%
2009 Great Recession 2.64 million 0.9%
2020 The Great Lockdown 22.03 million 6.7%

Nothing has ever utterly destroyed the American economy like the Great Lockdown. Nothing. Ever.
The worst loss of jobs in America, by a factor of 10.

Real numbers. Real lives. Real people. Not a Faucian bargain-basement model.

https://www.visualcapitalist.com/charts-historic-u-s-job-losses-perspective/

13. Briggs says:

Kent,

Because of massive job losses and the growing lack of food, and money to pay for food, I propose we gather up all Twitter Blue Checks and send them to Smithfields for processing and distribution to the poor.

This will, sadly, result in losing some good with the bad. Sacrifice is needed.

14. Kent Clizbe says:

“I propose we gather up all Twitter Blue Checks and send them to Smithfields for processing and distribution to the poor.”

Unfortunately, the Chinese own Smithfield now. And they’ve shut it down.

Blue checks? We don’t need no steenkin’ Blue checks!

https://youtu.be/u8WbLU958Lg

15. Josh Postema says:

“And the King shall answer and say unto them, Verily I say unto you, Inasmuch as ye have done it unto one of the least of these my brethren, ye have done it unto me.”

Well, we’ve crashed the world economy, which, if it remains for a while, will result in millions upon millions of deaths from starvation, drug overdose, and depression, not to mention the collapse of first-world medical research that could prevent future epidemics and help people live longer.

Your shallow piety is noted, however.

16. laffo says:

Josh:
“if it remains for a while, will result in millions upon millions of deaths from starvation, drug overdose, and depression”
‘Model’ ‘Projection’ or ‘Assumption’?

17. Excellent analysis of the Bill Gates vaccines connection by the redoubtable Richard D Hall, whose arguments are nigh on unassailable in my opinion.

I see he’s hosting his own videos lately having had a number of recent problems with YouTube (and Vimeo!) taking his content down for supposed “violations”

Please watch, listen and disseminate far and wide:

[video]https://www.richplanet.net/richp_genre.php?ref=282&part=1&gen=99[/video]

18. Kung Flu rule of sixes:
Only one in six will catch it.
Only one in six of them will become seriously ill.
Only one in six of them will become critically ill.
Only one in six of them will survive.

19. Fredo says:

Great graphic Kent: The arc of this entire episode, specifically the
lock down, has ploughed a fertile field of bankruptcy and foreclosure
never before tapped at these levels by the cartel. The only asset that
really exists and retains value no matter what happens is the roof over