Remember the old stock scam? Call up 1,000 people and tell half Stock A will rise, and tell the other half it will fall. Five hundred people will be slightly impressed by your picking prowess.
Next day call those 500 and tell half of them the stock will rise, and tell the other half it will fall. You’re down to 250, but you have two hits in a row with these people.
Now you can dump the 250 that heard the wrong advice, but you can also try them again with the third call. These folks won’t be as impressed by the results, but why throw away good phone numbers? Use the same routine and tell half of whomever you call the stock will rise, et cetera.
Keep this up and after a few days you have an audience primed to hang on to your words. Only now you’re charging for them.
Charge whatever you like! Justify your large fee on the research that you put into your picks. Time is money. Only be sure to append this notice about side effects with every forecast: past performance is no guarantee of future results; plus, you might go broke.
Keep that in mind as you consider the nifty new paper: “New drugs: where did we go wrong and what can we do better?” It’s in the British Medical Journal by Beate Wieseler, Natalie McGauran, and Thomas Kaiser.
Gist: “More than half of new drugs entering the German healthcare system have not been shown to add benefit.”
Between 2011 and 2017, IQWiG assessed 216 drugs entering the German market following regulatory approval—152 new molecular entities and 64 drugs granted a new indication. Almost all of these drugs were approved by the European Medicines Agency for use throughout Europe. Thus our results also reflect the outcome of European drug development processes and policies.
Only 54 of the 216 assessed drugs (25%) were judged to have a considerable or major added benefit. In 35 (16%), the added benefit was either minor or could not be quantified. For 125 drugs (58%), the available evidence did not prove an added benefit over standard care for mortality, morbidity, or health related quality of life in the approved patient population (fig 1). Table 1 provides examples of assessment outcomes in the different categories of added benefit. As the effects of drugs often vary between patients, there might be subpopulations benefiting despite no relevant effects in overall study populations. However, IQWiG already considers subgroups by age, sex, disease severity, and further disease specific factors. Of the 89 drugs with an added benefit, 52 (58%) showed an added benefit in the whole approved patient population, and 37 (42%) had an added benefit in only part of the approved patient population.
The IQWiG is the Institute for Quality and Efficiency in Health Care. Figure 1 tops the post. That 25%, or 54 drugs, are busted out into “considerable” (32) and “major” (22) effects.
In any case, the 75% of drugs showing little or no benefit, or even “negative” benefits, are the key. How could such a thing happen, when the people putting out these drugs and molecules are surely smart?
Wait! Don’t answer yet! Because I also have to layer some p-values and parameters on you. I haven’t read every paper introducing the “effect size” of every molecule, but I’d bet that most, and probably all, the evidence showing efficacy is p-value or parameter based. Meaning it is exaggerated—and by a lot. I’m not going to rehash why for the eighty-two millionth time here. Read “Reality-Based Probability & Statistics: Ending The Tyranny Of Parameters!” for why.
The point is that even the 25% is too high. The real number is—and this is a wild guess based on my experience—half that, or less. Just for argument’s sake, make it 10%. Ten percent of all new drugs that make it trough the research or regulatory process are good—and really only in oncology, says the evidence—the rest mostly useless or even harmful.
How could this happen? The authors of the paper say more or better regulations are needed. Some of that is surely true.
On the other hand, we also have our stock example married to over-certain statistical methods. The analogy is the researchers selling themselves their own stock tips, in part. In mistaking statistical correlation for cause.
Researchers start with a bunch of chemicals and start testing them. Every time classical statistical methods are used to infer correlation is causation to winnow down the list for the next stage in testing, over-certainty is created.
Some molecules which are only correlational eventually make it through, like the stock picks, and because correlation and causation are sometimes married, some causative molecules pass, too.
It had to be this way, especially because causation is becoming harder and harder to identify. All or most the low hanging molecules have already been picked. The work required to find something truly useful, and not overly harmful, will only increase.
To support this site using credit card or PayPal click here
Great read. Sad it’s happening in US w/FDA. Antibiotics are a prime example yet the new molecules are developed by small biopharm who can’t grease the wheels or afford the additional CRL fees. Check “collapse of antibiotic” articles. Ex below
Sue: I don’t buy the conspiracy nature of your comment. Truly usable drugs, even things like fish oil, DO make it through. Laziness is the major reason none of these ever make it—it’s a fight all the way and too many people don’t care to fight. It may be easier for large corporations, but if I found a drug that cured west nile or necrotizing fasciiatis, I guarantee I’d fight to my last breath and hire someone to continue to fight thereafter if needed, until it was approved. I’d use whatever I could—ads, talks, etc and pages and pages of documentation put out there on the internet. Heck, I’d drop pamphlets from a plane if necessary. If these antibiotics, etc, do exist, why are the creators not screaming from the rooftops, presenting the irrefutable evidence, etc? What, no GoFundMe pages?
The only US drugs in the past 10 years or more that were actually helpful were the one that cured Hep C and the topical for eczema that has no steroids. The rest are cosmetic meds for skin conditions (vanity drugs sold as being for “major” skin conditions but sold to people with MINOR skin conditions and MARKETED THAT WAY) at a cost of $40,000 a year, insulin that shows no better performance than older ones, but coupled with a fancy pen can be sold to millions who fear needles at three times the price of the older versions, drugs marketed by CHILDREN (climate science is not the only field to use children for fear—children now check the internet to find out what drugs their parents need—see Phil Nickelson’s daughter for example. At least that’s what the child-abusing commercial says) and so forth. However, the Europeans don’t have the filthy drug ads that lie as we do here, so I am guessing their doctors are naive fools that listen to marketers and don’t read the literature or don’t understand it if they do. (I will say I have far more extensive reading on drug development than my doctor—he listens to his drug rep. Scary—yes. Then again, he’s not the one taking the drug, I am, so my stake is much higher than his.) I know advertising intensifies the low-hanging molecules phenomena, but I’m not sure what does that in Europe.
As for p-values, like global warming, p-values constitute a religion and one not easily given up. Especially when you call the religion “math” or “science”. An added reason to cling to the lie.
I would never claim to understand P values as you do, that’s why I read your blog. However, there is a significant issue in the antibiotic market. Granted, it is multi factorial but due to FDA regulations, FDA costs, stock market and many other aspects, we are in serious trouble. There are serval drugs in development but many of them are old molecules with a new ad on or a different ad on so not truly new hence causing resistance early on. Like you, I am a patient and I actually have been impacted by the cost of insulin. I don’t say this as an industry person I say it as a patient who is concerned as to what is happening. Regulations would prohibit an antibiotic co going out to promote funding etc prior to approval. As I said, this specific issue is complex. I would actually love for you to dig in more and then do a blog about this problem. Antibiotic resistance is on the rise while new, truly unique antibiotics are not.
Antibiotic resistance has been a problem for tens of years.
The situation is imporoving. Not just by the use of new antibiotics. Prevention is better than cure.
There are other tecniques, UV, used years ago by phsyios, now used in thatres.
The threat is being responded to and is keeping pace. Not for those who deveop post operative infection but there is also the problem that insulin dependant diabetes affects the immune system. Healing is slowed potentially in all disease types.
Decluttering wards, use of disposable curtains and linen, improved antimicrobial flooring and surfaces, where possible, visitor use of hand snitisers, improved staff awarness of infection control matters, all go to help with the reduction of the numer of cases of MRSA and other types of anti-biotic resistance.
The pitcure is not as black as s painted. As usual.
The new threat is the rise of TB, once again.
The point is that the battle is never won, it goes on all the time, always did.
Catch with antibiotic resistance is that it has always been there. Mass use selects for varieties that are sufficiently resistant to the currently used chemicals. Remember also that “nature” produced most of them for us. We mostly modify them to get around resistance to the most currently used ones. Occasionally truly new ones get made; yet no chemical man can make cannot be made by the rest of nature, and vice-versa. Plus, everything material is a chemical or a mixture of chemicals, some more organized than others.
Also, seemingly forgotten, is that within group variability is often greater than between group variability. And finally, the law of diminishing returns still exists. Most chemically mediated actions/reactions follow the curve called the logistic function, which is a sum of two exponential functions.
…er…Joy I believe “tens of years” are called decades!…or did I just miss a joke?
Yes! I forgot. I’ll think of a joke in about two weeks…
What’s the difference between a black rino and a white rino?
Well the black rino has a pointy jaw and the white one a wide squarer jaw.
When somebody told me that the other day, in serious mode, interesting fact, I couldn’t stop laughing. It was my Mum, my Dad and I were in big trouble. You had to be there.
“They’re both grey!”
Low-Hanging Molecules Have Already Been Picked
Epidemiology has a similar problem. The easy and obvious problems (diseases, syndromes, whatnot) have already been discovered. But the number of epidemiologists has increased and must publish to survive. So, we see questionable findings pulled out of noise levels. Using p-values of course. No one needs a statistical study to know that being shot in the head is bad for your health. But you need one to conclude that X is linked to cancer in 0.00004 % of the population and follow-ups for verification are rarely done. The sad thing is that X is banned based on these worthless studies.
Sheri, for what it’s worth:
I’d be interested to hear about the fasciitis problem if you want to mail me.
There isn’t and won’t be (I don’t think), a drug for that. It isn’t incurable. A multi pronged approach, as usual and a dedication to the right kind of specific exercise can make an enormous difference. (Necrotic tissue due to lack of perfusion and or infection needs removal).
I have noticed that the cause is not purely bio-mechanical specifically in plantar fasciitis, but systemic, probably immune system mediated. In particular when the cause appears not to be easily pinpointed to a mechanical stress, and where onset was sudden or without a trigger, (very common). In non diabetic patients I have also noticed multiple cases occurring in close groups and perhaps at the times of the season change. Made me also wonder if common viral stresses might also be a precipitating factor. If there’s ever a complex interrelationship it is between the nervous system and immune system. Needs bigger brains to help in research, not peanuts from the gallery. Less statistics, more critical thinking and working closer with clinical people.
American medicine is a nest of cartels.
The doctor part of the cartel is explained here:
https://www.politico.com/agenda/story/2017/10/25/doctors-salaries-pay-disparities-000557
tl;dr: Doctors have to serve apprenticeships called “residencies”. Residencies are paid for out of federal medicare. Panels of doctors & bureaucrats decide how much medicare money which defines numbers of residencies that ripples back into limiting medical school openings. Restrict supply drive up price. Classic cartel.