I have long told you that when drug companies produce statistics, for the consumption of regulatory agencies or the public, their products look better than the same statistics produced by others, or on trials when the drugs are in the wild.
Partly this is because of the greater care pharmaceuticals take in running their clinical trials. It’s true. They are forced into this vigilance by regulatory agencies. I do not mean the results are therefore more likely to be correct. I mean trials proceed in a more orderly manner, including the writing of the statistics at the end, that the process is better controlled for them than in teams of researchers laboring away in the hope of papers.
Drug companies know how to get wee Ps better than anybody, since it is their business. Regulatory agencies require wee Ps, you see. Alas, wee Ps are proof of nothing. Nothing having to do with whether a drug works, or it causes harm, or is useless, or anything like that. I know I am almost alone in saying this. For now. Maybe after reading this, you will join me.
In the speech I gave to the at Hillsdale (blog/Substack), which included many nervous scientists who met beforehand to plan a strategy of reaction, I remarked, “The British Medical Journal 2017 review of New & Improved cancer drugs found that for only about 35% of new drugs was there an important effect, and that ‘The magnitude of the benefit on overall survival ranged from 1.0 to 5.8 months.’ That’s it. An average of three months.”
In other words, two-thirds of new drugs that waved their wee Ps in the faces of regulators failed when released into the wild. And the ones that succeeded provided a barely there effect.
This is only one of many similar stories of what happens when drugs are out of the hands of manufacturers and put into the hands of the people who you ask if the drugs are right for you. The well known John Ioannidis examined forty nine top medical papers and found: “…7 (16%) were contradicted by subsequent studies, 7 others (16%) had found effects that were stronger than those of subsequent studies, 20 (44%) were replicated, and 11 (24%) remained largely unchallenged.”
Today’s headline: “Drug Trials Funded By Manufacturers Find 50 Percent Greater Drug Effectiveness“.
Drug studies sponsored by drug manufacturers tend to report higher drug efficacy than studies not sponsored by the drug company, a new report published in the Journal of Political Economy on Oct. 7 finds.
The report found a “sponsorship effect” that tends to bias sponsored studies toward reporting higher drug efficacies. The author could not find differences in study design between those funded by drug companies and those not.
“Removing the sponsorship effect would reduce the difference in efficacy … by about 50%,” Tamar Oostrom, an assistant professor of economics at Ohio State University, said in her paper.
“This effect was larger than I expected,” Oostrom told The Epoch Times over email. “My results suggest that sponsored arms of trials should be discounted substantially.”
Here’s a link to the paper itself. Oostrom reminds us “Trials are also expensive and risky investments. The average cost of a late-stage clinical trial is $35 million, an estimated 70% of trials are funded by industry, and the pharmaceuticals market in the United States alone is valued at $480 billion”.
I won’t bother us with the details of the paper, which is a dry read. His conclusion, which can be no surprise to anybody involved in clinical trials: “I assemble new psychiatric trial data to estimate that a drug appears substantially more effective when the trial is sponsored by that drug’s manufacturer, compared with the same drug tested against the same combination of drugs but without sponsorship.”
As I’ve said a hundred times, every scientist believes in confirmation bias. But they also believe it only happens to the other guy.
Then there’s this new JAMA paper: “Payments by Drug and Medical Device Manufacturers to US Peer Reviewers of Major Medical Journals“. As summarized by one doctor, “Peer reviewers for the major medical journals received $1.2 billion from drug and device companies from 2020 to 2022. 59% of all reviewers.” He adds, “Peer review is dead.”
It was never alive, my friend. Getting one to three people in all the world to say “Go ahead and publish” a paper was never a good system to lead to Truth. Some researchers instead put hope in “conflict of interest statements”. Which are useless when so many are on the take. And, bizarrely, because money coming from the government is never seen as tainting. Boy, is that wrong.
Subscribe or donate to support this site and its wholly independent host using credit card click here. Or use the paid subscription at Substack. Cash App: \$WilliamMBriggs. For Zelle, use my email: matt@wmbriggs.com, and please include yours so I know who to thank.
Have read some recent stories that Ivermectin is effective against osteosarcoma (and some other cancers?). The $35 million wall blocking a drug trial study seems to render our medical research system a hopeless morass if only profit seeking corporations have the funds to examine potential cures. A friend is in much pain now from bone cancer and says Ivermectin is helping relieve the pain from his lesions. How may we address the problem of removing the profit incentive from Big Pharma? Is there some way to publicly fund research for inexpensive cures and how could worthy studies be identified?
Ivermectin versus CoVid would certainly be another example.
Briggs, I bet the authors mentioned in the post also used p-values to make conclusions. Nonetheless, their conclusions support your viewpoint, making them valid regardless of sample sizes, study designs, and so on. Lol.
—-
If I were paid to review a paper, I would be more willing to pay closer attention to all the details and spend more time providing constructive feedback.
I assume that researchers asked to review papers are more established and thus have received more funding from various agencies. The issue arises if referees are asked to review studies related to their sources of funding, for example, the studies are supported by their funding agency or the agency’s adversaries. Is there evidence of the issue?
Peer review is still a crucial aspect of research. And post-publication reviews occur. It’s the reason why fraudulent papers are retracted. Journal editors and reviewers volunteer their time, and if the publisher truly values researchers’ time, they should compensate editors and reviewers for their efforts. Unfortunately, this is not the case. Publishers are happy to make money, and nerdy editors and reviewers are often just happy to learn from others’ research and contribute to the scientific community.
Shawn:
It’s very simple, but won’t happen. Eliminate all subsidies for research and all but the most basic regulations on medicine, medical practices, and insurance. That includes the FDA and CDC – gone both. Let the free market prevail.
Pk,
Here is a withdrawn clinical trial of “Ivermectin and Pembrolizumab for the Treatment of Metastatic Triple Negative Breast Cancer.”
https://www.cancer.gov/research/participate/clinical-trials-search/v?id=NCI-2022-02421
Next, of course, you can google why a clinical trial might be withdrawn.
Clinical trials are typically funded by the pharmaceutical industry. It’s important to consider the role of the FDA in ensuring the safety and efficacy of drugs. Just imagine what big pharma would do without the FDA. Things are not as simple as we’d like them to be.
JH:
That the FDA should be abolished does not mean that there should not be proper oversight. Independent test agencies along the lines of UL (Underwriters Laboratories) would be an option. Or, 50 state run departments. We got along fine before the FDA started punishing Amish dairy farmers.
All,
We’ll get to this in the class, but suppose, like here, the finding is something like of 10 new drugs, 8 were later found to be bad, and 2 not as good as claimed. Then suppose, as usually happens, that observed data is “confirmed” with a wee P.
The wee P is obviously superfluous, useless, misleading, wrong. The observations stand.
The cause is sought. What can it be? Well, one possibility is the obvious one. And that the wee Ps that “confirmed” the initial findings (which claimed all 10 were good) was wrong.
How about just having dire consequences for any scientists and peers whenever their papers are found to be bunk long after approving and shipping to market?
Like being forced to take their own drugs daily, for life?
That’s all quite interesting, Briggs, as usual. But what I really want to know is what kind of polish do you use on the Certitude? She always sparkles, dazzling, in the sun. Hypnotic. What is that? Thinking of using it on my car. Thanks.
Johnno, not bad — but how about a law that says you just take assholes out and shoot ’em in the head? That would fix a problem, right? All right, maybe that’s a bit much… how about you just shoot ’em in the dick? Or the foot? All right, all right, you can’t just shoot ’em. Whatever. I just want that polish.
Something that people rarely talk about is how peer review is opposed to an atmosphere of “publish or perish.”
If you know that the guy who submitted the paper needs to maintain a certain publication count to maintain his job, you will be less likely to reject his paper. If not out of compassion, then out of the hope that he will repay your favor in the future (say when you desperately need to improve YOUR publication count.)
Everyone involved knows that there’s now way for the average academic to reach the expected level of papers published by restricting only to meaningful results with plentiful evidence. The “honest” approach to this dilemma is to print a bunch of data manipulation that no one cares about. Statistics is great for this; repeat your old methods on a new data set, get another set of p’s (wee or not) and call it a day. But since even the authors of many of these papers do not care about the results that they have found, you will inevitably get people who find it easier to incorrectly apply methods, make up data, or just straight up lie about everything. And if your peers have the same attitude, peer review won’t help at all. The problem is made worse by the focus on a world scientific community, since there are countries that do not value honesty in the first place yet we must pretend that THEIR papers are as equally valuable as everyone elses.
Briggs,
I clicked on the links and didn’t see anything read like what’s stated above. Maybe I missed it. An outcome similar to this example of 10 new drugs is just too simplistic to warrant a paper. Furthermore, it probably won’t produce a small p-value.
And again, if you believe that all inferences based on p-values are flawed, you should not accept the inferences made in, e.g., the cited paper titled “Payments by Drug and Medical Device Manufacturers to US Peer Reviewers of Major Medical Journals” and present them as if they were true.
After skimming through one of papers cited here, IMO, it seems that approval agencies such as the EMA and FDA, should strengthen and uphold their standards on efficacy and effectiveness.
It is now the year 2024. Why are you citing papers published in 2005 and 2009? One may be able to make another Mount Everest out of existing subpar papers ever, which might contain some of yours (not mine :)). There is a wealth of outstanding research available that is driving human progress and worthy of exploration. Right?
A couple years back I was prescribed an inhaler to deal with COPD. My weight has always held steady at 150-155 pounds. Withing a month of using the inhaler my weight spiked to 165-170, and I had a pot gut that wasn’t there before. I cut back on eating lunch, and soon I was getting by on two apples and one solid meal per day, and the weight kept climbing. I googled it, of course. Does Spiriva cause weight gain? The question popped up in those frequently asked lists on all the sites that were linked by google. All the answers on all the websites, said, No. No evidence. Has not been shown to cause weight gain. On impulse I opened up the Russian search engine, Yandex, and put the question there. The second hit had a whole long list of people complaining about the exact thing I was experiencing. Both men and women were complaining that their formerly slim profiles had been ruined after using the stuff. A little more searching led to, “Well it has been known to cause weight gain, but so infrequently it wasn’t included in the side effects.” It took several months before my weight got closer to normal, but the pot gut remains.
JWM