Half of science may be wrong? That may be an underestimate. But at least Richard Horton, the editor in chief of The Lancet, is in the right ballpark.
Ballpark? That might be the wrong metaphor. It implies a game with rules, winners and losers. Science may have been like that, once, but it’s now, far too often, a mechanism to provide support and cover for faddish politics and speculations.
The slide into the abyss is not new. Writing in 2003 in Fads and Fallacies in the Social Sciences1 Steve Goldberg said there “was a time when you could assume that an intelligent person looking for the truth was guided by the most basic of scientific intuitions: nature will give you a lift only if you are going her way.” That time is no more.
Particularly in sociology “we find large and increasing numbers of ideologues who act as if nature is not something to be discovered no matter what she should turn out to be, but a handmaiden whose purpose is to satisfy one’s psychological and ideological needs.” (It’s worth noting that Goldberg is a self-declared liberal.)
The situation is no better in medicine. Horton:
The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue. Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness.
Horton was at a “reproducibility and reliability of biomedical research” conference at which a colleague told him a fundamental truth: “poor methods get results”.
They do. They are also lucrative and career enhancing. Need I mention that welter of putrid “studies” which “show” the horrors that await us once global warming strikes? I’ll mention them regardless. It’s not just the nauseating “World Ends: Poor, Women, and People of Color Hardest Hit” nonsense, but our friends of the forest will feel the pain, too. Any animal which is cute, photogenic, cuddly, or delicious is promised to teeter on the precipice of extinction, but those which prick, bite, poison, main and kill or are pestilential will thrive. Global warming is selective.
Horton: “The apparent endemicity of bad research behaviour is alarming. In their quest for telling a compelling story, scientists too often sculpt data to fit their preferred theory of the world.” It is for good reason that that most valuable word endemicity is not dissimilar to enema.
Horton admitted journal editors “aid and abet the worst behaviours”. Why do they do this? Because of that idiot-pleasing quantification called an “impact factor”. No, it is not a measure of physical force, which would make sense, but a ridiculous pseudo-measure of how “influential” a journal is. Influence, I need hardly add, is value-free word. A Senator threatening to subpoena her enemies for producing research “designed to confuse the public” is influential. Right, Barbie?
Our love of “significance” pollutes the literature with many a statistical fairy-tale
End hypothesis testing now. Ban it. Purge it. Eliminate it. Consign it to the ever-growing fetid pile of failed intellectual ideas, along with socialism, empiricism, equality, atheism, and car alarms.
How to fix the system is a good question. Horton has a semi-workable idea: “remove incentives altogether.” Scientists, if you don’t already know, are expected to publish, publish, publish, which allows them to seek grants, more grants, and even more grants which pays their salary and provides overhead for their deans to lavish on their fiefdoms. But unless we ban scientists from publishing anything more than, say, one book or paper a year, this fix won’t stick.
One idea that is terrible is: “emphasise collaboration, not competition.” Good grief, no. If we enforced collaboration men would flee science faster than a democrat running from a Fox News camera.
The conclusion of the symposium was that something must be done. Indeed, all seemed to agree that it was within our power to do that something. But as to precisely what to do or how to do it, there were no firm answers. Those who have the power to act seem to think somebody else should act first.
Ask me, I don’t think the system can be fixed. We have to let it burn itself out, like a tire fire.
————————————————————————–
1I’m shocked this essential book is no longer in print, except via Kindle. Finding even used copies is difficult.
Thanks to Nate West, Ken, and Dave Morris and anybody I might have forgotten who brought this article to our attention.
That’s funny…civilization is doomed because of bad statistics!
‘Half is wrong’ implies random outcomes, a flip between right and wrong. I doubt if it is that good. Bias drives it to more wrong than right.
Yawrate:
No funnier than “For want of a nail…”
“I don’t think the system can be fixed … let it burn itself out, like a tire fire. ”
I’m waiting for the rise of Bokononism and the invention Ice-9.
I was just recently reading in National Geographic that spiders will get bigger and faster due to global warming. Or at least the species they studied probably will.
“poor methods get results”. Maybe this is the flip side of observing that many discoveries occur as the result of accident.
“emphasise collaboration, not competition”. Whatever the problem the answer is more socialism? Science by committee will produce complete junk.
Possible solutions: abolish government research funding and the Ph.D. Octopus. A great deal of questionable research projects come out of the need for student theses.
Sheri:
Spiders like this one?
(from the guy who brought the backward brain bicycle)
https://www.youtube.com/watch?v=JN0VtHez9xI
John B(): Fortunately, the spiders in question were in Denmark. Who knew you could get differential math into a discussion of a spider and priapism? I’m impressed!
A very unpleasant “cure” there. Should I wonder why people do things like placing sensitive parts of their anatomy near a venomous spider? Nah, too scary to even think about. I hope alcohol was involved.
John B(): Yes, yes!
So, can we modify another well-known quotation?: “All science is wrong; some (small but undetermined bit of) science is useful.
I doubt the system will burn out unless much of civilization collapses. Like tabloid journalism it will hang around making a nuisance of itself, take up most of the space, and entertain the public. Here and there real discoveries will be made and swiftly co-opted to fit the political agenda. There’s no strong incentive for it to be otherwise.
to all: science advances in steps, two forward for every one backward. It’s like a particle with net mean average velocity forward and random steps (brownian) backward.
In my own experience, I’ve been wrong several times, and the wrongly published results have been corrected. On the other hand, my best piece of work was corrected an inadequate (not necessarily wrong) explanation of paramagnetic effects in nmr. And the inadequate theory was from some high placed guys, who congratulated me afterwards. That was …let’s see…40 years ago, and I guess times have changed. I wonder how many scientists are actually seeking after “truth” (quote marks intentional) rather than self-aggrandizement.
Kuhn’s work needs to be updated. Scientists are no longer puzzle solvers operating within a paradigm. They are paradigm-pushers operating with statistics instead of rulers and scales.
Dear Dr. Briggs:
Mr. Horton’s view is, I think, both superficial and indefensible. It is superficial in that he treats subjects like sociology as “science” when in fact that claim on the part of those involved in them is sheer professional aggrandizement . The joke that the social sciences feature universal conclusions drawn from comparisons between two halves of a sample of three is an exaggeration, but the lack of discipline, rigor, and repeatability coupled with the value accorded consensus and authority marks these subjects as Not Science.
It is indefensible in that almost everything we think we know in science is at least partially incorrect with respect to some specific set of conditions but very little in science is wrong with respect to the core conditions under which the knowledge applies.
Newton, for example, really was wrong on just about everything but also quite right in the context to which his work applies.
Thus what Horton should have said is something like “perhaps around half of all conclusions being drawn or hinted at in peer reviewed academic publications outside core science and engineering disciplines is either unsupported, or inadequately supported, in the data available to the authors.”
To a certain extent you are correct Paul but the corruption is spreading. In another of our favourite fields Christopher Monckton of Brenchley has a great post at WUWT that illustrates this.
http://wattsupwiththat.com/2015/05/27/the-royal-anti-science-society-of-edinburgh/
In the core disciplines that you mention a major problem is that much that is published is not so much wrong as trivial or so poorly done as to be of no use to anyone. A lot of grant money is wasted on this in order to pad resumes or provide graduate theses that no one outside the supervisory committee reads (maybe not even them). It is always amusing to be asked to reproduce and expand another’s groundbreaking research only to find that it was poorly done or fatally flawed. At one point in my career I did a lot of this. Like Bob I even talked on the phone to the aggrieved party, at the suggestion of my supervisor, but I think that he was less pleased.
Another interesting anecdote concerns crystal samples that I grew for my Ph.D. Thesis. When I graduated I left them behind, and my supervisor gave them to a visitor who used them to reproduce some of the measurements that I had preformed. The good part is that my results were confirmed. The bad part is that my previous work was not referenced and this visitor presented the work as original, although the source of the crystals was acknowledged. I just laughed it off.
Also, your last paragraph is just a polite way of saying wrong. I prefer the direct. It should not be the business of scientific journals to publish unsupported speculation. That’s what web comments are for. 😉
“Why Most Published Research Findings Are False”
“There is increasing concern that most current published research findings are false…. simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias.”
http://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124
The newer sciences have a long way to go to get nowhere. They re worth pursuing though, because sometimes they are productive. Just the same, sociology and psychology, at this point, can only be seen as collections of observations. If they were predictive beyond how to sell popcorn (hint: in a movie theater) we’d probably all be saying “Our Ford” today. Really, these are sciences that by their very nature, via ours, can never be all that predictive. Our incessant reflexive responses to each other and the world around us are roughly as predictive today as they were in ancient Greece.
JMJ
The core problem these days is publishing record is equated with value. If you publish 100 papers you are twice as prestigious as someone who has published 50 papers. It doesn’t matter if the guy who published 50 papers (or 10) has 3 brilliant papers and the guy who published 100 papers can’t replicate any of that body of work.
Incentives are now, unfortunately, perverse.
Pingback: Lancet Editor: ‘Half Of Science Is Wrong’ | The Global Warming Policy Forum (GWPF)
Global warming is fun because the believers are forever getting stuck on their own horns. Like when No Coal meant Go Gas–until someone discovered Gas burns into CO2 and water.
And to the point of this blog, I was horrified to hear that rising CO2 concentration and added heat would make poison ivy stronger (more poisonous?), forgetting altogether that it might also make more oats ‘n grits for us all to eat.
But the top was when I read recently that our sins would also make marijuana stronger…is that a good or a bad thing? I can just see the protesters in Berkeley squirming over this little twist of fate.
Cool. So half the solar eclipse predictions are wrong, and lots of peoppe travel to places where there’s nothing to see. Aren’t they foolish? And why is nobody complainimg about this?
Please read slowly Sander, as this may be difficult for you to understand. It doesn’t mean that 50% if Einstein’s papers were wrong, but that around 50% of newly published research is junk. This is actually very much on the low side. The paper I cited suggests around 80% of published research is junk. If you’re ever wondered why the news reports tell us that coffee is bad for you one week and then good for you the next and then bad for you the week after, then now you know why. I used to be an avid reader of science news. I gave up when I eventually worked out, although it took me 1-2 decades to reach this conclusion, that it was nearly all nonsense.
Anyone can produce a scientific publication.
http://io9.com/i-fooled-millions-into-thinking-chocolate-helps-weight-1707251800
This article ( and the Lancet piece cited) both have a lot of merit. However, in both cases they have a fundamental, major flaw: they fail to distinguish between Science and scientists.
Science is a process and is never “wrong.”
All the examples cited above are actually errors by scientists — and in every case that are situations where scientists deviated from Science.
Look at ScienceUnderAssault.info for a better understanding.
There is no thing as a bad meal. There are only bad cooks as cooking is a process and cannot be wrong.
Articles can not contain major flaws as writing is a process. Writers make mistakes when they deviate from good writing practice.
Pingback: Technology is Self-Correcting; Science, Not So Much | Yard Sale of the Mind
It’s sad, this is an old theme rehashed…and apparently nobody reading this realized that; see/recall:
“Why Most Published Research Findings Are False,” by John P. A. Ioannidis; published in 2005; http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1182327/
That’s an analysis that’s objective…avoiding the temptation that some have to contrive a reason to rant & condemn particular themes with which they disagree under a thinly veiled pretense of ‘holier than thou’ objectivity.
Pingback: The bureaucratic science-machine broke science… | Unified Physics Institute of Technology
Pingback: This Week In Doom: Bad Science Edition | William M. Briggs
Getting the private, self-perpetuating group of distinguished scholars that President Lincoln selected in 1863 to advise the government on matters related to science and technology –
The NAS (National Academy of Sciences) –
out of the business of reviewing the annual budgets of federal research agencies (NSF, DOE, NASA, EPA, etc.) worth billions of dollars in 2015 for Congress –
without outside supervision or audits to avoid obvious conflict-of-interest –
is the obvious first step.
The 2015 membership of NAS is not restricted to distinguished scholars.
Pingback: This Week in Reaction (2015/05/31) | The Reactivity Place
” Half Of Science Is Wrong.”
I thought the number was supposed to be closer to 80%.
The problem with science is twofold. One part is the gradual breakdown of Feynman’s “brutal honesty” and the need to publish null results at the rate at which they occur, which is (if we are doing things right) at least three or four to one if not ten to one. One startup company succeeds for some ten started, for some reasonable measure of success, for a reason. We should encourage the proposal and investigation of iconoclastic ideas, as long as we publish the failures with brutal honesty as Feynman suggests in his famous Cargo Cult lecture.
The second one (after the failure of simple honesty and the lack of an overall system that rewards null results as much as it rewards verified hypotheses) is the sad fact that never before have so many “scientists” been so poorly educated in the basic concepts and methods of statistics. I recently read a brave attempt to set at least part of this right titled “Statistics Done Wrong, a woefully complete guide” by Alex Reinhart. However, the text itself makes it clear that even with regard to humble things like the p-value and data dredging, fixing things is an uphill task because many of the scientists doing the work are clueless about both.
Now is not the time or place to go down the list of abuses of statistics and statistical terminology like “confidence” used in contexts where absolutely no support for a claim of confidence can be made on the basis of any accepted statistical theory in climate science, e.g. the places where they take climate models, form averages and grand averages produced by single (but not independent) individual models, take those average results and without even weighting them according to the contributing grand average data, superaveraging the grand averages into a curve that is then used as if it has some meaning derivable from statistics. Nor is it a good place to point out that a mere glance at things like the average of the Feigenbaum trees produced by chaotic systems is enough to see that the average over the tree is not an unbiased estimate of the extrapolated “mean” trajectory without the chaos, meaning that the very first step in all of this averaging is deeply suspect and probably produces results with a systematic bias, not results without a systematic bias, if one can get past the fallacy of treating the single climate trajectory of the actual chaotic climate of the Earth as having any reasonable relationship with any of the trajectories produced by a perfect chaotic model with perturbed initial conditions or parameters, let alone with some sort of average over those trajectories. Nor is it a good time to list the sins observed in medicine especially, but the other sciences as well.
It might be a good time to propose one reasonable remedy. Alter the official, stated position of the granting agencies and lean heavily on at least the Universities that receive government grant money to remove the implicit rewards associated with proposing “successful” hypotheses from the system. Deliberately retune the entire system to reward well-done science even, or perhaps even especially when it produces null results.
Disproving a hypothesis is just as valuable as proving one. More so, really — we can never accept a supported hypothesis as true beyond all doubt, but we can reject a properly failed hypothesis as being false beyond any reasonable doubt. One may never be able to prove the existence of God by any experiment, but every experiment that attempts to observe it but fails properly reduces our degree of belief in the hypothesis, eventually to absurdly low values, pending the occurrence of some positive evidence. The same is just as true of magnetic monopoles or Higgs bosons or “darkons”.
In physics as much as anywhere, of course, we reward those that find the Higgs boson with lavish rewards and Nobel Prizes, but somehow forget all of the researchers that failed to find it in experiment after experiment, even though those experiments were just as much a part of our eventual knowledge of the Higgs boson (if it is ever positively verified) as the direct observations that lead to its verification. In medicine, the rewards are far more lavish, and the punishment for null results still more dire. This alone explains much, and this part of the system could be changed, if we had the will.
rgb
Pingback: Recent Energy And Environmental News – June 1st 2015 | PA Pundits - International
Pingback: Lightning Round – 2015/06/02 | Free Northerner
Pingback: Why Global Warming Models Are Hot And Bothered | William M. Briggs
Pingback: The Trouble With Scientists | Basic Rules of Life
Pingback: Obviously this Guy’s a Science Denier!
Pingback: Left is Right - The Times of New Rome
Pingback: The Left is Right - The Times of New Rome
Pingback: Betrayers of the Truth – Watts Up With That?
Pingback: Betrayers of the Truth – Climate- Science.press
Pingback: Betrayers of the Truth – TECH LIFE
Pingback: Massive Over-Certainties In Medicine: Bad Statistics, Bad Experiments – William M. Briggs
Pingback: Massive Over-Certainties In Medicine: Bad Statistics, Bad Experiments - Free Life
Pingback: RE: Enlightenment Is Obvious - Discussion - www.dharmaoverground.org
If half of science is wrong, then half of airplanes should fall out of the sky. The base premise is wrong for both.
Les,
What a curiously wrong argument. Can you tell us how you came to it?