Scientific Peer Review Update

Scientific Peer Review Update

Two stories today, both the same but on opposite ends.

Story One: Accurate Paper Rescinded

We recently highlighted a paper by Bob Uttl and others in “The World’s Easiest Sociological Prediction Has Been Verified” (blog, Substack). This was the one where they showed, as was easy to guess, that the average intelligence of college students was decreasing. This had to be so: given the greater the proportion of kids going to college, the more the average intelligence of students must decrease and approach the population mean.

Or maybe even dip below it, since kids who are well above average intelligence may begin eschewing college altogether.

The journal yanked the paper! (The paper now has a new home; I also updated the original story.)

The journal was Frontiers in Psychology. As Uttl says on his site (he also emailed me), the “peer-reviews reviews were finalized…and…the article was accepted for publication by the editor.” He has a copy of the acceptance email.

Uttl’s work obviously contradicts a key cultural belief, which is Equality. So the journal pulled the paper and emailed Uttl saying “a number of overstated claims were brought to the attention of our Research Integrity team”.

But—and you can see this coming—“the email did not disclose what the allegations were, did not disclose who made them, and Frontiers in Psychology never bothered to contact any of the authors regarding the allegations.”

The journal became YouTube. You’re canceled, only told that you sinned, but not how.

After Uttl battled back, he received another email that said in part “the [published] abstract was flagged to our attention due to several posts being made online on the social media platform X.”

One of those was me, I’m proud to say.

Uttl also had a difficult time getting his money back, too. See, Frontiers is one of those journals that charges you for the privilege of them publishing the work. This amounted to about $4,000. Yes, you read that right.

Science publishing is a brilliant economic, uh, system. The journals charge you to be published. Then they charge libraries to carry copies. And they keep the copyright.

What do you get? You get to say “I have a paper.”

Now I read Uttl’s paper, as did many of you, and you can see there was no problem with it, and that it was anyway obviously correct, as it had to be. But it was not politically correct.

His isn’t the first paper that ran afoul of The Message. Remember that mask paper that came out during the covid panic, the one that showed the harms masks cause? (Blog, Substack). It, too, was fine until it was picked up on Twitter. Then it was pulled.

Many such cases.

Science is having a rough time. They keep canceling good work, mainly through the policing of peer review, which ensures contrary views on sensitive topics stays hidden. But then they also let in tons of sludge, as we see next.

Incidentally, Uttl sent me another paper of his that surely didn’t go over well with nervous Experts: “Student Evaluation of Teaching (SET): Why the Emperor Has No Clothes and What We Should Do About It“.

Story Two

You’ve likely seen headlines like this: “More than 10,000 research papers were retracted in 2023 — a new record” (Nature), and “‘The situation has become appalling’: fake scientific papers push research credibility to crisis point” (Guardian).

The Nature article has a graph showing retractions of a bit over 1,000 in 2013 until last year’s 10,000+.

The bulk of 2023’s retractions were from journals owned by Hindawi, a London-based subsidiary of the publisher Wiley (see ‘A bumper year for retractions’). So far this year, Hindawi journals have pulled more than 8,000 articles, citing factors such as “concerns that the peer review process has been compromised” and “systematic manipulation of the publication and peer-review process”, after investigations prompted by internal editors and by research-integrity sleuths who raised questions about incoherent text and irrelevant references in thousands of papers.

Also, “A Wiley spokesperson said that the publisher anticipated further retractions — they did not say how many — but that the company takes the view that ‘special issues continue to play a valuable role in serving the research community’.”

What is interesting, to me anyway, is what proportion of, let us call them, goofy papers exist. These aren’t papers like Uttl’s, which are right but in the wrong direction. Goofy papers are wrong in the right direction—the direction preferred by Experts and the Regime.

In 2023, it’s at least 10,000 divided by the number of papers published. But it’s surely larger than that, because the 8,000+ from Wiley’s subsidiary are only there because they were caught. How many goofy papers slipped through the peer-review net?

Nature has this graph:

If 0.23% of papers are goofy, and there are 10,000 known goofies, then that’s 4,347,826 total papers.

That’s on the low side of estimates I’ve seen. Another estimate is 5.14 million a year, but I’ve seen claims of 8 million and higher. Depends on how you count, and what counts.

Of cheating, they say: “The number of articles produced by ‘paper mills’ — businesses that sell bogus work and authorships to scientists — is estimated to be in the hundreds of thousands alone [summed over years], quite apart from genuine papers that might be scientifically flawed.”

In any case, less than 1% sure seems low to me. Especially since you and I, dear reader, have investigated hundreds of what are touted as the best papers, and we have found them wanting. Few were outright frauds, and few contained outright

Of course, many papers are not goofy, but are useless or boring or unnecessary. These comprise the bulk of publications.

Bonus: PEER REVIEW history article.

Subscribe or donate to support this site and its wholly independent host using credit card click here. Or use the paid subscription at Substack. Cash App: $WilliamMBriggs. For Zelle, use my email: matt@wmbriggs.com, and please include yours so I know who to thank.

10 Comments

  1. Stan Young

    A large computer search was done several years ago looking for evidence of p-hacking, Head, 2016, Plos. They say, “Here, we use text-mining to demonstrate that p-hacking is widespread throughout science.” I have counted out many papers by hand. The number of questions at issue in a paper can be hundreds to millions. It is quite believable that over half of the published papers are p-hacks.

    https://doi.org/10.1371/journal.pbio.1002106.

  2. Robin

    Pay $4000 to publish in a journal with readership in the single digits? Booze and hookers would be a better investment.

  3. Mario

    That an already admitted and reviewed paper gets retracted because of things posted on X is crazy all by itself. It reveals a complete lack of trust in and respect for the process.

  4. Cary D Cotterman

    “Or maybe even dip below it [the intelligence mean], since kids who are well above average intelligence may begin eschewing college altogether.

    No doubt. Some of the brightest people I’ve encountered have been mechanics and electricians. Some of the dimmest have had PhDs.

  5. Incitadus

    We’re in the Land of Oz: “If I say it three times it’s true.” What scares me is the work of John Ioannidis out of Stanford where he demonstrated most published medical research cannot be replicated and is based on
    primarily on wee p-values. The storm this initially set off by the usual suspects Science, Nature, Scientific American, The Atlantic, Marvel Comics etc. is revealing though as ‘the replicability crisis’ became glaringly
    obvious they now have changed their tune (somewhat). I think they really liked things the way they were
    as they are still the bullhorns for a lot of politicized crap science.

    Why Most Published Research Findings Are False
    https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124

  6. Tom Welsh

    “The journal became YouTube. You’re canceled, only told that you sinned, but not how”.

    You can almost hear the contemptuous tone: “You know what you did”.

  7. Tom Welsh

    ‘What do you get? You get to say “I have a paper.”’

    It really is an aesthetically beautiful system – especially if, like so many of our modern elites, you take a perverse joy in asserting the absurd and getting away with it. “Logic, facts, honesty, and common sense lose: I win”.

    Why do scientists need to say, “I have a paper”? Because their lazy, stupid donkeys of bosses measure and reward them according to how many papers they have published. “Never mind the quality, feel the width”. I learned from a good friend who knew quite a bit about science that the greatest scientists make one, two, or maybe three important discoveries in a lifetime. (Einstein, he pointed out, made four in one year – but he was Einstein). Publish or starve.

    So what does a scientist need to do to get published? Two things: pay the Man, and toe the line. Otherwise a paper can be trivial, wrong, or even complete gibberish and still get published.

    It’a fine system for enriching the rich, suppressing honest thought, and eventually destroying a nation or a culture.The West is suffering from terminal cancer – of the intellect.

  8. Tom Welsh

    Even Dirac, when he met Feynman, did not open the conversation with “I have a paper”. Instead, he said “I have an equation”. A universe of difference!

  9. DAA

    In the end, although unconfessed, it is all about the job, the position, the survival doing that which we like to do. However, if you decide to evaluate someone, how do you do it? The method here determines the outcome. If you evaluate for for amount of work, you use publications. If you evaluate for impact, you use citations. But how did one evaluate impact before computers and h-indices? You didn’t. The impact was connected to the content and the solution of a real or supposed problem, perhaps some suggestion of a path to a solution. Or just showing that something was there to be seen and nobody had yet looked at it. Whaterver it was, it as the impact in the peers, the other researchers, not so much if you were cited or not. It was a very human enterprise, albeit not excessively humane. Furthermore, what does it mean having a lot of papers? The writing and reviewing is time consuming. Writing more papers means less good papers. Simonton has some research saying that the average does not change much with number of papers. If you assume a normal distribution, it all amounts to having more papers so that you have a larger right tail. That is all. But as I asked the other day, how many papers by Euler (not yet the epoch of papers per se) or Lord Kelvin, or what not, does one remember? University is a guild. Do you evaluate in a guild by invinting from elsewhere? Do you evaluate from a paper or CV? Some thoughts for your leisure.

  10. Garbled memo

    DAA – you hit the nail on the head.

    You get what you pay for. In this case, if you judge professors by output regardless of quality, you get output and not quality.

Leave a Reply

Your email address will not be published. Required fields are marked *