This is from Schekman’s “How journals like Nature, Cell and Science are damaging science: The incentives offered by top journals distort science, just as big bonuses distort banking” in Monday’s Guardian.
Other nuggets:
The prevailing structures of personal reputation and career advancement mean the biggest rewards often follow the flashiest work, not the best.
Or the work that brings in that overhead, baby! Judging by money alone, the major business of colleges are two: squeezing money out of Leviathan and sports. Everything else lags.
While [luxury journals] publish many outstanding papers, they do not publish only outstanding papers.
Steer clear especially of anything hot, current, “sexy”. These papers are likely to be dreck.
The exclusive brands are then marketed with a gimmick called “impact factor” — a score for each journal, measuring the number of times its papers are cited by subsequent research…
It is common, and encouraged by many journals, for research to be judged by the impact factor of the journal that publishes it. But as a journal’s score is an average, it says little about the quality of any individual piece of research. What is more, citation is sometimes, but not always, linked to quality. A paper can become highly cited because it is good science — or because it is eye-catching, provocative or wrong. Luxury-journal editors know this, so they accept papers that will make waves because they explore sexy subjects or make challenging claims. This influences the science that scientists do. It builds bubbles in fashionable fields where researchers can make the bold claims these journals want, while discouraging other important work, such as replication studies.
Amen and amen! Much more time, especially in the vast intellectual backwaters like education and the like, in replicating—to the letter—so-called foundational studies, i.e. those works which everybody believes are true but have never been precisely checked. “Near” checking is not checking, incidentally.
And pay attention: a journal’s “impact” factor—which should measure the force with which it hits the trash can (that’s a joke)—is only a weak, tepid indicator of the quality of its papers. But it’s a pretty good take on how “hot” the journal is. What civilians don’t understand is when a journal’s “impact” factor is on the increase, more scientists start sending papers to it with the mindset, “Hey, might as well try.” It soon becomes the thing to have your work appear in this journal. “Did you hear Jones has a new paper in JASA?” That it appeared in a luxury journal is all that is remembered. What the paper was about is only secondary.
It all goes back to the money, of course. More highly cited papers, the bigger the chance of brining in the overhead. But never mind.
One group I was in scored a paper in JAMA (or was it Lancet?). The egos who run the place not only sent a letter of acceptance but also an invitation to have the front page of the new article bronzed (for a large fee) and framed “suitable for hanging.” I kid you not. Now that’s science.
Solution?
There is a better way, through the new breed of open-access journals that are free for anybody to read, and have no expensive subscriptions to promote. Born on the web, they can accept all papers that meet quality standards, with no artificial caps. Many are edited by working scientists, who can assess the worth of papers without regard for citations.
This is okay, but what Schekman forgets is that open-access journals are pay-as-you-go: scientists must pay “page charges” for their work to appear. This leaves out folks like Yours Truly who has no grants, no income, and no secretarial support.
Slightly better, and the position adopted (by now by almost 100%) by physicists and mathematicians is to deposit your “pre-print”, i.e. an unedited paper, on arxiv.org. Free to place, free to download, which guarantees a larger readership.
Or one could even launch one’s ideas on some public space, like a blog. About 70,000 people a month drop by here; I’m not even sure 7 people ever read any of the official papers I wrote. Plenty of open, hearty, peer-review here, too: more than from any journal.
The egos who run the place not only sent a letter of acceptance but also an invitation to have the front page of the new article bronzed (for a large fee) and framed “suitable for hanging.â€
Was it a first time appearance? Everybody knows the first baby is the most darling.
—
It would seem the real solution is for editors to insist on quality. Everything else would fall in line. Peer review was supposed to support this but obviously it does not. What also should be curbed are “scientific” press releases — the sole purpose is to eventually receive more money. Sadly, neither will happen soon.
Now that I think about it, it’s the press releases which likely do the most damage. Yes Flash will get someone ahead — in almost every field and not just science — but eventually the chickens will come home to roost.
“Plenty of open, hearty, peer-review here, too….”
Much more important at wmbriggs.com is “non-peer” review. If a scientist seeks to have his or her discovery understood, then the discovery should be communicated in words and means understood by the public. When they can’t or won’t obscurity attaches. Scientists have an implicit duty to communicate and educate, not impress with mathematical or language prowess.
An Engineer, I understand the sentiment but I have to disagree, especially with the statement “the discovery should be communicated in words and means understood by the public”. Who is the public and what level of understanding is one obliged to aim at? With this restriction scientific advance would be impossible. Where would this leave Isaac Newton, as one example, whose work was impenetrable to most. There is a place for the communication to fellow experts just as there is a place for the popular science article. Very few people have both talents, maybe even a null set. Call it a division of labor. This is sort of an extensive of the great cake debate. Why the urge to tell others how to do their jobs?
But yes peer review is becoming a problem and maybe it is the “publish or perish” effect that Briggs has discussed in the past. The field will find its own solution which may very well be the arxiv approach which is somewhat similar to the method used by scientific societies in the days before peer review.
All,
Just as a proof, from today’s press this hot ticket: “In two papers posted on the arXiv repository, Yoshifumi Hyakutake of Ibaraki University in Japan and his colleagues now provide, if not an actual proof, at least compelling evidence that Maldacena’s conjecture is true.”
Scotian: While I understand objections to “public” review, there is a value in having non-experts reading papers. Sometimes researchers overlook what others will pick up as obvious. I don’t think you need people “telling others how to do their jobs” but I do think that learning to answer questions at all levels makes one a better researcher. You may not be able explain the calculus to the questioner, but you should be able to explain how you got there. In the case of extremely complex ideas, this may be difficult, but I read bloggers who try and make the complex understandable (theoretical physicist). If you think of it as clarifying your ideas and research, rather than having your competence or level of understanding questioned, it can be a valuable thing.
It is easy to blame the journals, but the dreck is written by real individuals, who are government-funded if not (and usually) actual government employees. The journals are also government-funded.
Science today is a government program, like welfare or food stamps. Science is political because it depends on political support. Political “correctness” trumps scientific integrity every time.
Scientists today are welfare dependent. Indeed, for every dollar paid to a scientist, some truly needy child goes hungry.
So excuse me if I don’t care how little integrity is displayed by scientific journals. That’s not their purpose. I don’t expect it. What should be expected is harum-scarum bloviating designed to strike fear into the hearts of the unwashed masses. What better way to gin up $$$ than rank paranoia? What better way to keep the gravy flowing than to kiss the ring of the Big Benefactor?
I’m afraid that “open-access journals” are just more of the same. Science is dead. The bell tolls.
My view Sheri is that research is a wild and varied frontier, sort of like the wild west where the opinions of green horns and easterners are looked upon with suspicion. You say that you do not want to tell other people how to do their jobs but then you proceed to do exactly that. Would you dismiss the work of Paul Erdos because he could do none of the things that you consider important?
http://www.jcu.edu/math/Faculty/Shick/erdos.pdf
Scotian: No.
Back in the old days when people didn’t realize that they didn’t have the Internet, older colleagues told me to file paper technical reports while the papers were under review, just to share ideas and solutions and solicit suggestions, and if necessary, to claim to be the first one to come up with the idea and solutions.
How science and technology have taken important places in many aspects of our life!
Now people are using Arxiv.org for such purposes. It allows researchers to disseminate results quickly to interested parties, regardless of its significance and correctness. Of course, there are researchers who don’t want to share… thinking of the discovery of DNA structure.
I haven’t heard anyone citing impact factors to show their contributions in any talks, which I think is a laughable and embarrassing thing to do. Researchers can usually judge the quality of publications in their areas. A researcher would and should know the quality of his own work. Who are we trying to kid? Who are we trying to impress by citing impact factors?
JH,
You’re right. We don’t as much about “impact” factors in statistics, but in medicine they are work almost like a badge.
JH, You are forgetting about tenure, job applications (CV) and the like. Papers are rarely read under these circumstances only counted and weighted with respect to impact factors. The applicant does not have to state the impact factor as that would be gauche. It is the search or tenure committee that takes them into account.
Scotian, if someone (e.g., the Dean of a college) doesn’t want to put in extra effort or can’t judge the research, then I guess they’d have to rely on someone else or some metrics such the number of publications, impact factors, and journal prestige.
JH, as a rule the dean does not serve on the search committee which is composed by members, sometimes all, of the department. Of course recommendations have to be approved by the dean and eventually the university president, but the committee is rarely overruled. To give an idea of the scope of the problem let us say there are one hundred applicants each of which has twenty publications on average for a total of three thousand highly technical papers. These are typical numbers, by the way, drawn from personal experience. It is also likely that none of the committee members is an expert in the field that the department is hiring in. You want a variety of specialists to fill out the departmental roster, which is especially true in a small department, so you can see this is a formidable task. So we do the best that we can and it is not just a matter of counting papers or reading letters of recommendation. The real problem is judging the personality of the applicant as, after all, you might get stuck with a Paul Erdos. 🙂 But then again I slipped though, so the system is clearly not perfect. Tenure and promotion committees have a somewhat different problem since its membership is even broader, but they also have expert input from the department and often from outside the university.
I retired several years ago from a major university where during the latter part of my career I was my department’s P&T chair. It is certainly true that impact factors have become a fetish among the faculty, but deans and chairs want money and lots of it. The quota nowadays is about $300,000 per year of research monies. It is substantially higher at elite research institutes.
The pressure on untenured new hires is enormous, and their lives are filled with dread and unhappiness. I cannot understand why anyone would want such a life.
Open access journals are good but only if they have high quality peer review or at least enough expert editorial review to prevent publication of obviously flawed papers. You nightlight two nonsense studies – Nervous Fish and Existence of Fast Food joints effects the psychology of people (whether they eat in them or not).
I contacted a Omics/SciTechnol journal (online only, author pays) while doing an investigation last year and was told — after all else was said and done — that they would publish anything I sent them.
The once venerable journals, Science and Nature, have been publishing such silliness lately that for me they have eclipsed Popular Science magazine for entertainment value.
Sorry if it is the qrong thread but no one ansewered
A general question:
Could a gun shop owner be able to refuse to sell a gun to someone because he is identified to republican. He owns all gun shop in a state for the sake of this scenario
Sylvain: I will give you my answer (so you’ll stop thread jumping, hopefully) and that’s it. No discussion-I won’t be back.
Yes, a gun shop owner should be able to sell to whom he wants and not to someone he does not, for whatever reason. In your scenario, the republican can shop out of state or our of country (since if we remove the law about discrimination, it seems likely a lot of other laws will go, too) or a gun show or an individual. Now, if you want to stretch your already unbelievable example to include no one will sell the republican a gun, yes, that’s still okay. Virtually impossible, but okay. There are suitable alternatives to firearms which the republican can obtain. You can stretch the scenario till it explodes, but if one is libertarian or believes in free enterprise, the answer remains the same.
That’s it–my answer and now I’m gone.
bob sykes,
Aha! It’s good to have a figure to think about.
Sheri,
Thanks for admitting you lost the argument by posting the answer.
You had two choices, either lie to support your previous point, or tell the truth. I am not surprised that you have chosen to lie.
You describe yourself has a libertarian!!!!! I think you have no idea what a libertarian really his since tolerance is very important to libertarian. It is evident that you have very little tolerance for people that don’t agree with you.