It is a cliche, but it is true, that academics must publish or perish. Papers, and more papers, and more papers still, are what makes for a professorial life.
It’s often—it’s very often—not the quality of the papers that counts, it’s the count of the papers that qualifies one for promotion, tenure, and other glories. In many, or even most, places an informal target number exists, saying have this many or it’s out the door you go.
So it should not be surprising that people eek out every ounce of information from a study and try to write it up in as many ways as possible. If you run, say, a clinical trial, and you can only get one paper out of it, then, if you’ll pardon me saying so, there is something wrong with you.
What usually happens, in for example clinical trials, is that a paper is written describing the trial methodology, even if the study design is no different than dozens of hundreds of other studies. Another paper is written with the main results. Then as many as can are done on subsets of the data, or on the data with various “scales” that are added on to pad out a trial. A “scale” is a questionnaire about a subject like quality of life. Any good clinical trial should be able to generate a minimum of eight papers, and a dozen or more is not unheard of.
The only problem with this, is that the civilians are starting to catch on, pace this article in The Guardian.
The author, Ben Goldacre, caught a drug company publishing a trial result twice. There was one main paper, and another tying the main results to a “depression scale”. Goldacre was aghast and said that some people will look to the journals and say, about this new drug, “There are two studies showing its efficacy.”
In other words, they will be more certain than is actually warranted. Goldacre also frets about meta-analysis, and how that strange technique will be fooled, too. Meta-analysis is a tool that gathers studies together to show an effect is real even though the effect was not found in most or any of the individual studies (we’ll talk about this subject another day).
But that drug company was doing nothing unusual. The people who work for it need papers too.
Paper churning—for that is the informal name of the phenomenon—is not limited to medicine. We all have stories of the professor down the hall who has been publishing the same paper for years, here and there adding a small twist to make it seem different. There are so many journals, and new ones appearing regularly, that there is always a market for his work.
This has the result that, in my own field of statistics, there are about 100—yes, 100—monthly or quarterly journals. Each has roughly 10-30 articles. In medicine, there are about 2000-3000 regular journals. There is no way to keep up. It is impossible to read more than a tiny fraction of papers. Most, and it pains me to say this, are not worth reading anyway.
Every now and then, academics will gather and beat their breasts and say “No more! From now on we shall also value teaching and service and not just quantities of papers published!” But their resolve lasts only until the end of the meeting. The next day, they go back to tallying.
There doesn’t appear to be a solution. You can’t limit numbers of papers that people are allowed to publish. You could, I guess, insist that all publications go to open source journals, where the authors are required to pay for “page charges”, that is, pay for publication. Professors aren’t rich and don’t have unlimited research funding, so this would slow the rate of papers. But what about graduate students or independents who have no funding? Forcing them to pay is silly.
The old fashioned filter, allowing the creme to rise to the top—in the form of books, usually—is probably all that will work. Unless we somehow can return to the roots of what a university is meant to be. We’ll leave that for another time.
“Meta-analysis is a tool that gathers studies together to show an effect is real even though the effect was not found in most or any of the individual studies (we’ll talk about this subject another day).”
I eagerly await your talk.
I think a lot of good can come from LPUs as well. Forces disaggregation and gives more information. Also, for people like Steve McIntyre that are writing averse, it promotes picking some results to pulbish, that are easy, so that they can get moving…rather than being daunted by a vision of some grand opus.
The tenured professors have the system all figured out. When they get a lot of good results they put them in storage much like a squirrel stashing acorns, saving them for leaner times. As an undergraduate I worked in a physics professor’s lab and played a non-trivial role in getting some noteworthy results. The professor told me I’d be included as a co-author when the paper was published. I waited.. and waited.. and waited… Eventually 3 years after I freaking graduated I got an e-mail from him with a copy of the submitted for publication paper.
Matt,
I regret that one of my few posts to your splendiferous blog is that of being a grammar N**i, but I do believe you mean the word eke and not eek.
This is all so true. The endless supply of supposed ‘new studies’ tweaking an old idea leave me cold. It baffles me that people get excited about them. In the recent effort to push Physiotherapy, more and more people are now doing PHD’s. Masters are old hat. I could never devote years of my life to one ligament.
In pain management it is hard to be sure about anything. The field uses depression, quality of life scales and pain scores of various flavours. It is frowned upon to be sceptical of these. They do have their place but are not the answer for everyone. This is slightly off the main point but I find them better therapeutic ‘tools’ in their own right rather than purely ‘diagnostic’ ones. Some people need to see boxes ticked and ‘official’ evidence of depression, for example, to be convinced. Others see them for what they are. A vexing administrative and medico-legal paper exercise that takes valuable time.
Tesla
Don’t mention the squirrels.
Is there some sort of prohibition on mentioning squirrels?
Tesla:
Not really, but you could wake Ted, the man that’s cruel to black squirrels. There seems to be a lot of them around. JH knows what he does with them, I’m trying not to think about it.
If colleges hire and promote professors on the basis of quantity of papers published, rather than on the basis of the quality of teaching and the quality of the research, colleges are incredibly stupid. Oops, never mind.