https://twitter.com/AmirSariaslan/status/862393511622184960
Here are some interesting numbers.
Citation counts are our primary measure of a paper’s scholarly impact, and yet citation counts on average are distressingly low. By one count, 12 percent of medicine articles were never cited, nor were 27 percent of natural science papers, 32 percent in the social sciences and 82 percent in the humanities. Another study found that 59 percent of articles in the top science and social-science journals were not cited in the period from 2002 to 2006.
The author’s take on that is to move from functions of citation counts that quantify the unquantifiable to some other, as-yet-undiscovered system of quantifying the unquantifiable. (Impact, incidentally, is what happens when a paper is dropped to the ground.)
But skip that. Notice 82% “humanities” articles are never cited. This is worse than it sounds. Maybe you didn’t know it, but academics cite themselves, if nobody else, and often. Patting oneself on one’s back, and the boost this gives to one’s ego is a prime reason academics are hypersensitive to criticism. So the high number of uncited papers must mean we not only have papers nobody cares to notice, but that we also have a flood of newbie papers.
These numbers are not surprising given that, as everybody knows and everybody dislikes but that nobody will do anything about, paper quantity counts more than quality. (What counts most is dollars-with-overhead brought in. A prof who can really rake it in, with juicy 50%-plus overheads, need publish nothing, as far as individual institutions go.)
There are rare exceptions. Some work really is “novel”, the latest academic buzzword, a word seemingly de rigueur in fields like education. Which reminds: Did you read the excerpt from the tweet atop this post? The world would be better were this kind of nonsense never published. Professionalizinig methods!
Here’s another:
Why on earth did @peterboghossian and @GodDoesnt suspect that the journal NORMA has an anti-male bias? https://t.co/nuxUReUxxH pic.twitter.com/BvHQXMF3oS
— New Real Peer Review (@RealPeerReview) May 28, 2017
Anyway, an author with a paper count of one but with a genuine discovery will be as or more successful than a mediocre professor with dozens of works.
But it’s weight over wisdom most times. Even I fall into this, and have found myself listing numbers of papers as an accomplishment. Awful habit.
Solution? There is none. You might have thought, “Just don’t read the papers” would work. That’s already happening, as the lack of citations proves. Even unread, the papers are still there. They build and push inland like a tsunami. Nobody could read them.
The loneliness this induces is what causes some academics to sweeten titles and results, particularly in those fields directly touching on mankind. “Maybe the press will notice our new paper ‘Going For Walks Deadlier Than Cancer’.” Well, the press does, and then we do. Regular readers have seen dozens upon dozens upon dozens of these creations dissected, the fault always being obvious.
You might propose as a solution, “Stop requiring academics to publish.” To which I reply, you must not be an academic. This will not happen. Once the bureaucracy—administrators and the horde of deanlettes inside schools and the apparatchiks in government—get hold of a “metric”, there is no shaking them loose of it. Paper count (or functions of same) will always be important.
So, fine, there is one solution. But you won’t like it. Ready?
Tear down maybe sixty to seventy percent of colleges and universities. Stop requiring people to go to college. Stop telling folks a college education is a “must have”. Fewer colleges means fewer professors, and fewer professors means fewer papers, and fewer papers in the milieu of fewer and better colleges means a greater proportion of wheat over chaff.
That’s not going to happen, either. As we talked about, it can’t be long until government, as it has with primary then secondary, will soon (this is my bet) mandate tertiary education. You can never have enough lessons that racism is bad.
“But Briggs, isn’t tertiary education necessary because so much time was spent in the younger years imbibing official ideology that there was no time to learn to read, write, and cipher?”
True. This is, after all, why most colleges have “pre calculus”, remedial English (or whatever they call it), How To Spell Your Name, and so forth. Far too many matriculating kiddies can’t read, can’t write, can’t add two numbers without the aide of a “device.” Yet they will in four short years be awarded a “degree”.
This leads to speculation. Given the government sees its duty to grow, and will therefore (we accept arguendo) tertiary education will be mandated, it follows the Masters and PhD will also be devalued (for the obvious reasons). I ask readers: Will this lead to a “degree” greater than the PhD, one which recognizes actual merit?, or will the system collapse entirely?
Bonus! This article related. A Plan to Kill High School Transcripts… and Transform College Admissions
The level of scholarship here is typical of Dr. Briggs. For example:
“Notice 82% “humanities” articles are never cited. This is worse than it sounds. Maybe you didn’t know it, but academics cite themselves, if nobody else, and often.”
The origin of this citation statistic is this paper:
https://arxiv.org/abs/0809.5250
where citation numbers are defined to exclude self-citations.
All,
Lee’s right. I missed that. No conclusion changes.
“No conclusion changes.”
What about what quickly follows?:
“So the high number of uncited papers must mean we not only have papers nobody cares to notice, but that we also have a flood of newbie papers.”
And who know what else? I only skimmed the rest. But it’s clear that you draw conclusions based on numbers whose meaning you haven’t bothered to learn.
Regarding the killing of high school transcripts, the reason the idea of showing examples of subject mastery (ie, portfolios) arises is that judging the admissibility of students based on course transcripts depends on the rigor of courses and consistency across school districts — which everybody knows doesn’t exist. Admissions counselors already factor in the heterogeneity, however. Evaluating examples of student work consumes immense amounts of resources, even with some degree of automation, hence the transcript as a proxy. But let this experiment continue. Maybe someone will find a novel solution to admissions decisions.
The number of papers that are ever read may be lower than citations would indicate given that people often cite papers they’ve never read. Of course people also read papers they don’t cite, so it’s complicated.
There are a couple patterns in citing unread papers. One is asking a colleague for a reference and citing it without reading it. (Maybe the colleague hasn’t read it either.) Another is adding citations to flatter a boss, reviewer, or editor.
Pingback: Outliers (#54) « Amerika
Briggs —
Can those papers be copyrighted (which implies the protection of “intellectual” property)?
In the USA, any creative work you do that is original or is an original derivation of some other original work is automatically copyrighted, whether you register it or not. You can, of course, waive those and put your work directly into ‘public domain’.
Be aware that with derivations, the previous holder of copyrights is contacted for permission to use their work in order to create the derivative work.
without the cloak of the university or college the papers won’t be generated.
What I’m going to say is obvious but I’m still saying.
The solution which starts ‘tear down…’ is a good one but if it’s done fast it will be resisted. Violently, it seems. You’re nobody if you haven’t been in a violent protest which makes me a nobody.
Re-education and rehabilitation of the public is required contemporaneously to removal of funding by government in chunks by closing certain courses (anything ending in studies’) and in over all percentages of government spending. Always emphasising quality over quantity. If ‘studies’ can’t be removed without rendering the title nonsense it has to go for review.
Offer free online downloads of all course content to those who claim unfairness. They can study from home and nobody could be said to be completely excluded from acquiring information. With the internet and public libraries nobody can claim exclusion as they used to, many years ago.
In large companies when a trouble shooter or change management consultant is sent in they have all sorts of dastardly tricks. The well known one being making everybody reapply for their jobs.
This way nobody has to be ‘picked on’ and the best staff are retained.
So, all course titles must be re-evaluated for relevance given
‘…the changing fortunes of time…’ and decisions must be made by executives from the relevant field of ‘study’. Those at the coal face if one exists. NOT academics.
Practical people and people who have made a success in the real world are best equipped to say what and where money should be spent over and above the very few core subjects which get a pass:
Maths, physics, chemistry, engineering, computing, biology, modern and ancient languages, philosophy, medicine, history…theology, if they can be polite.
Humanities will have to ahem, start taking seriously what they are trying to achieve. Then justify it to a stoney faced realist without an apparent good side.
Decisions need to be made. Somebody just needs to put in place those who make them.
This means all academics which rely on public money are potentially out of a job.
(the other strategy used by change management consultants) This is a great way to concentrate the mind. Mean? Yep.
That’s just one idea. There’s more than one way to swing a cat.
Another is to let the courses slowly lose students as it slowly dawns on the public that it’s a waste of time and money and of no value to go to a.university. In fact, it’s detrimental.
I wish I were rich enough to start my own. Then it would be a franchise and could be copied.
Surely someone out there’s rich enough and imaginative enough. There’s enough clever enough.
Someone just needs to joint the dots.
In any event, I wanted to say abstruse or recondite but I couldn’t make it complicated enough to work it in.
How could I forgot Music? Do they even need a university? The arts have to go through the mill. They’re part of the problem.
“12 percent of medicine articles were never cited…”
I am surprised that citation rates are actually that high. Between the fact that 80% of results that cannot be reproduced (and likely wrong), and, the “long-tail” (the phenomenon that once a source is deemed “seminal” it will be repeatedly cited, crowding out more obscure research), would lead me to think that 90% of would be on the in the academic dustbin. Or, 90% of everything is crap.
Professionalising methods?
That’s as bad as ‘care pathways’.
I was once asked in interview,
“what do you think about evidence based practice?”
I hadn’t heard the term before! It showed, and I said,
“Isn’t that what we’ve been doing all along?”
Knowing the GP who asked me, I think that might have gone in my favour.
Somebody who coined the phrase, an academic, thought they knew better than those in practice.
cdquarles —
I was using sarcasm and implying that there is nothing intellectual about those papers, hence the quotes around intellectuals.
Dav,
“”That’s throwing out information isn’t it?
Perhaps but only the irrelevant information.”:
Most would agree that the point ‘intelligence’ definition is not clear and has it’s own inherent problem; being that the thing being defined is using the same entity to define itself. That’s a truly never ending problem.
It does, however, seem to me to be the place to start and if that’s not done satisfactorily then the basis of other conclusions will be questionable.
The AI quest has been misunderstood by spectators like myself but I was and secretly still am, under the impression that the aim is ultimately indeed to mimic a living intelligence let alone a human one. This might just be the media and or sci-Fi twits who want it to be the case in every given example of AI Rand D.
Artificial is the given. ‘Intelligence’ like you say is where the conflict lies.
A laptop is intelligent colloquially speaking but this would be a clearly dreadful example of actual intelligence. So what is the bar which makes that so unreachable? I say intellect is more than memory and processing, in and output, which is machine speak.
Piles and heaps are a distraction that only proves that the real world is not digital. It rather supports the argument against a fully artificially simulated intelligence. It is more analogy and relies on the controversial no clear definition.
That is the same problem as in the soul argument.
People know intelligence when they see it.
They might be fooled for a while but real intellect and intelligence is not something that can be fully understood in all it’s dimensions, fully orbed! One’d have to be outside of the system to have a chance.
~~~~~
“”It has also been recognized during early attempts at language processing that certain concepts have roots in shared experiences (such as having similar bodies) but that doesn’t mean an entity which doesn’t participate in this sharing can’t be intelligent — particularly since there is no firm definition of “intelligent” outside of a list of human characteristics.””
How could something which does not exist be designed by copying the thing of which there is no example?
“Several things:
Why does it need to be unpredictable? Merely because we don’t understand the workings? Why is that necessary.”” It isn’t except it’s one feature of human nature. It’s fallibility. Also, I thought the unpredictability was being used to show original thought but you covered that.
2) What is thinking and deciding and how are they not processes? How do you know that people aren’t also processing when they think and decide? What are people actually doing when they “think and decide”? How do they do it?
Nobody really knows where the knowing part resides. That is the puzzle. Processes occur in the body from a to b for consideration of the observer and even those are more complex the closer one looks. It’s superficial to consider that a process is linear when dealing with the body. It’s just a guide. A process implies one thing then the next and so on to render a result, that process being repeatable and predictable. Like a production line. A computer programme is a process, sneezing is a process but only in a superficial way.
3) What exactly does it mean “to know”? I mean beyond the “knowing” of interrelationships. Like above, what are people actually doing when they “know’ something? How does it work in people?”
THAT is an excellent question! That is what nobody has
ever been able to answer and I predict, never will. The reason being that the thing which you call ‘knowing’ is as certain to you as it is to anybody contemplating it. (feeling it, knowing is a type of feeling) and it’s what makes you you.
Everybody knows what they are talking about and yet nobody can talk about it properly.