Public Policy Polling’s Tom Jensen said recently, “The first lesson you learn as a pollster is that people are stupid.” Jensen represents a common attitude among media denizens that ordinary citizens are mostly an ignorant lot.
And this feeling is seemingly backed up by research, such as that put forth by Cornell’s David Dunning and NYU’s Justin Kruger. They find “that incompetent people are inherently unable to judge the competence of other people, or the quality of those people’s ideas.”
To some extent this is not controversial. Most people say they are “above average” in everything from sports to stock picking. If somebody has no experience in a particular area, this person will often be unable to judge what is accepted as quality work in this area. Suppose a citizen is asked to judge the formatting of a grant submission. The further he is from university life (for example), the less he will be able to say much that is interesting and useful about the process.
Also, D & K’s statement is not universally true. I do not know how to fly a jet airplane, but I can tell you that if one crashes it is not a good thing. Of course, Dunning and Kruger actually claim something more subtle:
For example, if people lack expertise on tax reform, it is very difficult for them to identify the candidates who are actual experts. They simply lack the mental tools needed to make meaningful judgments.
Taking up this theme was actor John Cleese who in a video discussing Dunning’s work said (beginning at 3:30):
The problem with people like this is that they are so stupid is that they have no idea how stupid they are…There’s a wonderful bit of research by a guy called David Dunning at Cornell…who’s pointed out that in order to know how good you are at something requires exactly the same skills as it does to be good at that thing in the first place. Which means…that if you’re absolutely no good at something at all then you lack exactly the skills that you need to know that you’re absolutely no good at it.
And this is false. Absolutely, positively false. It is so blatantly false that even those of moderate intelligence should be able to see it.
I do not know how to run the Large Hadron Collider; that is, I know that I do not know how to run it. I am not being facetious when I say that I do not know how to prepare my taxes, which are especially complicated because of the difficulties I have in getting people to pay their bills; my ignorance of the operation of filling out these forms is what led me to employ an accountant.
These are just two of very many things which I happen to know that I am not good at. And, it should (but doesn’t) go without saying that you will know of many things at which you are not good. Perhaps this is brain surgery or rocket science or writing legislation. Whatever it is, you might not rise to the level of intelligence required to understand these things, but you will certainly have the brain power to know that you are not good at these things.
Now it is true that there are some people who believe they are intelligent enough to be good at all these things and more, but who are actually ignorant. But we mustn’t confuse barroom bluster with actual claims of ability. Actual and genuine megalomaniacs are rare.
The other end of the spectrum are those who claim not to know about certain things but who secretly or in private believe they actually do know, or hold with the idea that they might not know, but they could if they put their minds to it. Again, these people are a minority.
Orthogonal to these three groups are Cleese, Jensen, and so forth. These are people who think they know everything they need to on just about all subjects, but who make statements like Cleese did and who do not trust the average citizen to be intelligent enough to know his own interests.
Here is another easy truth: It is a different skill to be good at a thing than to be good at judging who (else) is good at it. It is not clear that being good at the thing makes one a good judge of the thing. Hence movie and drama critics are often not directors or playwrights, and vice versa. Successful CEOs of engineering firms are often not engineers, and vice versa. Et cetera. And citizens are equipped, at most times and places, well able to judge who their political representatives are.
It is this truth that has escaped Cleese, Dunning, and the many others who would usher in a Brave New World for the good of those who, the elites believe, cannot judge their own affairs. Since this easy truth has escaped them, we are right to question their intelligence. It is surely less robust than these folks claim. This being the case, perhaps it is best if somebody is sent to look after them.
The Ultimate Failing
Ignore all this and focus on an even more fundamental, outrageous fallacy that lurks in Cleese-Dunning arguments, which I’ll leave as a homework question. Start with the premise that only somebody intelligent enough to do a thing can judge that thing. See where that leads you. Report back here.
Must I be as smart as a good brewer to distinguish good beer from bad?
I agree with your overall point, but I think there’s a bit more to the story.
For example, I work in video, and I’ve never met someone who hasn’t made a video who actually understands how long and how much work it takes to make a video. It’s *conceptually* simple: get a camera, videotape someone, put it into a computer, and keep the good parts. Maybe add music. Done… Until you see it takes an hour just to set up the lights.
Same with statistics: all you have to do is gather some data, get it into a computer, run the data through some statistical program, and report the p-value. Done… Until you understand that it might take several days just to check, explore, and understand your data before you do any actual statistics.
So I can see that someone who has not mastered anything complex might well underestimate the difficulty of most things: thereby overestimating their own ability to master it. The lines would not be drawn as simply as some make it out, though. For example, a college education *should* make you well aware of your limitations and expand your thinking, but in many cases it does the opposite since the bar for “mastering” a topic in a single semester is so low and easy to achieve.
As another example, it might be easy to think you’re a great programmer because you have a B.S. in Computer Science. Until you hit the real world and realize (or not) that nothing you work on will resemble your pitifully-simple class projects, nothing you do will be done in a few weeks, and no customer will be as clear as your professor in setting your goals.
It seems to me that one of the key issues here is when well-educated folks look at the unwashed masses (who are not as well-educated) and try to dig a moat around themselves: experts on the inside, peasants on the outside. In the U.S. this often comes out as Left- or Right-Coast elites mocking the South or Midwest, and I think that kind of prejudice is what you are arguing against. And quite rightly so. I just think there’s a bit more to it than that…
(On a tangent: this gets me thinking that perhaps a part of our educational system should require the mastery of *some* subject. Any subject, as long as you work long and hard enough at it to realize how much more there are to subjects than first meets the eye. This would be good in its own right, as you go through life, but would also hopefully temper your view of how good you are at other things.)
I don’t think one needs particularly much specialized knowledge to distinguish a good golfer from a bad golfer.
During my wasted years in post-secondary education, it was never hard to distinguish a good physics professor from a bad physics professor, yet few students who observed the distinction would have been sufficiently knowledgeable to teach the classes. After all, we were the *students*, which more or less implies we didn’t possess the knowledge, irrespective of our pretensions otherwise.
The type of arrogance displayed by the Cleese-Dunning arguments seems to be a particular brand of self-deception common among the so-called “intellectual elite”. My hypothesis is that they need some way of bringing order, in their minds, to a world that they’re too frightened to try to comprehend, and they convince themselves of the equivalence of adopting an affectation of superiority to actually *being* superior. It’s a technique not uncommon, from my observations, in people who don’t appear to have quite matured sufficiently to transition from being self-conscious to being self-aware. To support their almost desperate need to envision themselves as being superior, and perhaps to try to find affirmation among like-minded people, they must consistently assert their superiority, either implicitly or explicitly, and doing so requires distinguishing themselves from the inferior, usually ad nauseam. It’s no coincidence that their views on the “inferior” have a ready market, as there is a “bandwagon” effect among people who exhibit this particular form of immaturity: they will seldom contradict the views of those among whom they seek affirmation.
I think people who are genuinely and irrefutably superior in some particular way generally come to see the value of humility, and would never stoop to trying to diminish another person, much less attempt to invent a category of supposedly inferior people. Luke Donald needn’t waste his time pointing out my poor golf game, nor would doing so add to his esteem among golfers. And Luke Donald would also (I imagine) never presume to pass judgement on Richard Feynman’s skills at teaching physics.
I am a little unsure what to make of this – I feel Dunning and Kruger are being unfairly brought into this by both Cleese and Prof Briggs.
The Dunning Kruger effect is about how a person percieves their own performance, not the competence of others.
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.64.2655&rep=rep1&type=pdf
The less competent believe they performed better than they actually did, the more competent worse.
You can fit this into a debate about elitist attitudes, but I feel it is more subtle than either Cleese or Prof Briggs are intimating.
The ordinary Joe (in actual fact looking at the graphs in the link – the slightly above average joe) is likely to judge their performance more accurately than either the maestro or the dullard, but as far as I am aware Dunning Kruger hasn’t asked how any of these percentiles judge each other.
There seem to be two dominant rhetorics at the moment – the elitist technocrat lauding over the masses and the folksy wisdom of the down-to-earth Joe.
Does the Dunning Kruger effect give any insight to the unforseen consequences of either group controlling policy?
I’m not sure, but I suspect great Presidents have known of the difficulties of office and been humble about their abilities, while Mrs Palin exuded great confidence in her fitness for the role.
I have to say I am slightly glad that it looks likely we will never know how accurately Mrs Palin judged her ability, and I am at once fascinated and repelled by the current build up to the election season with its polarization into elites and folksy wisdom.
Oh, and to add, Bertrand Russell knew of the effect in the 1930s – 65 or so years before Kruger and Dunning provided the p-values!
“In the modern world the stupid are cocksure while the intelligent are full of doubt. â€
The Triumph of Stupidity” (1933-05-10) in Mortals and Others: Bertrand Russell’s American Essays, 1931-1935
‘Experts say people aren’t expert enough’ that is undoubtedly true in many significant situations, false in others, and relative contribution unclear in others still.
This is a good example of an issue in which parsing it so finely so early in its study is actually detrimental to understanding. But fun for those who like to do such things.
Consider a subset, a special case, where people ARE expert or ought to be at least smart enough to know thier limitations (a Clint Eastwood character, by the way, inspiringly “got” this concept). Here’s some references where this has been studied in some detail–what’s truly illuminating is the identification of why smart people fail:
Harvard Business Review article (from around 2003): “Delusions of Success; In planning major initiatives, executives routinely exaggerate the benefits and discount the costs, setting themselves up for failure. Here’s how to inject more reality into forecasting.” by Dan Lovallo and Daniel Kahneman (this is NOT to be confused with the easily found on-line article by these same authors involving “optimism”…though that’s got some good insights too).
“Why Smart Executives Fail,” by S. Finkelstein: http://www.amazon.com/Why-Smart-Executives-Fail-Mistakes/dp/1591840457
“Unskilled and Unaware of It: How Difficulties in Recognizing One’s Own Incompetence Lead to Inflated Self-Assessments,” by Justin Kruger and David Dunning, Cornell Univ.; Journal of Personality and Social Psychology 1999, Vol. 77, No. 6. ] 121-1134 THIS is readily found on-line; the NIH link, http://www.ncbi.nlm.nih.gov/pubmed/10626367 , provides links to related material as well, such as:
“Unskilled and unaware–but why? A reply to Krueger and Mueller (2002). — by Kruger J, Dunning D. where they dig into a particular area in some depth with some critical analysis; available at: http://rfrost.people.si.umich.edu/courses/SI110/readings/Cyberculture/Unskilled%20and%20Clueless%20about%20It.pdf
The point being that such issues do lend themselves to qualitative analytical study & informed debate–though such must necessarily be focused appropriately.
BRIGGS — it would appear that your recent trend to blog about stupid studies & their inane conclusions is supporting material for some essays you’ve done some time ago along the lines that: a) maybe not everybody should go to college/university, and b) with so many in college the need to publish–something–is diluting the quality of published material…or perhaps just making it harder to find the truly good stuff.
?
Review of the thinking underlying the USA’s Electoral College, at its inception, leads quickly to the recognition that the same sort of thought process & conclusions regarding the electorate & their ability to appreciate what’s good for them were observed & addressed in the structural rules for elections — pointed out only to illustrate that the basic phenomena asserted has been observed for a long time, basically since people have organized into societies.
KEY PRECEPTS/FACTORS:
Some things one is ignorant about will lead one to know they’re ignorant & prompt them to seek qualified experts and to do so effectively & consistently.
Some other things one is ignorant about will lead one to fail at recognizing & selecting a qualified expert and to make such failures consistently.
It all depends on the person & situation. To find a single, or even several, examples of one or the other only proves that one or the other are situational issues. To try & argue that because, sometimes, people will, despite their ignorance, consistently identify a real expert and/or recognize & correctly assess their substantial lack of expertise, does NOT alter the other real issue that the same people in other circumstances will, consistently, fail to grasp what an expert is/who is a real expert and will consistently make bad decisions. The implicit failure embedded in the blog article/essay is that while these are not mutually exclusive they are being treated as if they were (i.e. finding an example that violates a premise under a given situation does not invalidate the premise…it only establishes some limit at which it is no longer applicable).
The idea of analysis is to dig deeper, not work backwards to broad categories & generalizations, then argue the analysis is wrong because the generalization is wrong. To find an exception to a generalization–which is being done by leaping from the author’s articles to the Cleese quote, which is a gross overgeneralization–is like rejecting an entire study because the abstract was poorly written. That is a tactic employed to avoid a subject while pretending to be addressing it.
There’s a readily observed corrallary to this easily grasped by illustrating with top performing atheletes: Many exceptional athletes are not expert in how to perform what they perform so well — and those that get into positions of coaching (and its too common where top performing players get promoted to coaching/management and have no expertise whatsoever for that) regularly fail to appreciate the specific techniques that must be performed properly to achieve top performance…some go so far as to endorse (hire, call up from the minors, etc.) players with high potential but who have peaked due to thier skill at overcoming a particular bad habit. This is a well known example of skill–apparent expertise–being inadequate for competent evaluation & assessment of the skill.
Of course, John Cleese is a comedian.
Just sayin’
There’s a big difference between distinguishing good from bad and being able to distinguish good from better from best. The former only requires a modicum of understanding, but the latter can require a great deal of expertise to discern.
For example, I play Go at a fairly low (but not beginner) skill level. I can look at a game and tell if the players are roughly at my level, or get a decent estimate of how much weaker than me they are. But once they’re more than moderately better than me, I have a hard time determining just how good they are: I wouldn’t be able to tell the difference between a pro game and a game by moderately high ranked amateurs, for example.
Surely some things will be easier to judge than others, I would imagine activities with concrete measures would be the easiest to judge from a non-expert’s perspective, i.e. a golfer’s handicap is a pretty good indicator of their skill level. But there are a lot of things that don’t have easy, concrete measures; how do you tell a good 747 pilot from a great pilot when they both land smoothly?
I have absolutely no formal knowledge of psycology, yet I can smell bullsh!t.
Only someone intelligent can decide how much some lesser person is allowed to eat/drink/sleep/smoke/exercise/use the internet/watch tv/work….
Start with the premise that only somebody intelligent enough to do a thing can judge that thing. See where that leads you.
Who decides whether someone can do a thing?
Only someone intelligent can decide how much some lesser person is allowed to eat/drink/sleep/smoke/exercise/use the internet/watch tv/work
Left out: procreate.
Suppose we assume Dunning’s arguement is true, and apply it to politics as Dunning does:
Dunning says that we cannot identify the quality of a persons expertise much beyond our own level of expertise. But, we can indentify the truly inferior.
So, we have elections. The elections will eleminate the infirior. However, there will be some cases where a superior candidate losses to a mediocre candidate. This gives us a cast of representatives who range from medocre to execelent. The representatives elect thier own leadership.
Assume the same rules apply. The leadership of the legislature should range be at least above average. The leadership appoints the rank and file to committees. If the leadership is above average, then more often than not, the reps will be correctly assigned to the committee where they have knowlede / expertise.
Even if the elecorate is as dumb as Dunning suggests they are, it should, nonetheless, lead to a better political class than any other system devised to date. And, aknowledging that political class is far less than perfect, the system requires internal checks.
@Hasdrubal brings up an important topic: GO. @Briggs: Do you by any chance play GO? (My guess would be you’re a Chess kind of guy, but there’s always hope.)
I have sat in countless meetings with university faculty and listened to just these types of conversations. “People make bad decisions so we need to manipulate their environments so they make the decisions we want them to make– because we, of course, know what’s good for them.” It’s terrifying. What’s really amazing is that none of them ever say, “Hey, maybe we should let people decide for themselves.”
“But utopians do not pay heed to human nature and the inalterable conditions of human life. Godwin thought that man might become immortal after the abolition of private property. Charles Fourier babbled about the ocean containing lemonade instead of salt water.” Mises, Human Action.
Godwin and Fourier were smart gents. What would Jensen say about such learned nonsense?
Folks like Jensen really mean that anyone who does not share their concerns is decidedly stupid. Poppycock.
“It is no crime to be ignorant of economics, which is, after all, a specialized discipline and one that most people consider to be a ‘dismal science.’ But it is totally irresponsible to have a loud and vociferous opinion on economic subjects while remaining in this state of ignorance.” Murray Rothbard.
Yet this quote applies to folks like Keynes, who no one states was stupid. His system of economics? Nonsense. But the man was equal to Jensen and his ilk.
I’m usually reluctant to mention the Dunning–Krueger effect on the grounds that it’s commonly cited in debates between two groups of arrogant fools each claiming that the other side is unskilled and unaware of it. (Usually both are right.) It might be worth mentioning in a debate between someone who is unskilled and unaware vs. someone who is unskilled and aware of it.
In other words: The more you know, the more you you realize you don’t know. Or: Answers lead to more questions.
This doesn’t need a p value to make it so, and it’s hardly a new concept. I’m willing to bet money that the idea of “the more you know, the more you know you don’t know” has been around for a few thousand years. What is new, at least to me, is using this age old chestnut of wisdom to brand the whole of humanity as stupid.
This does raise some interesting questions though. Specifically, when someone mentions tax reform or political policy in general, what is it that makes someone an expert? It’s not as of public policy is like chess, where there is a simple measure of skill. What makes Joe Sixpack more, or less, qualified than anyone else to comment on public labour policies?
If I didn’t have any sense of humour I might think I was a very good judge of humour and find John Cleese not funny.
I certainly remember is law school that those who thought an exam was easy just did not understand the questions. So maybe there is something to this.
On the other hand, I can tell that Tiger Woods is extraordinarily good at golf even though I can barely play. I also think I can distinguish when he is playing well and when he is not.
I am not an expert in psychology, but I think that this suggests that it may be difficult to form accurate generalizations.
Actually, I just watched the Cleese clip, and it’s not quite what it’s described as. He mentions that if you’re absolutely and totally no good at something, you won’t have the skills necessary to determine who is good. Which is true in some tautological way.
The problem with the argument is: what kind of skill/experience is relevant to the task at hand?
For example, I can’t play golf very well, but I have played other sports, I can juggle, etc, so I actually do have some basis for judging whether Tiger Woods is a good athlete. It would be a mistake to say that since I’ve never stepped on a golf course, I couldn’t possible know a good golfer from a bad one, or that I couldn’t possibly know that I could not compete with him.
Taking this into the political realm: I’ve never been president or congressman. So do I have zero experience at it? Technically, yes, but practically NO. I’ve been reasonably successful in my workplace, so I have some skill at negotiation, influence, crafting proposals, etc. I’ve gotten similar experience in marriage, and in serving on the Board of our Condo Association. I also have experience at telling liars from truth-tellers, and in separating deep from shallow thinkers.
So even though I have zero experience at being a congressman — and in fact probably would overrate my ability to be one — that doesn’t mean I have no relevant experience that allows me to be an informed voter.
Dunning has some valuable insights. Briggs’ examples (LHC, taxes) do not cut against Dunning’s main arguments. Dunning (if memory serves) distinguishes between things that you have absolutely no experience with and know you don’t know, and those that you have dabbled in and think you know a lot about (but really don’t).
I didn’t watch the Cleese video, so can’t comment on his use of Dunning’s work. Nor can I comment on everything Dunning has done (I know I don’t know). But some of his insights into incompetence are very useful.
“On Experts Say People Aren’t Smart Enoughâ€
Dunning, quoted in Life’s Little Mysteries (28 February 2012) says “most people don’t have the sophistication to recognize how good an idea is.â€
There are several serious problems with the statement. First, history is littered with ‘good ideas’ that later turned out to be flat out wrong (phlogiston and astrology come to mind). Second, most ‘good ideas’ need serious refinement before they become the ‘good idea’ they are claimed to be.
These two points are particularly true when it comes to ideas in the realm of politics. A ‘good idea’ such as a progressive tax scheme can have serious unintended negative consequences (e.g., a large portion of the citizenry paying little or no tax). The unintended negative consequences then raise other questions. One question is whether or not a progressive tax system is such a good idea after all.
Finally, there are large numbers of ‘good ideas’ (the environmental movement is awash in them) ardently supported by millions of people that upon careful examination turn out to have serious deficiencies.
So I would argue that Dunning doesn’t have the sophistication to recognize a ‘good idea’ because what is being suggested is that there is a ‘good idea’ in some abstract world where there are no problems of implementation and the idea is a ‘good idea’ for all time. This is patent nonsense.
The more important question is “can humans develop systems to sort out the good ideas from the bad ideas?†Fortunately the answer to that question is a firm ‘yes’. Do these systems work perfectly? Of course not.
I think that Cleese’s problem (man, did he ever turn into an old fogey!) is that he has no actual critical thinking skills. As a result, he turns to “experts” with opinions that are consistent with what he already believes. If he can find one “expert” opinion that concurs with his own, then not only will he adopt that “expert”, but will insist that everyone else must as well.
Apparently, he does believe that the general public is NOT stupid where comedy is concerned. In the video, he relies on some poll to prove that the Monty Python Dead Parrot sketch is the funniest skit of all time even though the questioner’s teacher (a credentialed expert, perhaps?) didn’t find it funny at all.
Of course in his mind, only people like Sarah Palin and Foxnews employees are really stupid.
I think that it is interesting that of all possible examples of someone who is too stupid to realize that they are stupid, Chinahand chose Mrs. Palin, a lady who had a history of success at several levels of government and, at the time she was selected as the Republican vice presidential nominee, was the most successful governor in the country, based on the percentage of constituents who were happy with her performance in office.
Anyone remember when Cleese did the “Dead Parrot Sketch” on Saturday Night Live? Nobody laughed. Wow, was it uncomfortable to watch. I remember cringing and squirming in my chair… it was that painful. Really the biggest laid-egg I have ever seen.
You have missed the point rather thoroughly. When you say, “I do not know how to run the Large Hadron Collider,” you are claiming ignorance, not incompetence.
Only when you know enough to think that you can run the Large Hadron Collider can you be considered incompetent at it.