Two (main) kinds of bad statistics: (1) where they are used to claim what is not true, and (2) where they are used to claim what is true.
The second might not seem bad, because a truth is trumpeted, and that is what one does with truths. But it is bad just the same, because the truths statistics “validates” did not need support. They were already well known to be true. Saying “Statistics has finally verified that X” where everybody already knew X has two consequences.
It starts you doubting your powers of reason, which did not need doubting. After all, they led you to X in the first place, and X is true. The second replaces the doubt with unwarranted trust in white-coated men. And that makes it much more likely that bad statistics of the first type are produced and accepted.
The reason bad statistics of the second type exist is that they must or the white-coated men will perish. As in “publish or perish.” There is a quota of papers required of all academics. These works may be good or bad, right or wrong, useful or harmful, penetrating or indifference, but regardless they must be written. Academics must write even when they don’t wish to, when they have nothing to say, when it would have been better had they remained quiet.
It is this absurd requirement which produces bad works of the second kind. (It has many other harmful effects, notably boosting the egos of the academics, but we can examine these another day.)
Example? How about “Vividness of the Future Self Predicts Delinquency”? A peer-reviewed work in Psychological Science by Jean-Louis van Gelder and others. Van Gelder opens the paper with these words:
The tendency to live in the here and now, and the failure to think through the delayed consequences of behavior, is one of the strongest individual-level correlates of delinquency.
This is our X, which everybody whose powers of reason are not stunted—as they are in, say, infants and those who have received overdoses of NPR—already knew. There was no reason in the world, save saving van Gelder’s job, that we had to be told this again, as if it were new or in doubt.
After all, it was from Aesop we learned the tale of the ant and the grasshopper, a story which was ancient before he wrote it down. Do you recall the crushing final line where Mr Ant, PhD (Assistant prof. up for tenure) lamented, “If only the grasshopper had a more vivid picture of his future self!”?
I had planned on reviewing the rest of the paper, which like all these things isn’t content with the obvious until it is lathered with dubious theory, but why bother. Did we really need to learn how a group of “young people” filled out sets of questionnaires from which wee p-values were squeezed? All to prove that those who thought about the future fared better in that unknown country than those who fiddled the day long? No, we did not.
Today the grasshopper would receive a government provided house and social security disability so the grasshopper will do OK.
Better to remain quiet when I have nothing to say? I don’t think so, Mr. Briggs.
Ray,
And the ants would be taxed to pay for it all, until they too needed government housing and SSD.
“Fiddling,” as a metaphor for laying idle, while it is sometimes not a good thing, at other times is. It is not the same as delinquency. I don’t think the conclusion is as obvious as you imply that it is.
From your dismissal, I’d more likely infer that the title of the paper was “Vividness of the Future Self Predicts Laziness.” Not the same at all.
Rob,
Whatever, dude.
I’m not saying that I’ve reviewed the study and found it to be of value. Finding the opposite wouldn’t surprise me at all. But I didn’t review the study, I read your post.
I must say I’d be interested in interacting with my future self though.
One of the strongest correlates! How strong is it?
So there is good statistics! Anyway, this reminds me of a joke my daughter sent me.
There are two kinds of people. 1) Those who can extrapolate from incomplete data. ~George Takei
JH,
Love it!
JH,
There are 10 kinds of people, those who understand binary and those who don’t.
If they don’t understand binary they are likely the 10s kind.
Pingback: Freedom and Predestination - BIG PULPIT
Interact with my future self? No thanks, not at my age. I don’t think I’d want to interact with my past self, either – the bastard would just ignore me.
Do you have any studies available utilizing alternate reality interactions?
Oh come on chaps!
This is just sociology where everything is “trivial, obvious or wrong”.
Hardly worth a sneer.
My biggest pet peeve is when absurdly complex statistics are used when unnecessary. I recently reviewed a journal article in which the author was doing a simple experiment — comparing two ways of doing something — but instead of simply presenting the results as two numbers, he had a complex structural model, intermediate variables, and some sort of SEM analysis that he’d probably been told was all the rage in his academic discipline.
I was kind in the review because I realized that if he’d done the actual experiment and presented the results as they ought to have been presented, he’d have no chance of getting it published.