Two (main) kinds of bad statistics: (1) where they are used to claim what is not true, and (2) where they are used to claim what is true.
The second might not seem bad, because a truth is trumpeted, and that is what one does with truths. But it is bad just the same, because the truths statistics “validates” did not need support. They were already well known to be true. Saying “Statistics has finally verified that X” where everybody already knew X has two consequences.
It starts you doubting your powers of reason, which did not need doubting. After all, they led you to X in the first place, and X is true. The second replaces the doubt with unwarranted trust in white-coated men. And that makes it much more likely that bad statistics of the first type are produced and accepted.
The reason bad statistics of the second type exist is that they must or the white-coated men will perish. As in “publish or perish.” There is a quota of papers required of all academics. These works may be good or bad, right or wrong, useful or harmful, penetrating or indifference, but regardless they must be written. Academics must write even when they don’t wish to, when they have nothing to say, when it would have been better had they remained quiet.
It is this absurd requirement which produces bad works of the second kind. (It has many other harmful effects, notably boosting the egos of the academics, but we can examine these another day.)
Example? How about “Vividness of the Future Self Predicts Delinquency”? A peer-reviewed work in Psychological Science by Jean-Louis van Gelder and others. Van Gelder opens the paper with these words:
The tendency to live in the here and now, and the failure to think through the delayed consequences of behavior, is one of the strongest individual-level correlates of delinquency.
This is our X, which everybody whose powers of reason are not stunted—as they are in, say, infants and those who have received overdoses of NPR—already knew. There was no reason in the world, save saving van Gelder’s job, that we had to be told this again, as if it were new or in doubt.
After all, it was from Aesop we learned the tale of the ant and the grasshopper, a story which was ancient before he wrote it down. Do you recall the crushing final line where Mr Ant, PhD (Assistant prof. up for tenure) lamented, “If only the grasshopper had a more vivid picture of his future self!”?
I had planned on reviewing the rest of the paper, which like all these things isn’t content with the obvious until it is lathered with dubious theory, but why bother. Did we really need to learn how a group of “young people” filled out sets of questionnaires from which wee p-values were squeezed? All to prove that those who thought about the future fared better in that unknown country than those who fiddled the day long? No, we did not.