I was led by Dale Ahlquist, president of the American Chesterton Society, to an article by Chesterton on the kinds of statistics used in polls. Here is an excerpt (the second paragraph is added for readability):
It is an error to suppose that statistics are merely untrue. They are also wicked. As used to-day, they serve the purpose of making masses of men feel helpless and cowardly…
And I have another quarrel with statistics. I believe that even when they are correct they are entirely misleading. The thing they say may sometimes be positively and really true: but even then the thing they mean is false. And it must always be remembered that this meaning is not only the only thing to which we ought to pay attention, but is literally, as a rule, the only thing our mind receives. When a man says something to us in the street, we hear what he means: we do not hear what he says. When we read some sentence in a book, we read what it means: we cannot see what it says. And so when we read statistics. It is impossible for the human intellect (which is divine) to hear a fact as a fact. It always hears a fact as a truth, which is an entirely different thing. A truth is a fact with a meaning. Many facts have no meaning at all, as far as we can really discover: but the human intellect (which is divine) always adds a meaning to the fact which it hears…
If we hear nothing else at all but this, that a man in Worthing has a cat, our souls make a dark unconscious effort to find some connections between the spirit of Worthing and the love of domestic animals…So when some dull and respectable Blue Book or dictionary tells us some dull and respectable piece of statistics, as that the number of homicidal arch-deacons is twice that of homicidal deans, or that five thousand babies eat soap in Battersea and only four thousand in Chelsea, it is almost impossible to avoid making some unconscious deduction from the facts, or at least making the facts means something…It is psychologically impossible, in short, when we hear real scientific statistics, not to think that they mean something. Generally they mean nothing. Sometimes they mean something that isn’t true…
Statistics never give the truth, because they never give the reasons.
Chesterton gave an example of a poll in which it was learned a certain high percentage of folks breakfasted at some later hour, to which a reader might “react” (in his words) “Lazy Beasts!”, though each of the people polled who ate late had an admirable reason for doing so. These reasons were lost in the summary.
Now we, with a century’s more experience, are supposed to be more sophisticated about polls. We wouldn’t hear a poll that reported “62% of Catholics support the President” and put that support down to Catholicism (or its lack). And we wouldn’t read in a “scientific” study that “58.432% of men had a high hate score but only 53.918% (P < 0.001) women had a high score" and claim that maleness caused the higher percentage. Would we? And not having made those classic blunders, we surely wouldn't go further and say something asinine like "Catholics support the President" or "Men hate more than women". Right? Call this the mis-ascription or causal fallacy, the claim that the label assigned in the survey causes the answers given.
Now the ascription is not always in error. If somebody says to an exit pollster, “I voted for the Democrat candidate because I’m a Democrat” and if the pollster releases his results that say, “89% of those identifying as Democrat voted for the Democrat candidate”, then we tell the truth if we say, “At least one Democrat voted for the Democrat candidate because that person was a Democrat.” About the others in the sample, we do not know.
It is not an unreasonable assumption to say more than one voted because of his party status, though that assumption should be couched in probabilistic language, but it is clearly fallacious to say all the remaining did so. And it is fallacious even if everybody else in the sample voted on the party line because all were loyal party members. It is fallacious because the cause wasn’t measured, thus there is no warrant to claim the cause is known.
Chesterton is right. The ascription of some cause is a reaction, an irresistible temptation. Even those characteristics not part of the official data measurement are in game as “the” cause. This is why so many experts are terrific at saying why something happened, but why they are so terrible at making predictions.
GK Chesterton, The Illustrated London News, 18 Nov 1905, vol 37, no 967, p. 702.