Was it Justice Ginsberg who popularized the fallacy that statistics could prove discrimination? Somebody check me on that. Busy day here.
The fallacy is that statistical models which have “statistically significant” findings identify causes. Which they sometimes can, in an informal way, but probably don’t. Anyway, that’s for another day. Point now is that discovering “disparities” and “gaps” and “discrimination” via statistics is silly.
Headline from the New York Post: Goldman Sach’s differentiating stats dominate sex suit.
Standard story. Couple of dissatisfied women accused Goldman Sach’s of being boy friendly. “They are suing the financial powerhouse, alleging a pattern of underpaying women and promoting men over them.” The ladies’ lawyer and Sach’s each hired their own statistician. The fallacy is already in place, ready to be called upon.
If there was real discrimination against women because they were women what should happen is that it should be proved. How? Interviews with employees, managers, ex-employees, examination of emails, memos, that sort of thing. Hard work, which, given the nature of human interactions, may ultimately be ambiguous, useless to prove anything.
Statistics certainly can’t prove discrimination, because statistics don’t identify causes. And it’s what causes the alleged discrimination that is the point in question. Since statistics can’t answer that question—which everybody should know—why would anybody ever use it?
Laziness, for one. Who wants to do all that other work? For two, it’s easy to get people to accept “discrimination” happened because math. Lawyers working on commission therefore love statistics.
The Post said, “The bank’s expert, Michael Ward of Welch Consulting, said the pay disparities between men and women are statistically insignificant and said Farber [the ladies’ expert] was overly broad in his analysis.” “Significance” and “insignificance” are model and test dependent, so it’s easy for one expert to say “insignificant” and another “significant.” The data can “prove” both conclusions.
But something else is going on here, I think. Note that according to Farber, “Female vice presidents at Goldman made an average of 24 percent less than their male counterparts”. Ah, means. An easily abused statistic.
There more. Here’s the final two paragraphs (ellipsis original). See if you can spot the probable error. Hint: the mistake, if there is one, appears to be Farber’s. Of course, there might be no error at all. We’re just guessing.
Farber looked at divisions across the bank, rather than at smaller business units, which, according to Ward, muddied the statistical data.
“Breaking it up into these little pieces means you just won’t find these pay gaps,” Farber said. “It’s always a trade-off in this kind of analysis in getting lost in the trees…or saying something about the forest.”
Get it? Take a moment and think before reading further. It’s more fun for you to figure it out than for me to tell you.
This sentence is to fill space so you don’t easily see the answer.
So is this one.
This might be a case of Simpson’s (so-called) paradox. This happens when data looked at in the aggregate, such as mean pay for men and women at the Division level, shows (say) men with higher means, but when the same data is examined at finer levels like business units, it can show women with higher or the same means as men in each unit (or a mix).
The reason this happens is that the percent of men and women isn’t be the same inside each of the finer levels, and the mean pay differ by levels (no surprise). This link shows some easy examples. It’s more common than you’d think.
Farber looked at aggregates and Ward (properly) examined smaller units, a practice which Farber calls “muddying” the data. Well, it’s a strategy. The name-calling, I mean. Judges looking for an excuse appreciate it.
Even though it’s still looking at statistics, and to be discouraged, it’s better to look at the entire pay distributions, not just means, and at even finer levels, say business units and various years of experience in the same job title. But it’s chasing fairies. It can never prove anything.
And even if the data show a difference everywhere it could be that women in each unit are paid less, but not because they are women, but because women in negotiating their “packages” might do so inefficiently compared to men. Who knows?
Statistics is no substitute for hard work.