On The Supposed Newly Discovered “Gay Genes.” Or, The Importance Of Model Skill

Great news! The press and activists are touting a study which claims to have discovered the genes that “make” somebody gay. So get your enwombed baby’s DNA scanned and make that appointment with Planned Parenthood early. There’s sure to be a line out the door as parents rush to eliminate these fabulous “clumps of cells.”

Hey. Why not? Abortion is the law of the land, right? And the law of the land decides right and wrong, yes? And abortion, we are told by heretics, isn’t killing, right? (Right, Sylvain? JH? Hello?) So why not save your potential child a lot of trouble and put it out of its misery as early as possible. Right?

Never mind. I was only kidding. You’re saved from confronting this gut-wrenching progressive dilemma—sexual libertinism vs. unchecked bloodlust—because the study is almost certainly nonsense. Why? Wee p-values were the evidence for the claim.

Wee p-values are the smallest sins here. Worst is the claimed accuracy of a model and the absence of skill. What’s skill? Read on.

Here was one of the initial headlines: “The DNA test ‘that reveals if you’re gay’: Genetic code clue is 70% accurate, claim scientists.” This is silly on its face, because if a man is same-sex attracted he doesn’t stand in need of a chemical test to tell him so. But let it pass, because the idea was to discover biological drivers, which is to say causes, of homosexual desire (and not homosexuality: there is no such thing).

The headline was based on an abstract—a mini-paper of about one page, common in medicine—and a press release, unfortunately common in science. Anyway, some fellow named Tuck Ngun at UCLA did the study. According to the paper:

The study involved 37 pairs of twins in which one brother was homosexual and the other heterosexual, and 10 pairs in which both were homosexual.

Using a computer program called Fuzzy Forest they found that nine small regions of the genetic code played the key role on deciding whether someone is heterosexual or homosexual.

The research looked at a process called ‘methylation’ of the DNA — which has been compared to a switch on the DNA — making it have a stronger or weaker effect.

This process can be triggered by hormonal effects on the growing foetus in the womb.

Fuzzy forest? An algorithm to do classification of some thing, here same-sex attraction or not, based on input variables, here the genetic markers. These markers weren’t DNA per se, but epigenetical markers, these methylation sites. Wikipedia, for once, does not let us down: it’s simpler to read their article than have me explain it. It’s not necessary to understand epigenetics to follow the statistics.

And did you notice? Thirty-seven pairs of identical twin brothers, in which one brother did not suffer same-sex attraction and one did. Know what that proves? It proves same-sex attraction can’t be entirely genetically caused. If it were, then you’d find all pairs of identical twins with the same attractions, given we accept everybody tells researchers the truth. This study thus proves that attraction is at least partly environmentally caused (this includes epigenetic changes, which we’d expect should be mostly the same for both twins in the womb). Incidentally, one environmental cause is choice. Skip it.

Then came the day Ngun gave his paper. His audience, we are told by The Atlantic, was not satisfied.

[Ngun] analysed 140,000 regions in the genomes of the twins and looked for methylation marks…He whittled these down to around 6,000 regions of interest, and then built a computer model…

The best model used just five of the methylation marks, and correctly classified the twins 67 percent of the time. “To our knowledge, this is the first example of a biomarker-based predictive model for sexual orientation,” Ngun wrote in his abstract.

Ngun separated the data in a training and validation set, so the model was built on something like 20 or pairs, and tested on the other 20 or so. He built “several models” and selected the one which gave the best classification accuracy, the one with just five methylation marks. Did Ngun correct for multiple testing? No, sir, he did not.

That means Ngun is claiming sexual desire is controlled largely, but not entirely, by these five methylation marks. Sound plausible to you? No. It didn’t to other researchers The Atlantic contacted either.

Guy named John Greally from Albert Einstein gets it. He said Ngun “could not resist trying to interpret [his] findings mechanistically”. Of course, nearly everybody who discovers a wee p-value interprets their findings mechanistically, which is to say, they believe statistical models have discovered causes.

Greally wrote (and The Atlantic also quoted): “It’s not personal about [Ngun] or his colleagues, but we can no longer allow poor epigenetics studies to be given credibility if this field is to survive. By ‘poor,’ I mean uninterpretable.”

To which we say Amen.

Skill

Here’s the real criticism. 37 of 47 pairs were SSA. Suppose you invent the naive model “Say everybody is SSA”. What is that model’s accuracy here? Well, you’d be right 37 * 2 = 74 times and wrong 10 * 2 = 20 times, for 74/94 = 0.79, or 79% accurate. Damn good! And it beats Ngun’s fancy schmancy Fuzzy Forests by a long shot. That sophisticated model only got 67% accuracy. This assumes his training-validation split was equal across the 37 and 10 pairs, naturally, but you get the idea.

If accuracy is your goal, there is no reason in the world to use Ngun’s model and every reason to use the naive model. The naive model does a much better job. It lacks the just-so story of methylations causing SSA, but what of it?

This lack of skill is what I’m always going on about in climate models. Persistence beats the sophisticated models, so why not choose persistence? Models without skill compared to a natural reference model should not be used.

Update Greally discovered Ngun’s answers to some of his critics. From that, we can see wee p-values were the source of decision. But the most interesting attempt at rebuttal was to Greally’s criticism about interpreting results mechanistically. Here’s what Ngun said:

Let’s be real here: no one is going to pay attention unless you talk about implicated genes. It’s all about interpretability. We ultimately want to understand what’s going on in terms of the biology so of course we’re going to talk about any genes that seem related and are interesting.

“Being real” means, to him, juicing the press release with potentially misleading information, because, he implies, who’d be interested in the truth?

Ngun later goes on to answer The Atlantic (Greally didn’t link to this). Ngun said, “[My] approach is used widely in statistical/predictive modeling field. It is not an insidious issue or data manipulation…” This is true. And that’s the problem. The method, as regular readers know, stinks and is guaranteed to produce large quantities of false positives.

More confirmation of wee p-values: “The single test we did was to ask whether the final model we had built was performing better than random guessing. It seemed to be because its p-value was below the nearly universal statistical threshold of 0.05.”