Folks, a ripe entry for the Fourth Annual WMBriggs.com Bad Science Award (and what about the Third Annual? It’s coming, it’s coming…).
There is a good reason that most distrust statistical studies. That good reason is because many, maybe even most, statistical studies can’t be trusted.
Take the peer-reviewed paper “Difference-in-Differences Analysis of the Association Between State Same-Sex Marriage Policies and Adolescent Suicide Attempts” by Julia Raifman, Ellen Moscoe, and S. Bryn Austin, in the once-prestigious journal JAMA Pediatrics.
This widely touted work purports to have discovered, using statistical methods, “that same-sex marriage policies were associated with a 7% reduction in the proportion of all high school students reporting a suicide attempt within the past year.” The authors say there is now “empirical evidence for an association between same-sex marriage policies and mental health outcomes.”
Think about what the authors are implying. The mere presence of gmarriage—government-defined marriage, as opposed to marriage defined by reality—stops teens from reporting suicide attempts.
If what these authors are anxious to imply is true, it must have been that some teenagers before gmarriage reported trying to kill themselves because there was no such thing as gmarriage. Or it must be that some teenagers after gmarriage became the “law of the land” thought to themselves, “You know, I was going to report trying to kill myself. But now that Bert and Ernie can be gmarried, I won’t report it.” (Both could be true.)
About the number of teenagers who actually killed themselves because of the absence of gmarriage—or because of the presence of gmarriage—nobody knows. The study only relates how many kids self-reported suicide attempts. Since most of the kids giving answers were 15-16, it can’t have been because of actual of forbidden gmarriage or marriage ceremonies that caused reporting suicide attempts (of course, there could have been a handful of child brides or grooms in data).
This is among all teens, mind, and not just the minority reporting same-sex attraction or other non-biologically oriented sexual desires. The authors claim the effect was greater in the sexual minority.
Via a complicated massaging of numbers, the authors say that before gmarriage
…a weighted 8.6% of all high school students and 28.5% of 231?413 students who identified as sexual minorities reported suicide attempts before implementation of same-sex marriage policies. Same-sex marriage policies were associated with a 0.6-percentage point…reduction in suicide attempts, representing a 7% relative reduction in the proportion of high school students attempting suicide owing to same-sex marriage implementation.
A weighted 8.6% to a weighted 8%, they say. This is a 7% reduction, all right, but a minor tweak in the actual weighted number. Thee numbers are weighted averages across several states and the result of a statistical model called a linear regression. The 0.6 drop is not observed, but is the output from a model.
What’s odd is that the authors report the rate for teens reporting non-traditional sexual desires (a modeled 4% drop from 28.5%), and also for all teens (that modeled 0.6% drop), a group which includes the sexual minority. But they don’t report numbers for normal teens (did this number increase?). This omission leads one to suspect the authors are fooling themselves. This is suggested in two ways.
[To learn these two ways, you know what to do!]
I saved the best, and most devastating, criticism. Don’t you want to see it?