Adventurous reader Ted Poppke discovered a peer-reviewed paper that, according to the American Enterprise Institute (AEI), proved that increasing sales tax on booze “caused a 24% decrease in gonorrhea cases reported to the U.S. National Notifiable Disease Surveillance System, but had no effect on chlamydia.”
Caused. Strong word! The strongest there is in science. When a scientist says X causes Y, he has reached the pinnacle of Y-studies, for once we have learned the cause of Y, we have learned what most of what science can tell us of Y. Discovering cause is thus a terrible burden. Unfortunately, in many fields discovering a wee p-value has taken the place of discovering true cause, the consequences of which I detail in my just-in-time-for-Labor-Day Uncertainty: The Soul of Modeling, Probability & Statistics.
The paper is “Maryland Alcohol Sales Tax and Sexually Transmitted Infections: A Natural Experiment” in American Journal of Preventive Medicine by Stephanie A.S. Staras, Melvin D. Livingston, and Alexander C. Wagenaar. From the paper’s beginning:
Alcohol tax increases may decrease sexually transmitted infection rates overall and differentially across population subgroups by decreasing alcohol consumption in general and prior to sex, thus decreasing sexual risk taking and sexually transmitted infection acquisition…
Results strengthen the evidence from prior studies of alcohol taxes influencing gonorrhea rates and extend health prevention effects from alcohol excise to sales taxes. Alcohol tax increases may be an efficient strategy for reducing sexually transmitted infections.
To say “Alcohol tax increases may decrease sexually transmitted infection rates” is to invoke causal language. Somehow increasing the amount the government collects on bottles of beer will cause people who would have otherwise contracted gonorrhea to not contract gonorrhea.
Before examining the paper, think how this assertion can be proved. At least one man (or woman) who would have got gonorrhea when the sales tax was low would not have got it when the sales tax is high. How could raising a sales tax cause the absence of gonorrhea where that same gonorrhea would have necessarily been present under the low sales tax?
Obviously, a sales tax rate has no causative powers on gonorrhea, so the contention fails immediately. But the sales tax might have caused some other thing or things to happen, like setting off a chain of dominoes, which blocked the gonorrhea from setting up. And this is what the paper implies. A man who would have drank to excess and engaged in sex (or sex-like activities) with an infected woman (or vice versa) will now be stopped from drinking that little bit extra he would have under a cheaper tax, with the consequence he’ll now retain enough judgment to realize beer googles blur. Perhaps he’ll read a book instead.
Now the only way to tell this for sure is to run experiments on actual men and women, raising the tax for some, lowering it for others. But even this is dodgy, because we’ll always be left with a counterfactual question. Would this man had the sales tax been lower contracted the gonorrhea he safely avoided tonight when the tax was high? How can we ever know this? Answer: we cannot: we can only assume it.
This bewildering point is belabored and bothered to emphasize the over-certainty which is caused when mere correlations in external datasets take the place of (even imperfect) experiments. What the authors did was to collect gonorrhea and chlamydia rates before Maryland’s tax increase, and then collect them again after. Enter the statistical model, a regression-like creation with parameters for the state’s general tax rate, the state’s alcohol tax rate, a rate of gonorrhea and chlamydia infections for times and places other than in Maryland under the new rate, and an “ARIMA noise model”, which we can ignore.
The p-value associated with the alcohol tax rate parameter (under various manipulations) was wee for gonorrhea and not wee for chlamydia. What about parameter for rates of other sexually transmitted diseases? They didn’t check; or if they did, they remained silent about them (I’m guessing they didn’t check).
From the p-value weeness, they concluded “A 2011 Maryland 50% increase in alcohol-specific sales tax decreased statewide gonorrhea rates by an estimated 24%—preventing nearly 1,600 gonorrhea cases annually.”
Mighty bold claim. All gleaned from mixing databases and calculating a parameter inside a dicey statistical model.
The kicker was noticed by the AEI:
However, Staras et al. do not establish that alcohol consumption decreased as a result of the sales tax increase. In fact, the National Institute on Alcohol Abuse and Alcoholism, in Haughwout and colleagues, estimates that annual alcohol consumption per capita in Maryland increased by 0.03%, from 2.2058 gallons of ethanol in 2010 to 2.2065 gallons of ethanol in 2012 for people aged 14 years and older (the group studied by Staras et al.).
This too is indirect evidence, because we don’t know if some who would have drank more drank less because of the tax increase, or those who would have drank even still more still drank enough. And so on.
Not for the last time I ask all to abjure all hypothesis tests.