Reader Kip Hansen asks, “Can you please run a brief explanation of what the Mexican Hat fallacy is statistically?”
I can. The Mexican Hat Fallacy, or Falacia Sombrero, is when a man moves from sunny to cloudy climes, such as when a hombre shifts from Veracruz to Seattle, and thus believes he no longer need wear a hat. This is false. A gentleman always wears a hat—not a baseball cap—not just because it regulates heat and keeps you dry, but because it completes any outfit.
Well, that’s the best joke I could come up with.
The term was coined by Herren von Storch and Zwiers in the book Statistical Analysis in Climate Research and it came about like this: Some fellows were wandering in the Arizona dessert sin sombreros and came upon the curious rock formation pictured above (image source).
One fellow said to the other, “Something caused those rocks to resemble a sombrero.” The other fellow, more sun-stroked then the first, disagreed, “No, no thing was its cause. That’s my null.” Quoting from a paper by Herr Gerd Bürger (because I had never heard of this fallacy before):
By collecting enough natural stones from the area and comparing them to the Mexican Hat [formation], one would surely find that the null hypothesis ‘the stone is natural’ is quite unlikely, and it must be rejected in favor of human influence. In view of this obvious absurdity [von Storch and Zwiers] conclude: ‘The problem with these null hypotheses is that they were derived from the same data used to conduct the test. We already know from previous exploration that the Mexican Hat is unique, and its rarity leads us to conjecture that it is unnatural.’ A statistical test of this kind ‘can not [sic] be viewed as an objective and unbiased judge of the null hypothesis.’
Which leads me to (hilarious) joke number two. There are two kinds of people: those who find null hypotheses a useful philosophical concept, and those who don’t. This description is confusing—but then so ultimately are most stories about “null” hypotheses.
If the “fallacy” merely means that the closeness of model fit to observed data is not necessarily demonstrative of model truth, then I am with them (this is why p-values stink). You can always (as in always) find a model which fits data arbitrarily well—as psychoanalytic theory does human behavior, for example—but that does not mean the model/theory is true. Good fit is a necessary but not sufficient condition for model/theory truth. A (nearly) sufficient condition is if the model/theory predicts data not yet known, or not yet used (never used in any way) to fit or construct or posit the model/theory—as psychoanalytic theory does not well predict new behavior.
The parenthetical “nearly” is there to acknowledge that, in most cases, we are never (as in never) 100% certain an empirical model/theory is true. But we can be pretty sure. Thus we do not say “It is 100% certain evolutionary theory is true,” but we can say, “It is nearly certain evolutionary theory is true.”
So much is Stats 101. Yet I’m still perplexed by Bürger-von Storch-Zwiers’s example. If we “already know from previous exploration” that the Mexican Hat formation was caused by (say) weathering, then collecting rocks from nearby isn’t useful—unless one wants to play King of the Hill. And what does “comparing” these rocks to the formation mean? Should the individual stones resemble the formation in some way for the formation to be “natural”? The rocks nearest will be made of the same material as the formation, so this is no help.
Regarding the possible causes or hypotheses of formation, they are infinite. It is we who pick which to consider. It could be, for example, that we’ll soon see a History Channel “documentary” which claims ancient Egyptians were flown to Arizona by aliens under the guidance of angels to build the Sombrero so that the Hopi could use it in a religious ceremony that was eventually secretly used by Hitler in his bid to conquer the USSR.
Let’s call this the “null” hypothesis. Why not? The “null” is ours to choose, so it might as well be something juicy. I bet if we link this around that, give the ingenuity of internet denizens, within a week we would have enough corroborative evidence for it to satisfy any NPR listener.
Speaking of hats, if you’re looking for a genuine Panama to cool your pate in the summer months, may I recommend Panama Hats Direct? I get nothing for this endorsement, except the satisfaction of helping this fine company stay in business. (If this is your first, go for the $95 sub fino. It is a fantastic deal.)