Carla Antonaccio has written, “The term cult identifies a pattern of ritual behavior in connection with specific objects, within a framework of spatial and temporal coordinates. Rituals would include (but not necessarily be limited to) prayer, sacrifice, votive offerings, competitions, processions and construction of monuments. Some degree of recurrence in place and repetition over time of ritual action is necessary for a cult to be enacted, to be practiced”
Generally, “researchers” in the so-called soft science badly misinterpret, use, and understand parameters. The result is massive over-confidence, false beliefs, and yet more grants (see yesterday).
Most statistical models want to say something about some “y”, or outcome of interest, and how it “correlates” to various “x”s, i.e. how varying an “x” changes our uncertainty in the “y”. To work mathematically, the models must needs have unobservable parameters which are partly associated with the “x”s, but which are not of any direct interest to the main question.
The Cult of the Parameter so falls in love with the parameters that participants forget the original goal of the analysis and speak only of the parameters as if the parameters are reality. The Cult thus relies on the Deadly Sin of Reification (see earlier this week).
Here is an example. We’ve done regression too many times to count, but recall that there is some “y”, or outcome of interest, some number the uncertainty of which we want to quantify using a normal distribution.
It would be committing the a sin of reification to say that “y” is “normally distributed.” Instead, we say that our uncertainty in “y” is quantified by a normal. Normals have two parameters, a central and a spread (m and s). In regression, the central parameter is modeled as a function of a bunch of “x”s, i.e. other variables. Works like this:
m = b0 + b1x1 + … + xp
If, given each “x”, we knew m and we knew s, then we would know the uncertainty in “y”. That was our stated goal.
Suppose one of the “x”s is presence of male sex. The “b” associated with it, another parameter, in the presence of all the other “x”s, modifies the m, which is, I need remind us, the central parameter of the normal distribution of the “y”—which for a complete understanding also needs the spread parameter s.
Now if I were to announce to the world that male sex causes “y’ to change by the value of the b associated with male sex, I would be making a huge mistake.
Both Bayesians and frequentists make this mistake, though frequentists add the mistakes of p-values and misinterpreting confidence intervals, minor transgressions given the devastating over-certainty produced by the Cult.
Given the observed data, the “b” associated with each “x” will vary (usually) around some value, and be said to have some plus-or-minus. For instance, the software will say something like, “‘b’ equals 10.2 plus or minus 2.” The Cult will then say “‘X’ causes ‘y’ to take the value 10.2, plus or minus 2.”
The first error is mistaking probability for causality, and the second is to assume the “b” has anything directly to say about “y”. It does not. We wanted to know how “x” said something about “y”; we did not want to know anything about “b”.
The uncertainty (that plus-or-minus) in the unobservable not-interesting “b” is mistaken for the uncertainty is how “x” changes our uncertainty in “y”. Because of the mathematics of parameters, this true uncertainty (of “x’ informing “y”) is guaranteed to be larger than the uncertainty in each “b”.
The amount of error introduced by the Cult depends on the situation. If we quantify the error as the true plus-or-minus of the “y” as “x” varies divided by the plus-or-minus of the “b”, I have seen ratios anywhere from 2 to 20. Meaning, as promised, over-certainty, piled up paper upon peer-reviewed paper, is enormous.
At a guess, I’d say the Cult has influence over the majority of “research”, particularly in the so-called soft-sciences.
Examples? Here’s an old one. I’ll have more once my students this year present their projects. But you could just as easily generate your own.
Homework: read any paper which uses regression. Identify whether or not the Cult influenced the results.
Well, m is not a parameter, it is a linear function of the parameters bs. What do those parameters bs tell you about m? You simply cannot deny what the parameters mean in the context of linear equations, one of the first topics in algebra which is foundational in mathematics. Next, adding the uncertainty, what is the relation between the response y and the function m?
Briggs,
I think that I finally understand what you are saying. It might be your obscure way of expressing yourself. 😉
“The first error is mistaking probability for causality”
Now that’s a cult! Probably the most prevalent form of specious logic today.
JMJ
I’d like to compliment you on the images at the top of your daily articles – they’re very entertaining.
In my classes, I define a parameter as “a number that exists only in the Mind of God.” And we calculate prediction intervals, since it isn’t really wise to natter on about confidence intervals and try to second-guess the Almighty.
Briggs,
Your Classic posts are very interesting, and quite good fun to read. You describe many “truths” about applied statistical analysis. However, your antipathy to classical testing seems more religious than pragmatic.
You might consider this perspective on testing. It is the paradigm that is generally applied by people working in my field of finance.
Consider a patient who gets a blood test and asks her physician if she is in good health.
The doctor has p-value for all the blood components, maybe thirty items. Consider some cases:
1. All the scores are within 95% range of average: Is the patient necessarily healthy?
2. Some of the scores fall outside the 95% range of average: Is the patient necessarily unhealthy?
3. What other things might the physician consider before making a diagnosis?
4. What other tests might the physician undertake before giving a diagnosis?
Well, the answers to those questions might be long and complicated. But the point is this: the initial p-values contain information, but usually not enough information to definitively resolve a diagnosis.
Equally, the reporting of classical statistical tests in research papers contains diagnostic information.
Throwing away the p-values on blood tests does not help the diagnosis, even when such blood tests (for all sorts of reasons) are not especially reliable or meaningful. The blood tests contain information!
The challenge in published research is to ensure that analysts follow the strategy of our hypothetical physician. They need to ask more questions, do more tests, and consider all aspects of the problem. Then, either formally or informally, act Bayesian to formulate a conclusion.
The “bitch” of a lot of published research is not that it reports conventional statistics, but rather that the analysis stops at the t-stats.
Keep morale high.
Casey