William M. Briggs

Statistician to the Stars!

Idols With Wee P Values. Statistics As Ritual

Early scientists discover the god has a wee p-value---and a ritual is born.

Early scientists discover the god has a wee p-value—and a ritual is born.

Or, rather, wee p-values are idolized. And it isn’t just me saying so. Reader Dan Hughes points us to Gerd Gigerenzer and Julian Marewski’s peer-reviewed paper “Surrogate Science: The Idol of a Universal Method for Scientific Inference” (pdf) in the Journal of Management.

The paper can be read by anybody (well, you get the idea), but here are the juicy quotes and my comments. It’s long, but boy oh boy is it fun!

Determining significance has become a surrogate for good research.

Amen! Preach it, brother. Sing it loud. Hallelujah.

One of us reviewed an article in which the number of subjects was reported as 57. The authors calculated that the 95% confidence interval was between 47.3 and 66.7 subjects. Every figure was scrutinized in the same way, resulting in three dozen statistical tests. The only numbers with no confidence intervals or p values attached were the page numbers.

That author was nuts and forgot statistics are never needed to tell us what happened. Even though, yes, this unnecessary duplication and absurd quantification are the lifeblood of frequentism.

…in physics, Newton’s theory of simple cause and effect was replaced by the probabilistic causes in statistical mechanics and, eventually, by quantum theory.

The consequence is cause has long been forgotten. Or, rather, cause is whatever the research says it is. Terrible harm has been done because of this.

To understand how deeply the inference revolution changed the social sciences, it is helpful to realize that routine statistical tests, such as calculations of p values or other inferential statistics, are not common in the natural sciences. Moreover, they have played no role in any major discoveries in the social sciences. [emphasis mine]

Nor in any other science. P-values only prove—or “prove”—(a) what is already known (proof), or (b) what is probably false (“proof”).

The Null Ritual

The null ritual is an invention of statistical textbook writers in the social sciences.

…spearheaded by humble nonstatisticians who composed statistical textbooks for education, psychology, and other fields and by the editors of journals who found in “significance” a simple, “objective” criterion for deciding whether or not to accept a manuscript.

Thus has laziness triumphed and become ingrained in science. Gigerenzer is right: statistics is pagan ritual. And now just as effective as offering sacrifices to a volcano.

Some of the most prominent psychologists of their time vehemently objected…the founder of modern psychophysics, complained about a “meaningless ordeal of pedantic computations.” …one of the architects of mathematical psychology, spoke of a “wrongheaded view about what constituted scientific progress,”…

Not that it mattered. The Wee P-value is triumphant.

Unlike many of his followers, Savage carefully limited Bayesian decision theory to “small worlds” in which all alternatives, consequences, and probabilities are known. And he warned that it would be “utterly ridiculous” to apply Bayesian theory outside a well-defined world—for him, “to plan a picnic” was already outside because the planners cannot know all consequences in advance (Savage, 1954/1972: 16)

Amen again! Decision analysis was pushed far, far past the breaking point years ago. The EPA, and pretty much every other agency that wants to “prove” pre-decided conclusions, never remember that (unknown probability) x (unknown costs) = (who the hell knows what’s best). Instead, scientism and false quantification run amok.

A second version of Automatic Bayes can be found in the heuristics-and-biases research program—a program that is widely taught in business education courses. One of its conclusions is that the mind “is not Bayesian at all” (Kahneman & Tversky, 1972: 450). Instead, people are said to ignore base rates, which is called the base rate fallacy and attributed to cognitive limitations. According to these authors, all one has to do to find the correct answer to a textbook problem is to insert the numbers in the problem into Bayes’ rule—the content of the problem and content-related assumptions are immaterial. The consequence is a “schizophrenic” split between two standards of rationality: If experimental participants failed to use Bayes’ rule to make an inference from a sample, this was considered irrational. But when the researchers themselves made an inference about whether their participants were Bayesians, they did not use Bayes’ rule either. Instead, they went through the null ritual, relying only on the p value. In doing so, they themselves committed the base rate fallacy.

Hilarious.

…an automatic use of Bayes’ rule is a dangerously beautiful idol. But even for a devoted Bayesian, it is not a reality: Like frequentism, Bayesianism does not exist in the singular.

This isn’t so. But Gig and pal think, what is natural, that Bayes means subjective probability. Logical probability does not suffer from singularity. And any statistical method which is part of the Cult of the Parameter must eventually fall to ritual.

We use the term surrogate science in a more general sense, indicating the attempt to infer the quality of research using a single number or benchmark. The introduction of surrogates shifts researchers’ goal away from doing innovative science and redirects their efforts toward meeting the surrogate goal.

Laziness again. It’s everywhere—and government sponsored.

SPSS and other user-friendly software packages that automatically run tests facilitate this form of scientific misconduct: A hypothesis should not be tested with the same data from which it was derived…
A similarly bad practice, common in management, education, and sociology, is to routinely fit regressions and other statistical models to data, report R2 and significance, and stop there

The first point should be shouted at every PhD defense. It is the key—really the only—difference between good and bad science. It is a point so important that you should read it twice. Don’t forgot to visit the Classic Posts page to see the common abuses about regression.

Surrogate science does not end with statistical tests. Research assessment exercises tend to create surrogates as well. Citation counts, impact factors, and h-indices are also “inferential statistics” that administrators and search committees may (ab)use to infer the quality of research. …hiring committees and advisory boards study these surrogate numbers rather than the papers written by job candidates and faculty members.

Did somebody say laziness and pseudo-quantification again? Yes: somebody did.

An even greater danger is that surrogates transform science by warping researchers’ goals. If a university demands publication of X journal articles for promotion, this number provides an incentive for researchers to dissect a coherent paper into small pieces for several journals. These pieces are aptly called just publishable units. Peter Higgs, the 2013 Nobel Prize winner in physics, once said in an interview, “Today I wouldn’t get an academic job. It’s as simple as that. I don’t think I would be regarded as productive enough” (Aitkenhead, 2013). He added that because he was not churning out papers as expected at Edinburgh University, he had become “an embarrassment to the department when they did research assessment exercises” (Aitkenhead, 2013).

Did somebody say laziness and pseudo-quantification again, even though he just said it? Yes.

Update I forgot to include the popular press article which highlighted the paper: Science is heroic, with a tragic (statistical) flaw: Mindless use of statistical testing erodes confidence in research.

Update I also forgot to give you the current status on my book, which talks about all these kinds of things and gives a solution. It’s thisclose to being done.


19 Comments

Stream: Pope Francis’s Dr. Strangelove Craves One World Government

ransom66

Today’s post is at The Stream: Pope Francis’ Dr. Strangelove Craves One World Government Run by “Experts”: Hans Joachim Schellnhuber, the scientist behind Laudato Si, is an activist for a global state run by “enlightened” elites.

Who’s up for a one-world government run by radical ecologists? Herr Professor Doktor Hans Joachim, a.k.a. John, Schellnhuber, Director of the Potsdam Institute for Climate Impact Research, Commander of the British Empire, prophet of doom, avowed atheist, and the most prominent advisor to Pope Francis and the Vatican on global warming, is hot for it. And he has a “master plan” to bring it about.

No, no. It’s not what you think. He really means it. Like all good progressives, Schellnhuber does not like freedom for the lower castes (like us). And he’s doing his best to remove it. It’s for our, and for the planet’s, own good.

Go there, if you can stomach it, to read the rest.


18 Comments

Summary Against Modern Thought: God Is Pure Truth

This may be proved in three ways. The first...

This may be proved in three ways. The first…

See the first post in this series for an explanation and guide of our tour of Summa Contra Gentiles. All posts are under the category SAMT.

Previous post.

Last time we met, one month ago, we learned God is Truth. Go back and review. This week we’ll ease back into things with a short chapter, but one which contains some important points vis-à-vis realist vs. idealist or skeptical philosophy.

Chapter 61 That God Is The Most Pure Truth (alternate translation)

[1] THE foregoing being established it is evident that in God there is pure truth, in which there can be no alloy of falsehood or deception. For falsehood is incompatible with truth, even as black with white. Now God is not merely true, but is truth itself.[1] Therefore there can be no falsehood in Him.

[2] Moreover. The intellect is not deceived in knowing what a thing is, as neither is the sense about its proper sensible.[2] Now all knowledge of the divine intellect is as the knowledge of one who knows what a thing is, as was proved above.[3] Therefore it is impossible that there be error, deception or falsehood in the divine knowledge.

[3] Further. The intellect does not err about first principles, whereas it does sometimes about conclusions, to which it proceeds by arguing from first principles. Now the divine intellect is not argumentative or discursive, as we proved above.[4] Therefore there can be no falsehood or deception therein…

Notes Your senses do not err, you see what you see, or rather, your optical bits present, in whatever form they are currently capable, of information to your intellect, which sorts them out. Now you might hallucinate and, say, think that man ahead is Bob, and your intellect knows Bob. But that is not because your senses falsely reported Bob, but because you reasoned improperly from the sense impressions you had and possibly because of other false premises or thoughts you entertained (“Bob is supposed to meet me” but it turns out he was delayed).

Also, we would not know there were such things as hallucinations if we also didn’t know there were real perceptions. And while all of are are supplied, via induction, fundamental truths, i.e. first principles, which we know indubitably, each of us occasionally reasons incorrectly. Example: God is love, therefore love is God, therefore love is all you need.

Lastly, don’t forget that God does not need to reason, to move from premises to conclusion, because God knows all at once.

[5] Moreover. An intellectual virtue is a perfection of the intellect in knowing things. Now the intellect cannot, according to an intellectual virtue, speak false, but always speaks true: because to speak true is the good act of the intellect, and it belongs to virtue to perform a good act.[5] Now the divine intellect is more perfect by its nature than the human intellect is by a habit of virtue, for it is in the summit of perfection.[6] It remains, therefore, that falsehood cannot be in the divine intellect.

Notes The intellect cannot speak falsely, but our will can make us lie. We know we lie, and we know because of our intellects. A lie is an imperfection. Now lying is a huge subject, one which St Thomas considered fully. See this.

[6] Further. The knowledge of the human intellect is somewhat caused by things; the result being that man’s knowledge is measured by its objects: since the judgment of the intellect is true through being in accordance with things, and not vice versa. Now the divine intellect is the cause of things by its knowledge.[7] Wherefore His knowledge must needs be the measure of things: even as art is the measure of the products of art, each of which is so far perfect as it accords with art. Hence the divine intellect is compared to things as things to the human intellect.

Now falsehood resulting from inequality between man’s mind and things is not in things but in the mind. Wherefore if there were not perfect equality between the divine mind and things, falsehood would be in things but not in the divine mind. And yet there is no falsehood in things, because as much as a thing has of being, so much has it of truth. Therefore there is no inequality between the divine intellect and things: nor is any falsehood possible in the divine mind.

[7] Again. As the true is the good of the intellect, so is falsehood its evil:[8] for we naturally desire to know the true and shun to be deceived by the false. Now evil cannot be in God, as was proved above.[9] Therefore falsehood cannot be in Him.

Notes Notice that all knowledge of the human intellect is caused by things, only that some is. Some knowledge, that which we receive by induction or revelation, is caused divinely. But what is true is still that which accords with what is. And, as our good saint says, a falsehood is not in things but in us, in our intellects.

Plus, I’m sure the indirect point that falsity is evil did not escape your notice.

——————————

[1] Ch. lx.
[2] Cf. ch. lix.
[3] Ch. lviii.
[4] Ch. lvii.
[5] 2 Ethic. vi. 2.
[6] Ch. xxviii.
[7] Ch. 1.: In evidence…p. 109; Sum. Th. P. I., Q. xiii., A. 8.
[8] 6 Ethic. ii. 3.
[9] Ch. xxxix.


4 Comments

If The Constitution Is A Living Document, It Can Die

4727558424_83e4969293_b

On this glorious Fourth of July in these once United States, it’s well to remember that every living thing dies. And the Constitution, progressives say, is a “living document.” Since it is treated as such, it will die—and our nation with it.

Any “truth” which is subject to whim, to provision, to shifting opinion, to “evolving” history, is not a truth, or is so only by accident. Any government not based on Truth cannot last. A “living” Constitution, however, is a necessity if one wants to escape Truth, to get what one wants, and be on “the right side of history“.

If the meaning of the words of the Constitution—or any other foundational document or book, come to that—is allowed vary by whim, then because whim eventually leads to madness, and madness is deadly, again the Constitution will die.

Democracies suffer from the delusion that truth and falsity, that right and wrong, that justice and injustice, can be decided by vote. A check on this invariably fatal practice in these once United States was a document which limited powers of the people, and therefore of Government, from wandering over the abyss.

But that check is removed when the Constitution is seen as providing justification for desired or fashionable opinions. Our current operation is backward. An opinion is decided first and words in the Constitution are found which can be bent around this opinion and “justify” it. That operation is what makes the Constitution “alive”.

What should happen is that opinions are tested against the immovable Constitution. Those that accord with it are upheld, those which are at variance dismissed. Once this process is abandoned, anything goes.

Add to this that our guardians, the justices of the Supreme Court, are chosen by identity politics and not intelligence, ability, or even wisdom. And they cannot be fired for incompetence or idiocy. Thus a justice can say

At the heart of liberty the right to define one’s own concept of existence, of meaning, of the universe, and of the mystery of human life

and not be run out of town covered in tar and feathers. That quotation, incidentally, is one cherished by every inmate of Bedlam, and for obvious reasons.

So Happy Fourth, all. Enjoy it while you can. And remember: death isn’t always a bad thing.

From too much love of living,
From hope and fear set free,
We thank with brief thanksgiving
Whatever gods may be
That no man lives for ever;
That dead men rise up never;
That even the weariest river
Winds somewhere safe to sea.


33 Comments
« Older posts

© 2015 William M. Briggs

Theme by Anders NorenUp ↑