William M. Briggs

Statistician to the Stars!

Page 2 of 427

We Don’t Know Anything

Degrees for everybody.

Degrees for everybody.

The Appeal to Authority is not a formal fallacy, but an “informal” one, a fancy way of admitting that arguments in the form of “Because I said so” are often valid and sound. If these arguments were always a fallacy, there’d be no use asking potential employees for their resumes, no point in asking, “What are my chances, doc?”, really no reason to ask anybody anything about which you are uncertain.

On the other hand, the argument becomes a fallacy routinely in the hands of the media and politicians. Surf over to Slate (I won’t link), tune in to NPR, or listen to Debbie Wasserman Schultz speak on nearly any subject for examples.

So much is common knowledge. And I think fallacious instances of “Because I said so” are on the increase. This is because of many reasons—the usual suspects: scientism, ideology, political correctness, privilege, insularity, etc.—but one occasion for sin, a certain form of the fallacy, is not well known.

This is the form “We now know…”, usually put in service of some sociological, educational, psychological, or other loose science, like the effects of deadly rampant out-of-control tipping-point global warming.

Just like its father, the “We now know…” form of the argument from authority is sometimes valid and sound. A journalist might write, “We now know the neutrino has mass…” and cite some press release put out by some university. The journalist will be right, because in this case (you’ll have to trust me) the claim is true. But the “we” part is risible. The problem is not just that the reporter himself boasts indirectly of an expertise he does not have and has not earned, but that he encourages the same flippant behavior in his audience. And the audience, duly flattered, makes itself part of the “we”. “We now know” is then on everybody’s lips.

For many propositions from the hard sciences, as said, this is mostly harmless, because the “We now know…” won’t be fallacious. The problem is that the knowledge comes cheap and is thus subject to easy misinterpretation and incorrect extrapolation. This is because complex scientific propositions are usually highly conditional, filled with technical premises and other presuppositions, and these rarely make it to the popular level. People go off half cocked, as it were.

Actual hard scientists, in their own fields of competence, rarely fall into the trap, not taking anybody’s word for anything which they can prove for themselves. And so knowledge in the fields manned by rigorous technicians increases. But since nobody bats 1.000 and not every claim can be personally checked, the occasional error slips by.

No, the real problem, as usual, comes from fields which make fewer demands on their practitioners, and fewer still to none on their popular audiences. It’s going to be a man of some mental training who bothers to seek out and to read anything about neutrinos. But sociological claims and the like are available to one and all. Indeed, they are hard to escape, like (bad) music in restaurants.

The problem starts at the “top”. Here’s a typical example, the paper “Taking a Long View on What We Now Know about Social and Environmental Accountability and Reporting” in the Electronic Journal of Radical Organisation Theory. The paper is filled with “We now know…” propositions which are at best only sketchily supported, and others that are only wild surmises. Results from papers like this are fed to students and the public, and those who take joy in that most vague of notions “sustainability”, will uncritically add the propositions to the list of things “We now know…”

You can’t really blame the students, the dears, at least not fully. The serious fault is with inexpert experts, a large and growing class, a growth given impetus by the swelling of higher education. More people earning a “degree” means more professors, and since the gifts of intelligence are varied, this means a necessary expansion in “degrees” which require less effort (from both parties). It is in these fields the “We now know…” is mainly found. Compounding the problem is that the students who carry these “We now knows…” feel that their beliefs have been certified by their degrees.

The solution would thus appear to be a return to (or increase in, since it still partially exists) some idea of educational elitism, the idea that some forms of knowledge are better or more important than others. But give our insatiable craving for Equality, I don’t see it happening.

Paper Claims Surprisingly Strong Link Between Climate Change And Violence. Nonsense.

The official numbers

The official numbers

When does more crime happen, in winter or summer? Why? Too easy. How about this one: according to the FBI, what was the violent crime rate over time? No need to guess. It’s pictured above. The per capita all violent crime percent from 1960 to 2012 (the last year available). Looks to be coming down some since 1991, wouldn’t you say? (The plots for other crime types, including gun crimes, all have the same general shape.)

Say, isn’t the time range of this plot the period where the our-of-control global warming “climate catastrophe” began in earnest? Let’s look at what NOAA’s GISS says:

The official numbers

I’m not in the least interested in arguing about this data; for the sake of argument, let’s just accept it as it is. Look, however, only at the black dots, which are the actual data. The red line is a smoother, i.e. a model, and is not what happened. The model is not the data! Don’t smooth your time series data! (Look here and here for why.)

Let’s tie it all together. Does it look to you like climate change is “correlated” with the violent crime rate? If you’re Chris Mooney or an academic hot for a sensational paper or a member of the media anxious to signal your cooperation with government, you must say yes. Us ordinary folk, not addled by ideology, will say no.

The Washington Post put up yet another fantasy of Mooney’s entitled “There’s a surprisingly strong link between climate change and violence“. I don’t mean to be snarky, I really don’t. But this guy routinely provokes me beyond my ability to resist. May the Lord forgive me.

Mooney cites some new meta analysis, a study I’ll dissect in due course, “of the existing research examining the relationship between climate change and violence and conflict.” Here’s the meat:

Climate variables considered in these papers included temperature increases as well as drought and rainfall changes. Conflict was analyzed in terms of clashes between individuals (like fistfights) and fights between groups (like wars). After taking it all in, the authors found compelling evidence of a link between changes in temperature and increases in conflict, noting that “deviations from moderate temperatures and precipitation patterns systematically increase the risk of conflict, often substantially, with average effects that are highly statistically significant.” Bottom line: In an ever warming world, expect more wars, civil unrest, and strife, and also more violent crime in general.

Yes, that makes sense. A statistical model which analyzes simultaneously fist fights and wars. Almost as sensible as measuring how eight-year-olds spend their allowance and the machinations of the World Bank. Hey! It’s science!

The lesson is: never ever not ever never never believe a meta analysis at its face value. It is one of the most abused statistical techniques. Smoothing time series data is another. Never mind.

Mooney gets one thing partly right when he asks, “Why do hotter temperatures produce more violence?” The obvious answer—as long as we factor out all modern wars, many of which inconveniently occur in winter; in olden days, winter made it difficult to fight; who could have guessed?—is the one we started this post with. People are out in the summer’s long warm days, and inside in the winter’s short cold days. Easy.

Yet not so easy for Mooney and for academics for whom the obvious is never good enough.

Now I would have ignored the article, putting it down as yet another attempt to prove our lying eyes aren’t seeing what they’re seeing (the two graphs above). But Mooney had to go and mention baseball. (I’m a Tigers fan. I don’t want to talk about it.) Mooney thinks a paper he uncovered is terrific proof that climate change makes us more violent.

He quotes from the awful peer-reviewed paper “Temper, Temperature, and Temptation: Heat-Related Retaliation in Baseball” in Psychological Science (2011; 22(4) 423­–428) by Richard P. Larrick and some others. Larrick checked whether increasing temperatures were associated with more beanballs. The authors admitted they were not.

So, their theory busted but still desiring a paper, the authors had to try something else. How about retaliation? Do increasing temperatures cause more? Mooney shows a graph from their paper which is so silly that I refuse to picture it. He presents this graph, as do the authors, as if it were data. Which it is not. It is the output from a preposterously complex regression model (they “control” for 13 things!).

Baseball fans: when do more beanballs, and hence more retaliations take place, in chilly April when the season has just begun and all are of good cheer, or late in hot August when tempers are up and when games start to feel a lot more crucial? Is the observed discrepancy therefore caused by climate change?

Good grief, what a rotten paper, what a rotten theory.

The Problem Of Grue Isn’t; Or, A Gruesome Non-Paradox About Induction

This emerald does not appear to be green, nor grue. Maybe Goodman was right!

This emerald does not appear to be green, nor grue. Maybe Goodman was right!

Skepticism about induction happens only among academic philosophers, and only in print. Tell an induction skeptic to take a long walk off a short dock or hint that his health insurance will be cancelled and you will find an immediate and angry convert to Realism.

Some philosophers come to their skepticism about induction from puzzles which they are unable to solve and reason that, since they cannot solve the puzzles, it’s a good bet to side with skepticism. Well, in some ways this is natural.

A classic puzzle is Nelson Goodman’s “grue”. Goes like this. Grue is a predicate, like green or blue, but with a built-in ad hoc time component. Objects are grue if they are green and observed before 21 October 1978 or blue and observed after that date. A green grape observed 20 October 1978 and a blue bonnet observed 22 October 1978 are grue. But if you saw the green grape yesterday, or remember the blue bonnet from 1976, then neither are grue. The definition changes with the arbitrary date.

So imagine it’s before the Date and you’ve seen or heard of only green emeralds. Induction says future, or rather all unobserved, emeralds will also be green. But since it’s before the Date, these emeralds are also grue, thus induction also says all unobserved emeralds will be grue. Finally comes yesterday—and lo!—a green and not a blue emerald appears, thus not a grue emerald. Induction, which told us it should be grue, is broken!

There have been several exposures of the grue fallacy before, and up until the other day (another date!) I had thought David Stove’s in his Rationality of Induction was best. But I now cast my vote for Louis Groarke’s in his An Aristotelian Account of Induction. He calls belief in Goodman’s fallacy “an adamant will to doubt rather than an evidence-based example of a deep problem with induction” and likens it to the fallacy of the false question (e.g. “Have you stopped cheating on your taxes yet?”).

Groarke says (p. 65):

The proposition, “emeralds are grue,” [if true] can be unpacked into three separate claims: emeralds are green before time t (proposition1); emeralds are blue after time t (proposition2); and emeralds turn from green to blue at time t (proposition3). Goodman illegitimately translates support for proposition1 into support for proposition2 and proposition3. But the fact that we have evidence in support of proposition1 does not give us any evidence in support of all three propositions taken together.

What does the arbitrary time have to do with the essential composition of an emerald? Not much; or rather, nothing. The reason we expect (via induction) unobserved emeralds to be green is we expect that whatever is causing emeralds to be green will remain the same. That is, the essence of what it is to be an emerald is unchanging, and that is what induction is: the understanding of this essence, and awareness of cause.

Groarke emphasizes that the time we observe something is not a fact about the object, but a fact about us. And what is part of us is not part of the object. Plus, the only evidence anybody has, at this point in time, is that all observed emeralds have been green. We even have a chemical explanation for why this is so, which paradox enthusiasts must ignore. Thus “there is absolutely no evidence that any emeralds are blue if observed after time t.”

Two things Groake doesn’t mention. First is that, in real life, the arbitrary time t is ever receding into the future. I picked an obviously absurd date above; it’s absurd because we have all seen green emeralds but no blue ones up to today, which is well past 1978. The ad hoc date highlights the manufactured quality of the so-called paradox. When, exactly, should we use a grue-like predicate for anything?

Secondly, nobody not in search of reasons to be skeptical would have ever thought to apply a predicate like grue to anything. It is entirely artificial. If you doubt that, consider that you can substitute any other predicate after the arbitrary date. It doesn’t have to be blue. Try salty, hot, tall, or fast. An emerald that is green up until t then fast? That’s ridiculous! Yes, it is.

After showing the paradox isn’t, Groake goes on to explain the possible reasons why the paradox has been so eagerly embraced. Cartesian corrosion. That bottomless skepticism which dear old Descartes introduced in the hope of finding a bedrock of certainty. There isn’t space here to prove that, but anybody who has read deeply in epistemology will understand what that means.

Update A glimpse of how much angst the “problem” of grue has created, try this (or similar) searches. Also note the New & Improved title.

Strangers In A Strange Land: Archbishop Charles J. Chaput 2014 Erasmus Lecture


Just as did the Most Reverend Charles J. Chaput, O.F.M. CAP., Archbishop of Philadelphia, I am addressing my comments to the remnant. All are welcome to listen, but there will be much I won’t explain.

Archbishop Charles Chaput delivered the First Things Erasmus lecture last night, at the stately Union League Club on Park Avenue. Your intrepid reporter was there. I’m delighted that jack and tie were required; jeans were forbidden. The speech is on line, so instead of relating what his excellency said, what follows is a discussion of his main points.

The title, chosen for its topical relevance and because Chaput is a science fiction fan (I wonder if he knows our Mike Flynn?), describes us. In the world, but not of the world. I can’t quite agree. To me, it feels rather like barbarians have stormed the gates, which were left unlocked and unguarded. It’s our fault they’re here. Well, it used to be a free country. I only wish our guests would be better behaved.

Part of Chaput’s family hails from Quebec, which in 1950 saw 90% of the population attending weekly mass. Now it’s 6%. Sacré bleu no more. Now preaching that homosexual acts are a sin is a hate crime. Hate? Progressives hate being told they’re wrong. They won’t stand for it and they will punish you. No creature on earth has a thinner skin.

Anti-Catholic prejudice in these once United States historically ranged from virulent to mild to practically nonexistent. Chaput predicts its return. Chaput sets the “tipping point” as this past 6 October, when the Supreme Court punted on same sex “marriage”, a non-event which was

the dismemberment of privileged voice that Biblical faith once had in public square…The most disturbing thing about the debate around gay marriage is the destruction of public reason that it has accomplished. Emotion and sloganeering drove the argument. And the hatred that infected the conversation came far less from the so-called homophobes than from the many gay-issue activists themselves. People who uphold a traditional moral architecture for sexuality—marriage and sexuality—have gone in the space of just twenty years from mainstream conviction to the media equivalent or racists and bigots. Now this is impressive. It’s also profoundly dishonest. And evil.”

Remember when stores used to have signs which read “We reserve the right to refuse service to anyone”?

You bigot. Now even some of those who call themselves libertarian insist no one has that right. Not when the customer is a member of an officially designated victim group. Christian bakers must bake cakes for same-sex “weddings”, and must even attend reeducation camps for having the temerity to believe their religious convictions trump the “right” of people to pastry on demand.

Remember when we were told that nobody would ever force Christian ministers to perform a same sex “marriage”?

You fool. Two ministers have been threatened with jail for refusing to perform same sex “weddings”. Read about the Secular Inquisition here and here and here. Remember Brandon Eich (here and here)?

It’s always fun to put to progressives questions like, “So if the local KKK went to a black sign maker asking that business to print anti-black messages, then that owner does not have the right to refuse? Or if the Westboro Baptist Church sauntered into a restaurant in San Francisco’s Castro district, a restaurant run by LBGT owners and which often rents itself out to private groups, and demanded to hold their annual anti-homosexual meeting there, those owners have no right to refuse?”

It’s fun because you will find suddenly that the libertarian or progressive has an appointment he can’t miss; or you will hear the Distraction Fallacy. “Priests abused kids!” Like Chaput said. Emotion and sloganeering. Reasoned argument no longer has a place.

We need a better word for the enemies of Christianity. I suggest the old standby pagan. Chaput himself called the fallen Catholics of Canada “baptized pagans.” It is an apt word. It describes the coming world well. A self-infatuated oligarchy lording over a mass of self-infatuated people who eschew religion but embrace “spirituality”. Yoga, anyone? “Religion,” Chaput said, “is [now] just another form of self medicating.”

Democracy guarantees this outcome (this is me, not Chaput). This isn’t the place for a complete explanation, but here is a sketch. When the populace more-or-less agreed on Christian fundamentals, voting made sense; consensus was possible. But now that Christianity is ebbing, it must be replaced by something else. People do not vote based on nothing. We’re split now, a Civil Culture War, but the pagans will surely win.

What these earnest intolerant people don’t understand is that many of their notions are still Christian. As Chaput said, it was only twenty years ago that most pagans held the traditional Christian view of sexuality. That’s gone. But the pagans still hold the Christian view of the sanctity of life—for those who escape the womb—and of the family, and they have yet an instinctive respect for learning, ideas, and reasonable disputes. These are going.

Emotion and sensuality (Chaput’s word) will rule individuals. The intelligent, which increasingly means the rich, know how to manipulate emotions. People will vote cheerfully for their own demise and enslavement. The only possible escape I see is some crisis in which the classic Strong Man emerges, and either dictatorship or kingship arises.

Chaput’s solution? Well, what’s our goal? You already know. Thus prayer and joy and hope. Worship. Eliminate clericalism in clerics and the laity. Eliminate laziness in the laity and their instructors. We all also already know the principles by which we should live. Live them and don’t try and fit in.

As Tiny Tim said, God bless us, everyone!

Update We’ve heard from some of our non-Christian (but post-Christian) friends, but none so far have chosen to answer the KKK-Westboro hypotheticals. Of course, it’s a better question to put to progressives.

Update Chip convinced me about some libertarians, so I’ve modified above to “some”.

Please Don’t Smooth Your (Social Media) Data!

Friends don't let friends smooth.

Friends don’t let friends smooth.


Don’t smooth your data and then use that smoothed data as input to other analysis. You will fool yourself. You will make over-confident decisions. It is the wrong thing to do. It is a mistake. It is a guarantee of over-certainty. I don’t know how to put it more plainly. Lord knows I have tried. See below for a non-success story.

Smoothing means any kind of modeling, which includes running means, just-plain-means, filtering of any kind, regression, wavelets, Fourier analysis, ARIMA, GARCH; in short, any type of function where actual data comes in and something that is not data comes out.

Do not use the something-that-is-not-data as if it is data. This is a sin.

Don’t believe me. Try it yourself. The picture is from an upcoming paper I and some friends are writing.

It shows two simulated normal noise time series, with successively higher amounts of smoothing applied by a k-rolling mean. From top left clockwise: k = 1, 10, 20, 30; a k = 1 corresponds to no smoothing. The original time series are shown faintly for comparison. The correlation between the two series is indicated in the title.

More smoothing equals higher correlations. Since there are no causes between these series, the correlation should be hovering around 0, which it is in the first panel. And that correlation stays near 0—for the original real not fake un-smoothed data. But if you calculate the correlation between the smoothed series…the sky’s the limit!

Now it is not true that in each and every and all instances that smoothing will increase the correlation between two smoothed series. It might be that (in absolute value), for your one-time smoothing, correlation decreases or stays put. But it usually will increase, and usually by a lot.

Why? Imagine any two straight lines with non-zero slopes. These two straight lines will have perfect Pearson correlation, either +1 or -1. Regression and other measures will also show perfect agreement. The proof of this is trivial, and I leave it as an exercise (don’t be lazy; try it). Smoothing makes time series data look more like straight lines, as the pictures show. Simple as that.

There are all manner of fine points I’m skipping and would make wonderful Masters projects. Just what kind of data and what kind of smoothing and what statistical measures are affected and by what magnitude? All these questions are quantifiable and will make for fun puzzles. My experience with actual data and actual smoothing and typical measures shows that magnitude is large.

It happens

Now, without betraying any confidences, let me tell you of the latest in a long and growing string of bad examples. Two companies, one internationally known for their quantitative prowess, another even better known for its ability to make vast wads of money. Call them A (stats) and B (client). I did not work for either A or B, but know and advised certain parties.

B advertised and wondered how much of an effect this had on its measure of success. A said they could tell, using sophisticated Bayesian models incorporating social media data.

Social media!

Wowzee! Tell people you have busted open the secrets of social media and they will dump buckets of cold cash on you. Hint: everybody who says they have it figured out is either exaggerating to themselves or to their clients. (Say, that’s a pretty bold statement.)

Anyway, smoothing occurred. And correlations greater than 0.95 were boasted of. I’m not kidding about this number. Company A really did brag of enormous “impacts” of its smoothed measures. And Company B believed them—because they wanted to believe. Sophisticated Bayesian models incorporating social media data! How could you go wrong?

The real correlations, using unsmoothed data, were near 0. Just as you’d expect them to be for such noisy data as “social media” predicting a company’s measure of success. Do you really think Twitter streams contain magic?

I told all involved. I explained pictures like those above. I was emphatic and clear. I stood neither to gain nor lose regardless of the decision. Only two people (at B) believed me, neither of whom were in a position to make decisions.

At least I am comforted that Reality is my friend here. The company’s will eventually realize, but probably never admit, that their measures are spurious. Because they will realize but not admit, these measures will be quietly abandoned…

…As soon as the next computer self-programmed big data machine learning artificially intelligent smart-phone-data algorithm comes along and seduces them.

« Older posts Newer posts »

© 2014 William M. Briggs

Theme by Anders NorenUp ↑