Julian Champkin, editor of Significance magazine somehow came across the percipient insights of yours truly and asked me to write l’article controversé. Which I did. And with gusto. Champkin, a perspicacious individual with the insight and experience of one long accustomed to the peculiarities and peccadillos of publishing, added the word “silly” to the title. I find myself not objecting.
The article is here. I beg forgiveness that reading the piece requires a subscription (yours or an institution’s).
Statisticians are, in actual fact, as an Englishman would put it, Significance being an organ of the Royal Statistical Society, in the bad habit of answering questions in which nobody has the slightest interest. More rottenness is put forth in the name of Science because of the twisted cogitations of statisticians than because of any other cause.
The problem is that the questions statisticians answer are not the questions civilians put to us. But the poor trusting saps who come to us, on seeing the diplomas on our walls and upon viewing the perplexing mathematics in which we couch our responses, go away intimidated and convinced that what we have told them are the answers to their queries. They can’t, then, be blamed for writing results as if they had received the One Final Word.
There are many reasons why we lead our flocks astray, but the main culprit is we instill a sort of scientific cockiness. A civilian appears and asks, “How much more likely is drug B than drug A at curing this disease?” We do not answer this. We instead tell him which drug, in the opinion of our theory, is “better”, imputing a certainty to our pronouncement which is unwarranted.
We’re tired of these examples, but they are paradigmatic. It is through the wiles of statistics that sociologists can “conclude” that those who either watch a 4th of July parade or who see, oh so briefly, a miniature picture of the American flag can turn one into a Republican.
The old ways of statistics allowed over-certainty in the face of small samples sizes. The new ways of doing statistics (now not always called statistics, but perhaps artificial intelligence, data mining, and machine learning) allows over-the-top surety in the face of large sample sizes, a.k.a. Big Data. The difference being that the later methods are automated, while the former are hands-on. False beliefs can now be generated at a much faster rate, so some progress is being made.
If you followed last week’s “Let’s try this again” on temperatures, you’ll have an idea what I mean about over-certainty (incidentally, due to time constraints, I will not be able to answer questions posted there until tomorrow). Also click the Start Here tab at the top of this page and look under the various articles under Statistics.
Update Posting date change to allow more comments.
Dr. Briggs,
Ignoring the very small possibility that I have a subscription to Significance of which I am unaware, your article appears to be freely available to all, if my ability to download is any indication.
I’d suggest that your next thesis should be: Why Do People Who Posses Statistical Programs Believe That They Can Use Them Properly? (I’m not a statistician, by the way, but rather have been a person in possession of statistical programs who has used them improperly at times.) I’m not sure how much substance it would have, but it would be more broadly applicable to our times.
Case in point: your article on temperatures. The vast majority of those making claims about temperatures are not statisticians. In fact, that’s been a skeptical criticism of climate science: too many ad hoc methods and too few collaborations with statisticians.
I can echo what Big Mike says. I was able to download and read it. Is there a typo in the first sentence, though?
Wayne,
Great news about availability!
If it’s an article from me, the probability of a typo is high.
Very much enjoyed the article, by the way.
Slightly off-topic, but did I miss where you discussed the arguments counter to those that you presented in the “Time Series” articles? I don’t feel that the insight I feel I gained from that excellent sequence of articles can be complete without them.
I would have been inclined to answer the question in your title by saying “because the questions people do ask are even sillier”. And I would be thinking then of questions like “What is the probability that A is better than B?”
But in the article you say “a civilian needs little or no maths to understand what ‘the probability that A is better than B is 80%’ means”. So please tell me.
What *does* it mean?
The article is clear enough that a “civilian” could understand it. Until the last two columns where the civilian won’t understand.
Who is the academic teaching? Stats is math. Statisticians should be able to handle it. Civilians can’t be expected to, so they should not be using statistics. Almost every scientific field uses statistics (and a few pseudo-scientific ones) — how many of these users of statistics actually understand what they are doing? If they are civilians, what are they doing on the battlefield?
I hope the CAGW guys projecting linear regressions on time series don’t use this to justify their “predictive methods”.
The problem is when the statistician doesn’t get asked.
Alan,
It means the evidence is such that the probability “A is better than B” is 80%. Which is greater than 0% but less than 100%. Nothing more.
Outlier,
Of course, statistics isn’t math. Statisticians use math but so do accountants and carpenters. The primary results of statistics, a.k.a. probability, are epistemological not mathematical.
It is the great confusion that statistics is math which causes all the problems. Simply “submit” data to the statistical procedure and the math that pops out is taken as true, complete, final.
It still jars me when I read “maths”. Is there more than one math? And what’s up with “maths is”? We won in 1776. Don’t bow down to the royalists!
On Wikipedia, there is an entry for “Predictive Analytics”
http://en.wikipedia.org/wiki/Predictive_analytics
Are those the sort of techniques you are suggesting folks use? I have used most of the techniques, but I’m skeptical they would offer a framework for reaching objective conclusions.
“They can’t, then, be blamed for writing results as if they had received the One Final Word.”
Really? Are they obligated to be gullible dolts all their lives? Credere Non.
The opening paragraphs of this article are strangely reminiscent of the first scenes of the BBC TV Series “Yes, Minister”. The newly elected government’s minister (James Hacker) is introduced to the head of the ministry (Humphrey Appleby) by Hacker’s personal secretary (Bernard Wooley). It goes something like this:
BW: This is Humprey Appleby, ….
JH: Oh, yes. We’ve met at inquiries I sat on while in opposition.
HA: Good day, Minister.
JH: I was impressed by HA as he’d always answered my questions.
HA: I’m glad that you think so, Minister.
SteveBrooklineMA says:
13 February 2012 at 10:00 am
It still jars me when I read “mathsâ€. Is there more than one math? And what’s up with “maths isâ€? We won in 1776. Don’t bow down to the royalists!
===============
But maths is merely an abbreviation for mathematics, which has many mansions. Such as the noble art of geometry, and number theory, abstract algebra, ….. , down to the servants quarters where you find such things as wave mechanics, vector calculus, Hamiltonians, and similar deviances. Why! There are even those who claim it includes statistics 🙂
Steve,
Well, that Wiki page isn’t what I mean. The problem is “predictive statistics” has two meanings, the term was developed by two camps independently. Neither is wrong, but I suppose the one I mean is the worst usage. I mean, “our side” should be the one to give way and develop a new term. No current term (that I can think of) is especially beautiful, and nobody likes the trouble of learning a neologism. And I am not the guy to think of one, either. I’m all ears, however.
So your explanation of what “the probability that A is better than B is 80%” means is that “the evidence is such that the probability ‘A is better than B’ is 80%”. Thanks.
Now I also have a perfectly clear explanation also of what “the hoy is gerflumptive” means. It means that “the evidence is such that the hoy is gerflumptive”.
So many years wasted in study… but what’s worse is the thousands of others I’ve led into confusion by avoiding this simple universal explanation, and the thought that I’ll never be able to track them all down to take it back.
Stats is math. I don’t expect you to agree with me … but you should. Stats is a mathematical tool with an epistemological usage. I will go further out on that limb and even claim that science is math. There is hand-waving and there is science. The difference really comes down to that the science is based on a quantifiable model and that model is expressed by mathematics. If someone thinks that stats is plugging numbers into a program and that the result which “pops out is taken as true, complete, final”, we have a problem. There is not enough depth around. Too many people are building upon a foundation that they don’t have a clue about, based on assumptions they don’t have a clue even exist, not to mention what happens when they are not true.
Here are two entertaining rants, the first by Luboš Motl and the second linked by him.
http://motls.blogspot.com/2012/01/can-maths-in-physics-be-simpler-than-it.html
http://www.thebestpageintheuniverse.net/c.cgi?u=math
Outlier,
Here’s our pop-quiz for today:
If “Most scientist believe physics is math & Motl is a scientist” what is the probability that “Motl believes physics is math”? No math is needed to answer.
I counter that statistics, physics, even math are branches of logic.
Alan,
I wasn’t being glib. Probability (see above) is a measure of truth, or closeness to truth. 80% is closer than 70% and less close than 90% to being true. What you do with this number is different than what the number is.
I’m sorry, but giving “closeness to truth” as a definition of probability *is* glib.
(It’s also more than 75% wrong in that I can think of at least three measures of closeness to truth that are more common than anything to do with probability.)
Alan,
Really? What are they?
“I counter that statistics, physics, even math are branches of logic.”
Agreed, that is the evolution of reason (IMHO). The reasoning wasn’t very good at first. (Angels must have wings, for how else would they fly?) Philosophy improved on it a bit.
(The southern continent must exist to balance out the northern continents) And science improved it still more (man is evil). Oops, not everyone is good at logic.
In common language (as per my claim):
1. an approximate answer is often referred to as close to the truth
2. a false statement is sometimes referred to as close to the truth if its error arises from a fairly common misuse of terminology
3. a detective may be said to be getting close to the truth if he has a good idea of where to look for the deciding piece of evidence
etc.
Alan,
But two of these examples are non-probabilistic.
1. Given our background knowledge, an approximate answer is likely true
3. Ditto
2. You’ll have to clarify this. A falsity is not close to a truth; a mistake is still a mistake.
They were intended to be non-probabilistic as I was giving them as examples of why “closeness to truth” is not a good definition of probability.
1. The statement that the circumference of a circle is six times its radius has zero probability of being true but it is close to the truth.
3. Knowledge of the fact that the murdered duke wrote a deathbed note which will tell me whether it was Colonel Mustard or Professor Plum who poisoned him brings me closer to the truth without increasing the probability of either hypothesis.
2. Your attempt to define probability as “closeness to the truth” may be close to the truth but it has zero probability of actually providing a useful definition.
Alan,
I assume you meant your “2” as a joke, but it has backfired on you. In a useful way, however. Let’s see.
1. A = “The circumference of a circle is six times is radius.” Now, there is no such thing as
Pr(A).
But we can calculate:
Pr(A | E) = 0
where E = “My knowledge of geometry as might be found in any high school or higher text”. Notice that this is completely different than B = “A is a good approximation”. We still cannot calculate
Pr(B).
But we can calculate:
Pr(B | E & F) = 1
where we have the same E plus information F = “A good approximation is being within plus or minus 20% of the radius” or some other F (different F might change the probability, of course).
3. A = “Duke says M or P killed him”. If B = “M killed the Duke” then
Pr(B | A) = 1/2
and similarly for C = “P killed the Duke.” The probability
Pr(Duke was murdered | evidence of dead body & foul play) = 1
which is the same as
Pr(Duke was murdered by somebody | evidence of dead body & foul play) = 1.
But we cannot compute
Pr(Duke was murdered by M | evidence of dead body & foul play) = unknown,
unless we condition on something more, namely a list of suspects.
2. I could write this out, but you’ll get the idea. The probability that I have provided you the true definition, given all this (and other information on the blog) is 1.
I once subscribed to Significance, but only 5% of the issues showed up.
All I can say is that I think you must have missed my point – which was that there are common language senses of “closeness to truth” which have nothing to do with probability, and so that “closeness to truth” is not a good definition of probability.
This all started when I asked you what you would say ‘the probability that A is better than B is 80%’ means, and so far I haven’t seen anything not glib in response.
Alan,
I haven’t; you have failed to make yours. In order to disprove my thesis, you need to show an example that can’t be written in the forms (for example) that I’ve given.
Thanks for trying, but I don’t understand what you are saying. If you have given an intelligible answer to my question about the meaning of probability then I guess I’ll just have to accept that the subject is beyond me.