Skip to content

Category: Statistics

The general theory, methods, and philosophy of the Science of Guessing What Is.

June 29, 2018 | 6 Comments

Statistics Vs. Artificial Intelligence

This meme which heads today’s post (see discussion under this tweet a modification of an original cartoon) expresses the true distinction between statistics, machine learning, and artificial intelligence.

Which is to say, there is none. Rather, there are lots of differences in practices, but that AI is just a model in the same way non-linear regression is a model. Only AI is far, far more attractive. AI is art, statistics is dull. AI is bleeding edge, statistics is old and crusty.

AI attracts money, statistics repels it.

I imagine some statisticians are still kicking their own keisters over not thinking of putting the AI frame around their models. Computer scientists beat them to it. And have been beating them to it for years. Computer scientists have a genius for creating marketable names for dull and uninspiring models. Of course, statisticians went down the blind Hypothesis Testing Alley (p-values and Bayes Factors) hoping it would lead to the Fountain of Truth. It didn’t, and now they can’t find their way back to Probability again.

As I wrote in the Machine Learning, Big Data, Deep Learning, Data Mining, Statistics, Decision & Risk Analysis, Probability, Fuzzy Logic FAQ (to which I now realize I should have added AI):

What’s the difference between machine learning, deep learning, big data, statistics, decision & risk analysis, probability, fuzzy logic, and all the rest?

  • None, except for terminology, specific goals, and culture. They are all branches of probability, which is to say the understanding and sometime quantification of uncertainty. Probability itself is an extension of logic.

Computer scientists have going for them something statisticians never will, though. The metaphor that computers are brains; or rather, that brains are computers. That’s not true, but it’s so seductive an idea that it cannot be abandoned without much psychic grief.

An abacus does not suddenly become intelligent merely because the number of beads and slides pass some threshold, or are operated at some superior speed. Neither can multiplying a coefficient by a measured value be called “thought.” There is no philosophical difference between a wooden abacus and a computerized calculation. But given we are saturated in science fiction which shows AI (robots etc.) to be just as alive as we are, it’s hard to think our way past it.

Notice how AI is either a victim, our most glorious category, or an evil overlord, our worst? As with all such stories, they say much more about ourselves than our technology.

Anyway, read What Neural Nets Really Are: Or, Artificial Intelligence Pioneer Says Start Over, and especially Our Intellects Are Not Computers: The Abacus As Brain Part I and Machines Can’t Learn (Universals): The Abacus As Brain Part II.

Lastly, and most importantly, did you notice the crack? The Blonde Bombshell taught us Flaubert, which we modify slightly. “Models are a cracked kettle on which we beat out tunes for bears to dance to, while all the time we long to move the stars to pity.”

June 28, 2018 | 9 Comments

Making Random Draws Is Nuts

Let’s do a little science experiment together. Go into the closet and pull out an opaque sack or bag. Anything will do, even a large sock. If you can fit your hand in, it’s fine.

Have it?

Now reach in and pull out a random observation. Pinch between your fingers a piece of probability and carefully remove it. Hold on tight! Probability is slippery. We will call this a “draw from a probability distribution”.


What does yours look like? Nothing, you say? That’s odd. Mine also looks like nothing. Let’s try again, because drawing random observations is what statisticians do all the time. If we didn’t manage to find something, the fault must lie with us, and not with the idea.

The idea is that these “random draws” will tell us what the uncertainty in some proposition is (described below). “Random draws” are used in Gibbs sampling, Metropolis-Hastings, Markov Chain Monte Carlo, bootstrap, and the like, which we can call MCMC for short.

Maybe the sack is the problem. Doesn’t seem to be anything in there. Maybe that’s because there is no such thing as probability in the sense that it is not a physical thing, not a real property of objects? Nah.

Let’s leave that aside for now and trade the sack for a computer. After all, statisticians use computers. Reach instead inside your computer for a random draw. Still nothing? Did you have the ONOFF switch in the OFF position?

You probably got nothing because when you reach into the computer for a random draw you have to do it in a specific way. Here’s how.

Step one: select a number between 0 and 1. Any will do. Close your eyes when picking, because in order to make the magic work, you have to pretend not to see it. I do not jest. Now since this pick will be on a finite, discrete machine, the number you get will be in some finite discrete set on 0 to 1. There is no harm in thinking this set is 0.1, 0.2, …, 0.9 (or indeed only, say, 0.3333 and 0.6666!). Call your number s. (Sometimes you have to pick more than one s at a time, but that is irrelevant to the philosophy.)

Step two: Transform s with a function, f(s). The function will turn s into a “random draw” from the “distribution” specified by f(). This function is like a map, like adding 1 to any number is. In other words, f(s) could be f(s) = s + 1. It will be more complicated than that, of course, but it’s the same idea. (And s might be more than one number, as mentioned.)

Step three: That f(s) is usually fed into an algorithm which transforms it again with another function, sort of like g(f(s)). That f(s) becomes input to a new function. The output of g(f(s)) is the answer we wanted, almost, which was the uncertainty of the proposition of interest.

Step four: Repeat Steps one-three many times. The result will be a pile of g(f(s)), each having a different value for every s found in Step one. From this pile, we can ask, “How many g(f(s)) are larger than x?” and use that as a guess for the probability of seeing values larger than x. And so on.

Steps two-four are reasonable and even necessary because we cannot often solve for the uncertainty of the proposition of interest analytically. The math is too hard. So we have to derive approximations. If you know any calculus, it is like finding approximations to integrals that don’t have easy solutions. You plot the curve, and bust it up into lots of sections, compute the area of each section, then add them up.

Same idea here. Except for the bit about magic. Let’s figure out what that is.

Now instead of picking “randomly”, we could just cycle through the allowable, available s, which we imagined could equal 0.1, 0.2, …, 0.9. That would give us 9 g(f(s))s. And that pile could be used as in Step four. No problem!

Of course, having only 9 in the pile would have the same effect of only slicing an integral coarsely. The approximation won’t be great. But it will be an approximation. The solution is obvious: increase the number of possible s. Maybe 0.05, 0.10, 0.15, …, 0.95 is better. Try it yourself! (There are all sorts of niceties about selecting good s as part of the steps which do not interest us philosophically. We’re not after efficiency here, but understanding.)

This still doesn’t explain the magic. To us, random means not predictable with certainty. Our sampling from s is not random, because we know with certainty what s is (as well as what f() and g() are). We are just approximating some hard math in a straightforward fashion. There is no mystery.

To some, though, random it is a property of a thing. That’s why they insist on picking s with their eyes closed. The random of the s is real, in the same way probability is real. The idea is that the random of the s, as long as we keep our eyes closed, attaches itself to f(s), which inherits the random, and f(s) in turn paints g(f(s)) with its random. Thus, the questions we ask of the pile of g(f(s))s is also random. And random means real probability. The magic has been preserved!

As long as you keep your eyes closed, that is. Open them at any point and the random vanishes! Poof!

“Are you saying, Briggs, that those who believe in the random get the wrong answers?”

Nope. I said they believe in magic. Like I wrote in the link above, it’s like they believe gremlins are what make their cars go. It’s not that cars don’t go, it’s that gremlins are the wrong explanation.

“So what difference does it make, then, since they’re getting the right answers? You just like to complain.”

Because it’s nice to be right rather than wrong. Probability and randomness are not real features of the world. They are purely epistemic. Once people grasp that, we can leave behind frequentism and a whole host of other errors.

“You are a bad person. Why should I listen to you?”

Because I’m right.

June 26, 2018 | 13 Comments

There Is No Prior? What’s A Bayesian To Do? Relax, There’s No Model, Either

I saw colleague Deborah Mayo casting, or rather trying to cast, aspersions on Bayesian philosophy by saying there is “no prior”.

Bayesians might not agree, but it’s true. Mayo’s right. There is no prior.

There’s no prior, all right, but there’s no model, either. So into the tubes goes frequentism right ahead of (subjective) Bayes.

We here at adopt a third way: probability. Probability is neither frequentist nor Bayesian, as outlined in that magnificent book Uncertainty: The Soul of Modeling, Probability & Statistics.

Now specifically, Mayo tweeted “There’s no such thing a ‘the prior’! The onus on Bayesians is to defend & explain the meaning/obtainability of the ones they prefer out of many, types (even within a given umbrella. Is the principle of indifference snuck in?”

Way subjective Bayes works is to suppose an ad hoc, pulled from some mysterious dark realm, parameterized continuous probability model. Which is exactly the same way frequentism starts. Bayes puts ad hoc probabilities on the parameters, frequentism doesn’t. Bayes thus had two layers of ad hociness. Frquentism has at least that many, because while frequentism pretends there is no uncertainty in the parameters, except that which can be measured after infinite observations are taken, frequentism adds testing and other fallacious horrors on top of the ad hoc models.

The models don’t exist because they’re made up. The priors are also made up, and so they don’t exist, either. But a frequentist complaining about fiction to a (subjective) Bayesian is like a bureaucrat complaining about the size of government.

Frequentists and, yes, even objective Bayesians believe probability exists. That it’s a thing, that it’s a property of any measurement they care to conjure. Name a measurement, any measurement at all—number of scarf-wearing blue turtles that walk into North American EDs—and voilà!, a destination at infinity is instantaneously created at which the probability of this measurement lives. All we have to do know this probability is take an infinite number of measurements—and there it is! We’ll then automatically know the probability of the infinity-plus-one measurement without any error.

No frequentist can know any probability because no infinite number of measures has yet been taken. Bayesians of the objective stripe are in the same epistemic canoe. Subjectivists carve flotation devices out of their imaginations and simply make up probability, guesses which are influenced by such things as how many jalapeno peppers they ate the day before and whether a grant is due this week or next month.

Bayesians think like frequentists. That’s because all Bayesians are first initiated into frequentism before they are allowed to be Bayesians. This is like making Catholic priests first become Mormon missionaries. Sounds silly, I know. But it’s a way to fill classrooms.

Frequentists, like Bayesians, and even native probabilists like ourselves can assume probabilities. They can all make statements like “If the probability of this measurement, assuming such-and-such information, is p, then etc. etc. etc.” That’s usually done to turn the measurement into math, and math is easier to work with than logic. Leads to the Deadly Sin of Reification too often, though; but that’s a subject for another time. Point is: there is nothing, save the rare computational error, wrong with this math.

Back to Mayo. Frquentists never give themselves an onus. On justifying their ad hoc models, that is, because they figure probability is real, and that if they didn’t guess just the right parameterized continuous model, it’ll be close enough the happy trail ends at infinity.

Only infinity never comes.

You’d think that given all we know about the paradoxes that arise from the paths taken to reach infinity, and that most measurements are tiny in number, and that measurements themselves are often ambiguous to high degree, that frequentists would be more circumspect. You would be wrong.

The Third Way is just probability. We take what we know of the measurement and from that deduce the probability. Change this knowledge, change the probability. No big deal. Probability won’t always be quantifiable, or easy, and it won’t always be clear that the continuous infinite approximations we make to our discrete finite measurements will be adequate, but mama never said life was fair. We leave that to SJWs.

If I had my druthers, no student would learn of Bayes (I mean the philosophy; the formula is fine, but is itself is not necessary) or frequentism untill well on his way to a PhD in historical statistics. We’d start with probability and end with it.

Maybe that’s why they don’t let me teach the kiddies anymore.

Update I’ll have an article on the Strong law and why it doesn’t prove probability is ontic, and why using it to show probability is ontic is a circular argument, and why (again) frequentism fails. Look for it after the 4th of July week. Next week will be mostly quiet. If you’re in a hurry, buy the book!

June 25, 2018 | 5 Comments

Global Warming: Thirty Years Of Hype, Hysteria & Hullabaloo

In the late spring of 1988, Senator Tim Wirth from Colorado (guess his party) called the Weather Bureau and asked what historically was the hottest day of the year in Washington, DC.

He needed heat for the theater he was cooking up.

He got it. That June, he recalled, was “stiflingly hot.” To give nature a boost, on the night of 22 June, he snuck into the Senate hearing room and opened all the windows. The air conditioning was either switched off or chose that moment to break.

Wirth later boasted to PBS that

when the hearing occurred there was not only bliss, which is television cameras in double figures, but it was really hot…

So [NASA’s James] Hansen’s giving this testimony, you’ve got these television cameras back there heating up the room, and the air conditioning in the room didn’t appear to work. So it was sort of a perfect collection of events that happened that day, with the wonderful Jim Hansen, who was wiping his brow at the witness table and giving this remarkable testimony.

In that manufactured swelter on 23 June 1988, Global Warming was born. Happy Anniversary.

Today, thirty years later, Global Warming is dead. Make that undead. Its corpse still walks among us, awaiting its final stake through its heart.

Winter is Coming

Politicians like Wirth, and the scientists in their employ, had a streak of good luck. Hansen said it would get hotter, and then it did. People doing things like breathing and driving cars were adding “greenhouse gases” to the atmosphere at rates faster than ever before. The correlation between increasing heat and gas was obvious, and it quickly became a cause. Both in the physical and social sense.

There was some reason to believe Hansen was right in those early days. It is a trivial truth that man influences the climate. And scientists had already accepted the idea that man could significantly affect it. A decade before Hansen’s performance, the consensus was that man was driving temperatures dangerously down. Pollution from cars and the like was knocking back the sun’s rays, which was going to cause global cooling.

Global cooling was no small thing. In April 1975 Newsweek spoke fear in “Our Cooling World“. Global cooling was going to cause “serious political implications for just about every nation on earth,” “The drop in food production could begin quite soon,” “devastating outbreak of tornadoes”, “national boundaries make it impossible for starving peoples to migrate from their devastated fields,” and so forth.

Sound familiar?

When All Agreed

The global cooling consensus of the 1970s was small, but not insignificant because climatology was only then developing into a separate field from meteorology and other atmospheric sciences, and all these sciences were small.

That man was having a devastating, irreversible-without-government-action cooling effect on the earth’s climate would later become something of an embarrassment to climatologists. Which is why they did their best push global cooling into the memory hole. But there was so much material, it overflowed. Even Spock warned of the coming snow (don’t miss the earnest interview with Stephen Schneider, who would go on to give many earnest interviews about global warming).

But by the time Wirth staged his event, 1978 was ancient history. We knew lots more about the atmosphere in 1988 and had exponentially better computers. Besides, just look at those thermometers inching up! And did you see that scary hockey stick? We must Save The Planet!

Celebrities Are Our Leaders

It wasn’t long before a politician made an Oscar-winning documentary film about global warming. That the film was saturated with errors and made laughable predictions did not matter. Global warming had to be as frightening as the film portrayed, otherwise the politician-turned-actor hosting it would not have looked so serious.

It wasn’t just politicians. Important people the world over were warming to the coming heat. Actors, musicians, pastors, people famous for being famous, chefs, novelists, school teachers, bureaucrats, activists, academics in such widely varying fields as sociology to psychology, and of course lawyers and news readers. Oh, plus a handful of physical scientists (most kept their heads down).

When celebrities speak, we listen. Even if they’re wrong in the details, that so many elites knew global warming was on its way was sufficient and convincing evidence something had to be done.

Nine Most Terrifying Words

That something was government. More and larger government. The government took up this challenge and did what it did best: it spent money. Lots of it. Soon, every major grant-reliant scientific association, no matter how tenuous its connection to atmospheric physics, issued official statements on how horrible global warming was going to be—once it got here.

Researchers realized you were only half way through this article and that you have to click here to read the rest!