Skip to content

Category: Philosophy

The philosophy of science, empiricism, a priori reasoning, epistemology, and so on.

February 12, 2010 | 22 Comments

All Models Are Not Wrong

George Box said, “Remember that all models are wrong; the practical question is how wrong do they have to be to not be useful.”

This is usually misremembered as the equivalent, and pithier, “All models are wrong, but some are useful.”

Both versions are false. This is because the premise “All models are wrong” is false: all models are not false. Here is why.

First, readers should keep in mind that what follows is a philosophical argument. My burden is solely to show that there exists at least one model that is true. Whether the model I show is useful or interesting to you or not is irrelevant.

In math, if somebody starts a theorem with the statement, “For all x…” and later comes along a fellow who shows an x that, when plugged into the theorem, ends up in a different place than predicted by the theorem’s conclusion, then that theorem has been invalidated. In simpler words, it is false.

It might be that there is only once such x that invalidates the theorem, and that for all other x the theorem holds. But you cannot say, “The theorem is practically true.” Saying a theorem—or argument—is practically true is like saying, “She is practically a virgin.”

But all that is just boilerplate, because it’s going to turn out that many models are true and useful.

Suppose Model A states, “X will occur with a probability that is greater than 0 or less than 1.” And let Model B state that “X will occur”, which of course is equivalent to “X will occur with probability 1” (I’m using “probability 1” in its plain-English, and not measure-theoretic, sense).

Now, if X does not occur, Model B has been proved false. The popular way to say it is that Model B has been falsified. If X does occur, Model B has been proved true. It has been truified, if you like.

How about Model A? No matter if X occurs or not, Model A has not been falsified or truified. It is impossible for Model A to be falsified or truified.

It is a logical fact that if you cannot prove something true, you also cannot prove it false. We cannot prove Model A is true, therefore we also cannot prove it is false. Box’s premise assumes we have proof that all models are false. But here is a model which is not known to be true or known to be false.

Therefore, Box was wrong. We do not know that all models are false.

Many will object that I only allowed X to have one chance to occur. And the more training you have had in science, the temptation to ask for more chances irresistibly grows stronger: frequentist statisticians will insist on there being infinite chances for X, which requires that X be embedded in some uniquely definable sequence.

Resist this urge! There are many unique events which we can model and predict. Example: Hillary Clinton wins the 2012 presidential election. There is only one Hillary Clinton and she only has one chance to win the 2012 election (if you don’t like that, I challenge you to embed this event into a uniquely defined infinite sequence, along with proof that your sequence is the correct one).

For fun, let’s grant X more chances to occur. First know that all probability models fit the schema of Model A. (Note: there is no difference between probability and statistics models; the apparent division is artificial; however, you do not need to believe that to follow my argument. There is also no difference between probability and mathematical or physical models: models are models; you don’t have to believe this yet either.)

But it doesn’t matter how many times a probability model makes a prediction for X: it can never be proved true or be proved false, except in one case. Excepting this case, the more numbers of chances for X doesn’t change the conclusion that Box was wrong.

The exception is that a probability model—or any other kind of model—can be proved false if you can prove the premises of that models are false. Sometimes this is easily done. For example (see the post from two days ago), we know that grade point can only live on 0 to 4, but we use a normal distribution (premise) to model it. We know, given the properties of normal distributions, that it is false. Yet the model might still be a useful approximation (what happens here is that we internally change the model to an approximate version, which can be true).

And some probability models have true premises. Casinos operate under this belief. So does every statistician who has ever done a simulation: what is that but a true model? Physicists are ever after true models. And so on.

If Box had said, “Most models are wrong, but some are useful,” or, “All the models that I know of are wrong, but some are useful” then what he said would have been uncontroversial. I suggest that it is these modifications that most people actually hear when presented with Box’s “theorem.” Since it’s mostly true—to statisticians playing with normal distributions, anyway—they feel it’s OK to say completely true. However, they never would accept that lack of rigor in any of their own work.

That’s all we can do in 800 words, folks. We’ll surely come back to this topic.

Update The model for a unique-physical-measurable event has been objected to on the grounds of simplicity. These kinds of models do not have to be accepted for the main argument to be true (though I believe they are sufficient).

First recall that simulations are true models all. To add the premise “and this simulation is meant to represent a certain physical system” might make the model false. This has been discussed in Bernardo and Smith in depth (open versus closed universes, etc.), and I ask the reader to rely on those gentlemen for particulars.

However, even these extensions are not needed. Consider this: to say “all” models are false implies that a rigorous proof of this exists. So if I have a Probability Model A (for, let us suppose, a non-unique, physical, measurable, observable event), you cannot, in finite time prove that this model is false. No observation can do so, nor can any collection of observations.

The best you might do is to say “Model A is probably false.” But to say “probably” means to admit that a proof of its falsity does not exist. And again, to say “practically false” is equivalent to say “practically a virgin.”

So again, not all models are false. And all probability models are not—or, that is, they cannot be proven so.

Neither can, it might interest the reader to learn, climate models. They cannot be proven false, even if their predictions do not match observations. This is why we ask for a model to demonstrate skill (see the prior post).

February 7, 2010 | 47 Comments

Ayn Rand and the Differences Between Groups

Roger Kimball is causing a stink, a predictable yet enjoyable stink, by publishing Anthony Daniels’s review of an Ayn Rand biography in this month’s The New Criterion.

There are two enduring internet-subjects on which if any negative criticism appears, no matter how slight, can be counted on to generate tea-cup furies.

The first is Apple computer. For example, question the hubris of Steve Jobs, who last week introduced Apple’s tablet under a looming picture of a stone-carrying Moses descending Sinai, and legions of fanboys will descend upon your site and explain to you just how stupid you are, and why you will always be so since you cannot comprehend the simple logic of how the Israelites would have spent 50% less time wandering had they only been presented their Commandments via the iPad. (In colour!)

It used to be that any negative press of Ron Paul or Obama would produce the same attacks of splenetic fever. But Ron Paul is long dead (so I’ve heard) and the growing perception is that Obama has read from one teleprompter too many.

So we are left with Ayn Rand. Daniel knew the danger going in, which is why he took pains to present ideas of Rand’s which he thought were true. The first: “[S]he was among the first to appreciate that the notion of collective rights (a mirror image of racial discrimination) would ‘disintegrate a country into an institutionalized civil war of pressure groups, each fighting for legislative favors and special privileges at the expense of one another.'”

This was an empirical prediction which experienced has verified, and is therefore true, but not yet universally acknowledged.

Daniels also recommends “her observation that ‘Even if it were proved…that the incidence of men of potentially superior brain power is greater among the members of certain races than among the members of others, it would still tell us nothing about any given individual and it would be irrelevant to one’s judgment of him.'”

This is a (comforting) statement of philosophy and is false. Further, any statistician knows that it is false.

Suppose there are two group, M and N. And to avoid emotion, suppose M and N represent the sales (in dollars) of two rival products. The statement that the evidence shows the “incidence of…superior brain power is greater among the members of” a certain group is translated into “the evidence is that the probability of group M having higher sales than group N is greater than 50%.”

Writing this in traditional notion (for those comfortable with this) gives

    Pr( Sales[M] > Sales[N] | Our Evidence) > 0.5).

Another way to state this is that if you had to guess which product, M or N, would have greater sales, you would maximize your chance of confirming your guess by saying “M.” This does not, of course, prove that the sales of M will be greater. N can still beat M.

Rand would say that the evidence that it is probable that M > N “would still tell us nothing about any given individual and it would be irrelevant to one’s judgment of him.” Here there is only one individual per group (just the sales M and N), but knowing who that individual is tells us something about that individual and is not irrelevant to our judgment of him.

You might object that Rand obviously meant more than one individual per group. So suppose sales of M and N are generated by salesmen. That is, M has a host of salesmen hawking it and so does N. The number of salesmen in each group need not be equal.

Our equivalent evidence is that the salesmen in M sell more than do the salesmen in N. That is, given this evidence, the probability that a salesperson from M outperforms a salesperson from N is greater than 50%. Notationally,

    Pr( Individual[M] > Individual[N] | Our Evidence) > 0.5).

If all—pay attention to this “all”—we knew about the two individuals in front of us is that one sold M and the other sold N, then this would tell us something about these given individuals.

Our knowledge of what group these individuals belonged to would be relevant to our judgment of them. Our judgment is that, given the evidence we have of the two people in front of us, the guy that sold M is probably a better salesman than the guy who sold N.

This, again, does not prove guy M is better than guy N. We could learn new evidence that changes our perception: for example, guy M is drunk.

But in general, Rand’s statement is logically false.

Be careful to understand what we proved. We did not prove that, in comparing different groups of humans, there will be measurable differences. Whether or not there are is an empirical, and not a logical, question. This is what statistics is all about.

What we did prove was that if those differences are real, then that information would be relevant to judgments about members in different groups.

January 31, 2010 | 37 Comments

Tim Tebow’s Super Bowl Ad and the Washington Post Editorial

Very delicate ground, here.

I want to be as precise as I know how in discussing the language used in today’s Washington Post editorial about the upcoming Tim Tebow ad, while trying to avoid the extreme emotions that usually accompany this topic.

The ad is said to feature Tebow and his mother. Tebow’s mother was being treated for amoebic dysentery during her pregnancy and it was feared that the drugs used to treat her illness would cause grave harm to Tebow. She was advised to have an abortion. Obviously, she did not.

This is not just an anti-abortion ad, it is also a reminder that doctors can be wrong. Tebow’s mother makes a statement that having her baby was the correct choice. The implication is that this decision would be correct for some or all other women.

I do not want to discuss the politics of the abortion debate. So it is immaterial to our central topic whether it is right or wrong for CBS to run this ad.

Clarity must be paramount: let us carefully define our terms. The most common euphemism abortion supporters use is “pro-choice.” They mean by this that all women should be allowed to choose to kill their fetuses or not to kill them.

The emphasis is on choice, but it is the act which is at contention. The pro-abortion euphemism is meant to, and does, distract attention away from the act. For our case, this is important because of the way the Post uses this euphemism, about which more in a moment.

Anti-abortion supporters come closer to acknowledging the act of abortion with their slogan of “pro-life.” They mean by this that no woman should be allowed to choose to kill her fetus. The proper word is “kill” because the fetus is alive.

There are, as we all know, gradations and subtleties of both positions. Some anti-abortion people would make an exception and allow abortion if certain conditions held. And some pro-abortion people would disallow abortions if other conditions held. These subtleties are immaterial to our central point.

Which is this: We can take it that all agree that to murder is wrong and is a punishable act. But one can only murder another human. Anti-abortion supporters hold that a fetus becomes human at the moment of conception. Pro-abortion supporters hold that a fetus does not become human until it is delivered from its mother.

This is the point to argue. All other matters fade to insignificance or are political distractions. For example, the Post reports that “Erin Matson, the National Organization for Women’s new vice president, called the Tebow spot ‘hate masquerading as love.'” This is unintelligible philosophically, however revealing it may be politically. Thus, we will ignore it.

Now, if a fetus does become human at conception, then no woman may legally “choose” to kill it, for if she does, it is plainly murder. If a fetus does not become human until birth, then a woman may choose to kill it and cannot be punished for doing so.

It is, of course, possible and coherent to define the point at which a fetus becomes human at times intermediate of conception and birth, but these definitions are presently irrelevant to our discussion.

The Post editorial—which supports the “right” for Tebow to air his ad—is a typical example of muddled thinking that follows this debate. They say, “abortion is as tough and courageous a decision as is the decision to continue a pregnancy.” This is false. If a fetus is not human, it is no act of courage to undergo a medical procedure from which there is little risk of harm. But if a fetus is human, then the act of abortion is not courageous but villainous.

The writers (Frances Kissling and Kate Michelman) then descend into, what must be, an unspoken desire on their part. They say, “Pam Tebow was indeed courageous and had the legal right to choose…” Courageous she may have been, but the implication is that she might not have had the “legal right to choose.”

Since she was determined to have her child, and if she did not have the right to choose, then the choice whether to abort or not would have been made by others. Evidently, the Post is imagining that doctors should have that right, or that they would be in the best condition to judge, what defines a human.

In the same vein, while showing that support for abortion has decreased, the Post, repeats a common non-sequitur, “We read about successful fetal surgery; we don’t read about women dying in pools of blood on their bathroom floors after botched abortions, as we did when the procedure was illegal.”

If a fetus is human, then the harm caused a woman from a self-induced or botched abortion is not mitigating. She has still committed, or has been complicit to, a murder and does not have our sympathy. And if a fetus is not human, then the question shifts to one of stupidity (on the woman’s part) or possibly medical malpractice (on the part of whomever botched the abortion).

Doctors are in no better moral position than any of us to say what is or is not a human. But medical technology has evidently been partly responsible for the decrease in abortion support. Many, after seeing a colorful, three-dimensional picture of a young fetus, have concluded that the fetus is human. It is rational to suppose that these sorts of images and anecdotes will become more vivid and that support for abortion will continue to wane as more people conclude that fetuses are human.

The Post‘s writers tacitly admit this, and suggest some pro-abortion group produce a competing ad.

We’d go with a 30-second spot, too. The camera focuses on one woman after another, posed in the situations of daily life: rushing out the door in the morning for work, flipping through a magazine, washing dishes, teaching a class of sixth-graders, wheeling a baby stroller. Each woman looks calmly into the camera and describes her different and successful choice: having a baby and giving it up for adoption, having an abortion, having a baby and raising it lovingly. Each one being clear that making choices isn’t easy, but that life without tough choices doesn’t exist.

This brings us to probability and counterfactuals, which are unfortunately confusing subjects. Suppose a woman has an abortion. We cannot know, but can only guess, what her life would have been like had she not had the abortion. As the time from the abortion increases, the guesses become vaguer and more improbable. At best, then, any personal story it is weak evidence for the benefits of abortion.

However, we can generously suppose that the consequences of abortion are positive and as rich as you like. If a fetus (at whatever point in its development) is not human, then showing a post-abortion woman living a glamorous life would only serve to increase the number of abortions (try arguing the opposite), something few claim they support.

But if a fetus (from conception) is human, then showing a woman benefiting from her crime, and encouraging others to do the same, is evil.

January 18, 2010 | 14 Comments

Professions To Become Less Elitist In England

More from our Equality series: Just a sketch today; a longer article on this topic is (I hope to God) forthcoming.

England is about to—it hurts to type this—Unleash Aspiration!

According to the Daily Mail:

Labour will declare war on the benefits of a middle class childhood today as Gordon Brown spells out the latest steps in the government’s equality crusade.

Top lawyers, accountants, bankers and doctors will be ordered to draw up plans to make sure that their professions become less elitist – so they employ fewer middle class children.

Professionals will be told that poor children must be helped into the top jobs, at the expense of those who have benefited from their personal connections or education.

Universities will also be told to give the benefit of the doubt to poorer pupils when they are offering places and gloss over poorer marks if the applicant attended less illustrious schools.

The official report shows that the overwhelming majority of senior judges, doctors, CEOs, and such forth folk have gone to good schools. But only about 1 in 15 of “ordinary” Englanders have.

This represents an intolerable inequality. A wrong which must be lefted!

Giving an edge to poor people, who suffer undeservedly says the Brit Prime Minister, will “unleash a wave of social mobility.” He didn’t say if this “mobility” was the lemmings over the cliff kind.

You will be an ass if you claim that I and other detractors of the “unleashing” plan desire that the poor remain poor or that membership in a lower class should bar entry into a higher class.

You will also err if you forget that causality is a double-headed arrow.

It is true—nobody disputes this—that some poor people if, given a (monetary) chance, will prosper and become non-poor. They too can join the guilty class who look upon the poor as if they needed benevolent guidance. It is not even close to being proven that more poor people will become non-poor because of government, and not personal, intervention.

It is also true that some people are poor because, for whatever reason, they are incapable. Further, this incapability is be a permanent feature of some humans. Not some class of humans: I mean individuals.

James Fitzjames Stephen:

To establish by law rights and duties which assume that people are equal when they are not is like trying to make clumsy feet look handsome by the help of tight boots.

A mistaken assumption in the “Unleasing” report is that membership in the lower or upper classes is permanent. To emphasize: this is false. Membership is generational to some extent, but it is anything but fixed. Your author started with the clothes on his back, an (earned) monthly paycheck in the mid three digits, a wife (who didn’t work or drive), a pair of tongs and a used iron (given as wedding gifts). He is now, via hard work and because he was lucky with his genes, able to drink the finest beers.

The Left Honourable Alan Milburn, MP, ex-owner of a “radical” bookstore, authored “Unleashing.” He wants to “ensure everyone has the chance to fulfil their potential”. But this is not a desirable goal, in general. The world would have been better off had Lenin, Stalin, Pol Pot, Mao, etc., had not fulfilled their potentials. Best would have been if they died poor, uneducated, miserable, and obscure and unheeded.

Milburn also wants everybody to have a job which “rewarding and fulfilling.” By which he means a “professional” job—the kind which are populated by those who are waited on, in the finest restaurants, by the “unprofessional.”

What always comes as a shock to Milburnites is that not everybody has the same idea of what a “rewarding and fulfilling” life is. Some people find bliss in hanging drywall, or in driving a truck, or being the assistant manager of a local grocery. Still more find happiness in their family.

And not every judge, CEO, doctor, and other “professional” is feeling fine. Many will die unfulfilled, unrewarded, and unloved.

It isn’t at all clear who is doing better off, especially in countries like England (and the USA) where being “poor” means having only two large screen TVs and eating too much (all obesity “education” programs I’ve seen are aimed at the poor and lower class; so are all the free food efforts).

Nobody is claiming money isn’t nice, but it sure as hell isn’t everything. The paradox is that the guilty rich are certain it is everything. They think everybody judges life by the same standards they do. The “availability bias”, I think it’s called.

What really knocks these characters is that many of us would be happiest if we were just left alone by every group that says it “cares” for us.

This wasn’t entirely coherent. Did I accidentally buy decaffeinated? So, more on this coming…