Occam’s razor: The simplest hypothesis is usually the best. Simplest is easy to understand, and non-controversial, or relatively so. Consternation creeps in with usually.
Before that, first understand what is an induction. Not in its mathematical “if n + 1 then n” implementation, but in its logical form. The reason you do not leap from a fifty-story building to arrive at the ground in the quickest manner, is the same reason you do not stick your bare hand into a roaring fire. Every time somebody has done this has either died from rapid deceleration or has been toasted like a marshmallow.
It is logically possible that when you leap you will not be turned to jelly, just as it is logically possible that when you reach into the fire your hand remains unscathed. It could be, for instance, that a great gust of wind blows northwards slowing your southwards fall just before you reach your final stop. Or a fluke of the fire causes the flame to part and the heat to be directed away from your digits.
We induce that leaps from high places or reaching in to roaring flames will cause death or pain because they have always been seen to. Any contingent event—like these physical events—that we see always happens, we say via induction that they always will.
If we only die most but not all times we leap from tall buildings, or are only burned most but not all times when we thrust our hands into a fire, then we are still leery of these activities, but not we did not learn our fear from induction. We instead used probability, or non-deductive logic, but not induction.
Inductions are just one-off deductions, so to speak. Like a deductive one, an inductive conclusion is always seen to follow from its premises, but unlike the deductive conclusion which must always follow, the inductive conclusion is not logically necessary.
And both of these differ from non-deductive conclusions, which only sometimes, or rarely, or usually—anything but never or always—follow from their premises. These are crucial distinctions, easily misunderstood and the cause of much confusion.
Now, we so often derive explanations for events that it is not just second, but first nature. Why did the car go when we pushed the gas pedal? Why does the basement flood when it rains? Why do pencils fall off tables? Why does your wife become argumentative with the waxing of the moon? For all these things and innumerable others we have developed explanations, theories, models (all synonymous).
We have discovered, through experience, that usually—but not always—models which are simpler are often more useful or are more often correct than are models which are complex. Occam’s razor is thus not an induction, but a belief, or rule of thumb, based on non-deductive reasoning. If it were an induction we would have discovered that each and every time the simpler model was the better.
What does simpler mean? One with fewer premises. One without as many knobs to twist, contingencies that must be met, chains of complexities that must be followed. Occam is supposed to have said, “entities must not be multiplied beyond necessity.” But this begs the question of what is necessary.
If a model is not deductive or inductive, it is non-deductive; therefore its conclusion is only probable (a number strictly between 0 and 1) given its premises. If a model is deductive, it is always possible to strip away all unnecessary premises, here defined as those that do not change the validity of the conclusion. Necessary premises are those absolutely required to make the conclusion true or false (a probability of exactly 1 or 0).
If a model is inductive, it is usually clear which premises that can be taken away so that the conclusion is still seen to be likely, but then we have to define likely. Inductive conclusions are contingent, so none have probabilities exactly equal to 1 or 0, but probabilities which are as close to 0 or 1 as you like. How far from 1 or 0 does the conclusion have to change upon removal of a premises that this removal convinces us this premises was “necessary”?
Actually, any substantial, measurable change in the probability is enough. If removal of the premises shows the conclusion is not just contingent in general, but specifically such that the probability changes measurably, then that premise was necessary for the model.
For non-deductive models, removing any premise just changes the probability of the conclusion. A removal can make the probability go up or down, but so what? The conclusion is still not certain and so still not guaranteed, so the model (with or without the premise) is still not wrong.
We can only gauge the goodness of the model by how close the probability of the conclusion was to the actual event (whether it happened or not), and only after the fact. This is tough luck for those who would criticize a non-deductive model (which is all known physical models, like GCMs, etc.). What the punter wants is to, before the fact, have a basis on which to criticize or judge a non-deductive model: and that is why he invokes Occam’s razor.
After the fact, models which have closer probabilities are said to be better. So Occam’s razor comes to this: experience has shown that non-deductive models with fewer premises often have probabilities closer to actual events. In other words, Occam’s razor is a meta-probability statement, no different than probability statements for ordinary models.