To clarify: to prove false means to demonstrate a valid argument with a certain conclusion. If a theory/model says an event is merely unlikely—make it as unlikely as you like, as long as it remains possible—then if that event happens, the theory/model is not falsified. To say, “We may as well consider it falsified: it is practically falsified” is like saying, “She is practically a virgin.” False means false; it has no relationship with unlikely.
A theorist or statistician has in hand a priori evidence which says model M1, M2, …, Mk are possible. Some of these, conditional on the theorist’s evidence, may be more likely than others, but each might be true. If these models have probability components, as most models do, and these probability components say that any event is possible, no matter how unlikely, then none of these models may ever be falsified. Of course, those models in the set that say certain events are impossible, and these events subsequently are observed to occur, then this subset of models can be falsified; the remaining models then become more likely to be true.
In Bayesian statistics, there is a natural mechanism to adjudge the so-called posterior probability of model’s truth: it is called “posterior” because the model’s truth is conditional on observation statements, that is, on what happens empirically (this needn’t be empirical evidence, of course; any evidence will do; recall that probability is like logic in that they study the relationships between statements). Each models’ a priori probability is modified to its a posteriori probability via a (conceptually) simple algorithm.
These a posteriori probabilities may be ordered, from high to low and the model with the highest a posteriori probability picked as “the best.” The only reason one would want to do this is if a judgment must be made which theory/model will be subject to further actions. The most common example is a criminal trial. Here, the theories or models are suspects in some crime. At the end, only one theory/model/suspect will face punishment; that is, at most one will. It may be that no theory/model/suspect is sufficiently probable for a decision to be made. But if the suspect is found guilty, it is not that the convicted theory/model/suspect is certainly guilty, for the other theories/models/suspects might also have done the deed, yet the probability that they did so, given the evidence, is adjudged low. This implies what we all know: that the convicted might be innocent (the probability he is so is one minus the probability he is guilty).
It is often the case (not just in criminal trials) that one model (given the evidence and a priori information) is overwhelmingly likely, and that the others are extraordinarily improbable. In these cases, we make few errors by acting upon the belief that the most probable model is true. Our visual system works this way (or so it has been written). For example, your brain assures you that that object you’re reaching for is a cup of coffee, and not, say, cola. Sipping from it provides evidence that this model was true. But as we all know, our vision is sometimes fooled. I once picked up from the carpet a “cookie” which turned out to be a wiggling cockroach (big one, too).
Now, since we began with a specified set of suspects (like my cookie), one path to over-certainty is to not have included the truly guilty in the list. Given the specific evidence of omittance, the probability the other suspects are guilty is exactly zero (these theories/models are falsified). But, in the trial itself, that specific evidence is not included, so that we may, just as we did with the green men, calculate probabilities of guilt of the (not-guilty) suspects. Keep in mind that all probability and logic is conditional on specific, explicit premises. The probability or certainty of a conclusion changes when the premises do.
So what is the probability that we have not included the proper theory/model/suspect? That question cannot be answered: at least, it cannot be answered except in relation to some evidence or premises. This applies to all situations, not just criminal trials. What might this external evidence look like? We’ll find out in Part IV.