Preface: The Philosophy of Probability and Statistics, An Introduction

This is a full truck, which is not necessarily a truckful.
%This is the Preface of The Philosophy of Probability and Statistics, An Introduction coming before Preface and before the chapters on Truth and Logic. Note the New & Improved title! Since this is the Internet, I must explain that the Preface of a book is not obligated to provide an in-place proof of every claim. That is what the book itself is for. I left the Latex coding. It isn’t hard to read. Also since this is the Internet, it will do me no good whatsoever to claim that this is only a rough draft to give readers the idea of the book and that it is not the completed fully formed polished final product. So I won’t claim it.


If the only information you have is the eyewitness report “I saw a truckful of them walking south,” what is the probability that “Seven individuals were walking south”? Depends on what the fellow meant by “truckful”, right?

Right. But it also depends on what the fellow meant by “I”, “saw”, and all the rest, and what you mean by “seven”, “individuals”, etc., and on the rules of English grammar. It might also depend on whether you were expecting seven. Maybe you’re a cop chasing escapees. We need the rules of grammar because, for instance, the questions would not make sense to a monolingual Mandarin speaker. But your understanding is also conditional on taking the eyewitness at his word and on assuming that all his faculties are in order (eyesight and the like). Same thing for you: are you reasoning correctly about “seven”: that’s more than six, isn’t it?

Now suppose the only information you have is the premise, “All men are mortal and You, the reader, are a man.” What is the probability that “You, the reader” are mortal? We still need the tacit information about grammar, the plain meaning of the words, and the belief our faculties are functioning. Given these, and as everybody schoolboy used to know, it is certain you will turn back to dust.

Another way to say this is that it is {\it true} you will cease on an eventual morrow. Truth therefore has some meaning. Here, and everywhere, it is the correspondence between the thing claimed and reality. The claim is you die; the reality is you will. What about the truth of the proposition “Seven individuals were walking south”? It might be true. You don’t have enough information to tell. Somebody does know, though: the group heading south, presuming they can count, would know for sure whether they were seven. But you don’t know. You are uncertain.

There is truth or falsity lurking, but you can’t come to it, not with the information you were given. There is no way to come to a numerical representation of your uncertainty, either. Not without adding information you were not given. Of course, you might {\it decide} the proposition is true. Or you might {\it act} on the {\it belief} it was six and not seven. Decisions and actions, though they depend on it, are not probability. Consider that you can, even in the face of “All men are mortal, etc.,” still act as if you won’t die (a physical death). You might even believe it. But that won’t change the certainty of “You, the reader, are mortal.”

Probability, like logic, says something about the {\it connections} between propositions, and just like logic those connections are {\it fixed} by the propositions given and not subject to dispute, not once the language, terms, and rules are set. Probability is not subjective and subject to whim any more than logic is. Again like logic, probability is not ontological; it is not a measure of physical stuff or of reality; it is not, for example, frequencies, but frequencies are propositions and thus can inform probabilities. Probability is epistemological, it measures uncertainty.

Since this is so, before we can understand probability fully, we first need to understand truth. Truth is that which logic and probability aim at. Truth implies reality and both together imply knowledge. There are conditional truths—propositions which are so given contingent premises—and necessary truths—propositions which are so no matter what. Therefore, there will also be conditional and necessary probabilities. Belief is a decision, an act, and does not always imply truth, especially when we are uncertain. We can, however, have knowledge, which is justified true belief. That too comes in conditional and necessary forms.

The natural starting point is logic, where our certainty is greatest. But even logic must take some things for granted. Logic ultimately cannot be empirical: we must come equipped with some knowledge or we would never get any argument off the ground. Since probability is the extension or completion of logic, some of our “pre-packaged” knowledge is also of probability; actually, of its “rules.”

Ordinary “street logic”, i.e. Socratic or syllogistic logic, is highly informal but absolutely crucial to grasp because every human decision eventually involves ordinary language. Mathematical logicians and probabilists escape the problem of language ambiguity by assiduously defining their terms, so that no, or only very little, possibility of error exists. We don’t have that advantage when it comes to deciding how best to implement, or not, the results of a medical experiment.

In our first deduction, the premise “All men are mortal” is an induction. Some call probability “inductive logic”, but it isn’t, or isn’t always. Others say that induction is not “justified” or that it is “irrational”, but these judgments are false. Logic and probability rely on induction in a way which is under-appreciated or even unknown. Induction and logic, and therefore probability too, guide us in our perception of causality. Too often people mistake probability statements as evidence of causality. And this includes those people who ever have “Correlation does not imply causation” on their lips. The limitations of knowledge of causation are also important to know before speaking of probability models.

Probability, at this point, falls naturally from truth, logic, and induction. But there are a number of prominent misconceptions which have to be tackled, such as the frequency, complexity, and subjective interpretations. These are more or less harmful in our recognition of uncertainty, but of potentially greater harm when married to the practice of statistics, which is defined as the practical implementation of probability. Probability is usually not a number; it cannot always be quantified; though sometimes it can, and the beginnings of Cox and Kolmogorov become important.

Randomness and chance are intertwined with probability, but are widely misunderstood. These are nothing but different names for uncertainty. They are not physical entities. They cannot be causative. Therefore, the numerical methods (such as “MCMC”) which “use” randomness, even when they give reasonable answers, are strictly mistakes and are therefore not built a sound foundation. Randomness also has deep connections with predictability and complexity. We need the ideas of Kolmogorov or algorithmic randomness, which is when a string is shorter than any computer program which can produce that string. “True” randomness in this sense is related to compressibility, which itself is a kind of prediction. And then all these ideas have direct ties to entropy, which is a direct numerical measure of uncertainty, and is therefore intimately matched to probability, particularly in Bayesian statistics. But again the caution that most probability is not numerical must be kept in mind.

A weird tradition has developed wherein {\it ad hoc} probability “models” are proposed for propositions of interest, propositions which are then forgotten as focus is fixated on the parameters of these models. Hypotheses regarding these parameters, which nobody believes, or are provably false, and which are in any case conditional on models’ accuracy, are bruited and said to be “significant” or not, a technical word which has little relation to its English cousin. Creatures called p-values, despite innumerable strident warnings against their use, are worshiped to a degree which is hard to credit. Still, good work can be and is done even in the face of these strange practices. The question is how and under what circumstances?

The Cult of the Parameter is only one of the ways over-certainty is introduced. The others are reification, which is the mistaking of the models (which are almost universally mathematically correct) for reality, and unnecessary quantification. People feel only numbers are scientific, thus everything is given a quantification merely for the sake of subjecting the things to routines which require numbers as input. The mistake is two-fold: probability does not only speak of physical things, but of metaphysical reality, just like logic. And sometimes we’re stuck with non-quantifiable uncertainty. Putting numbers where they don’t belong is bound to cause grief, and does. Besides, the vast majority of uses to which probability are put are not in formal settings, but are like the trukful-walking-south situations.

Since vast armies of numbers are marshalled in support of research, and statistical methods are taken as proof absolute for nearly every kind of claim, we need to develop methods with procedures suited to the goal of understanding uncertainty about physical things. We also need these methods to make useful and accurate decisions. Thus {\it measurement}, the {\it sine qua non} of science, should drive these procedures. No human measurement (of reality) can be infinitely graduated; therefore, it is inappropriate to begin an analysis with an assumption they can be. Not only does this produce over-certainty, but it leads to unnecessary disputes about “priors” and parameters. Parameters naturally arise from considering natural, which is to say finite, measurement, and then imagining the number of measurements approach the limit. This is different than the current view of statistics, which tacitly assumes measurement is of the continuum.

Examples in simple risk and time series are developed. But recipes for handling every sort of formal quantified problem are not given. The big reason for this limitation is that these formal methods have not yet been worked out. Mathematically minded readers are invited to try their hands in constructing procedures based on the new guidelines.

Lastly, because most books on probability have a strong and natural bias towards science and mathematics, the reader might expect the same of this book. This is not the case. Probability is much richer than the narrow confines of science. Our goal is to understand the whole of it.


  1. Although you don’t solicit it, some advice: This reads more like an introductory chapter than a preface, which generally is an explanation of how the book came to be written or the ideas that inspired it. Sometimes there are acknowledgements of indebtedness by the author to those who helped the effort. These kinds of prefaces are friendlier and reveal a less formal facet of the author. Most audiences of anything but technical manuals prefer lectures to begin as a conversation (hence the advice to start with a joke to lighten the mood). Is it possible to recategorize this as an introductory road map to the later chapters and write a more personal preface?

  2. I agree with Gary.

    A road map should be included. You have an outline, right?
    The following thoughts (not necessarily inclusive) could go to impetus:

    A weird tradition has developed wherein {\it ad hoc} probability “models”…
    Randomness and chance are intertwined with probability,
    The Cult of the Parameter is only one of the ways over-certainty is introduced.

    You do have some road map and why here but mayve you should separate them with the whys coming first followed by this is where I’m going

  3. I have read the prefaces of many textbooks. It generally consists of the goal of the books, the structure/ organization of the book, background knowledge required and whom the book is for. Of course, it usually ends with acknowledgements.

    If you have original ideas, be clear about it in the preface.

    I understand it is not easy to be original in philosophy. I personally will also be interested in reading thorough, clear and systematic summaries of scholarly writings published. A good literature review is time consuming, but it might be a worthwhile endeavor. It’d also prevent your thoughts from flying and darting in a Brownian motion.

    If you want to show me that you understand different philosophies, you will need to explain to me each one of them accurately… knowing their merits and shortcomings.

  4. Examples in simple risk and time series are developed. But recipes for handling every sort of formal quantified problem are not given. The big reason for this limitation is that these formal methods have not yet been worked out. Mathematically minded readers are invited to try their hands in constructing procedures based on the new guidelines.

    If formal methods have not yet been worked out, how do you develop examples and guidelines first? How would you know they are correct guidelines? You need professional help, and one way to get those help is to write them up and submit a paper for peer-review. Peer review works, you know it!

  5. Some textbooks have stogy prefaces. Some set the stage for what is to come without formalities. I find those the best. This looks like one of the better ones.

    Since when did peer review constitute professional help?

  6. Mr. Alvarez, since at least 1991, the year I received the referee reports on my first paper.

    Mr. Briggs, making stuff up is for imps. ^_^

  7. What do you mean when you say MCMC is “a mistake?” Are you saying that MCMC cannot estimate a pdf for a completely correct model?

    Several large craters in Nevada would disagree.

  8. fizzymagic,

    I mean just what I say: that it is a mistake in thinking. But notice I also say that we can get good answers from these and other mistaken routines. How to fix it? Well, stick around.

Leave a Comment

Your email address will not be published. Required fields are marked *