Jaynes’s book (first part): https://bayes.wustl.edu/etj/prob/book.pdf

Permanent class page: https://www.wmbriggs.com/class/

Uncertainty & Probability Theory: The Logic of Science

Link to all Classes. Jaynes’s book (first part):

Video

Links:

YouTube

Twitter

Rumble

Bitchute (often a day or so behind, for whatever reason)

Link to all Classes.

HOMEWORK:

Lecture

This is an excerpt from Chapter 6 of Uncertainty. For readability, the citations have been removed.

Randomness is not a thing; neither is chance. Standard statistical interpretation assumes randomness is a real physical property. Both randomness and chance are measures of uncertainty and express ignorance of causes and essences. Because randomness and chance are not ontologically real, they cannot cause anything to happen. Immaterial measures of information are never and can never be physically operative. It is always a mistake, and the beginning of vast confusion, to say things like “due to chance”, “caused by random (chance, spontaneous) mutations”, “these results are significant and not due to chance”, “no different than chance”, “these results are explainable by chance”, “random effects”, “random variable”, “that isn’t random”, “only random samples count”, and the like.

A coincidence is a concurrence of observations where one thing is said to be the cause, directly or indirectly, of another thing, but where the cause of the concurrence (and not the events) is unknown or immeasurable or suspected to be directed by certain higher powers. The invocation of randomness or chance as this unknown cause is always wrong (but that the higher powers exist might not be). There is an enormous amount of magical thinking which plagues probability and statistics on these questions, including in physics with quantum mechanics and in information theory.

All this holds in quantum mechanics, where the evidence for physical chance appears strongest. What also follows, although it is not at first apparent, is that simulations are not needed. This statement will appear striking and even obviously false, until it is understood that the so-called “randomness” driving simulations is anything but “random”. Lastly, how this ties in with information theory and the notion of randomness in that field is given.

Randomness

The English random has its roots, so says the Oxford English Dictionary, from the French, with implications of impetuousness, haste, and violence. It once expressed the range of a piece of ordnance. It wasn’t that this ranging distance was variable or chaotic per se: random was the maximum. One form of random in 1624 meant a haphazard route or path. Of course, (mis)hap and hazard themselves are tied to randomness, pace a modern definition: “Having no definite aim or purpose; not sent or guided in a particular direction; made, done, occurring, etc., without method or conscious choice; haphazard.” A definition of random noise is “unwanted electrical signals caused by randomly occurring transient disturbances…a signal component whose instantaneous amplitudes follow a statistically random or Gaussian distribution.” Finally, random number, ” a number selected from a given set of numbers in such a way that all the numbers in the set have the same chance of selection”. Although not from the OED, people will say of observing some quirky event, “That was random.”

Random, to us and to science, means unknown cause. This view is contrary to many authors who claim, without proof, randomness is a real property and found in, say, (realistically impossible) infinite sets. Random does not and cannot mean no cause. Any change (as we shall see much later) must be brought about by something actual, and something actual cannot be “randomness”. Variables, therefore, cannot be “random”; variables are propositions that take specific values, such as “The temperature will be $t$”, where $t$ is a placeholder for potential values, or is some stated value. Yet some thing or things will cause the eventual $t$, and this cause or these causes cannot be randomness. Determine is a dangerous word. It can mean caused or made known by. We may know (as we learn next Chapter) what determines the truth of a proposition in the sense of what makes the value known, but we may be ignorant of the cause. Randomness is the absence of knowledge of cause or of what determines whether a proposition is true. If you don’t know behind which of three doors is the prize, the proposition “It is behind door number 1” is not known to be true because you don’t know the cause of the prize being wherever it is and because there is no other information that would let you deduce where the prize is. The outcome is random, even though the prize was put there by some agency.

Coin flips, dice throws, sheeps-knuckle tosses, and the like are caused. But these kinds of events have their own interest. The results are sensitive to their initial and environmental conditions and are therefore chaotic, which as we earlier learned does not mean “not-caused”, but they are sensitive to initial (or just-plain) conditions. For some events, it is so difficult to physically manipulate conditions that the event must be ever practically (but not necessarily theoretically) unpredictable. But because these events are as sensitive as they are, tiny, even possibly quantum mechanically sized, deviations in conditions can cause the event to go a certain way or another easily. This is taken advantage of in two ways. Here is on illustration.

So they proposed two, Joseph called Barsabbas, who was also known as Justus, and Matthias. Then they prayed, “You, Lord, who know the hearts of all, show which one of these two you have chosen to take the place in this apostolic ministry from which Judas turned away to go to his own place.” Then they gave lots to them, and the lot fell upon Matthias, and he was counted with the eleven apostles. Acts 1, 23–26.

It is here that agency might enter the story, as it often does when speaking of randomness. The apostles reasoned in one of two ways. The first is that they trusted that God would “tweak” the conditions of the tumbling lots so that they would land in the optimal way, in the sense of selecting the optimal apostle. I have seen Buddhists at temples in Taiwan, for instance, do a similar thing with crescent-shaped blocks of wood called bwa bwei. This are a pair of asymmetric, hand-sized blocks which are thrown onto stone or dirt floors; they bounce around a bit, and come to rest with one or both bwa bwei having the round or flat size uppermost. Questions are asked of the “device” and answered depending whether the sides match or mismatch. The appeal is to a higher power, but one which is somehow unwilling to perform a macro feat, as God could easily do, for instance in the case of the lots, by having had the apostles places the lots on the ground and then God could turn them so that they pointed to Matthias. The same is true of bwa bwei. The faithful could merely place the bwa bwei on the ground and ask the local deity to move them to the position which matches the correct answer. But in both cases this seems like asking for a great amount of work from God or the deity. Instead, when asking for interventions, we ask for the smallest possible assistance, the tiniest adjustment to the conditions, that which requires an almost infinitesimal physical force, so that the device is caused to take its eventual state in such a way that the higher power is not unduly taxed. This act on our part recognizes the sensitive and even precarious nature of the device; indeed, it makes active use of it.

But there is a second sense in which we can interpret the choosing of the substitute apostle which is vastly more plausible. This sense won’t work for the bwa bwei petitioner, who simply is asking for a physical intervention. Because the apostles understood that the tossing of the lots is unpredictable and nearly impossible to gaff, i.e. to finagle or scam, there would be no sense of human agency in the choice of the next apostle. If the eleven would have had a vote, Joseph might have won or Matthias would have. There would have been some apostles in favor of Joseph, and some in favor of Matthias. A discussion would begin and politics would enter. And people have long memories. Feelings could be hurt. Since both men were eligible, why not let some unpredictable device make the selection so that everybody is excused from making a choice? This is, after all, why we let referees decide who gets the ball first by coin flips (I have more to say about this below). Randomness, i.e. unpredictableness, solves some political conundrums.

What makes the first example different from the statistician waiting to see what value a “random variable” takes? Only this: the higher power is not usually thought to be a wilful agency, and is instead some vague, almost mystical hidden power. One example is “noise”, the “error” or “residual” or “$\epsilon$” term in a regression or in a simulation (about which, more below). The value of some $\epsilon$ is thought to come about “randomly”, and if this “randomly” is thought about at all, and often it is not, it is often thought “random” mystical forces are performing the cause. Hence statisticians will talk about “sampling” so that these mysterious forces “cancel” each other out upon repeated “trials”. It is often said that “probability distributions” underlie a set of observations, which again imply probability is cause. The main exception to this magical thinking is electronic engineering and the like, where engineers are forced to think about causes of everything that happens, though even in these fields, thinking that randomness is a cause is not unknown. Opposite this are those uses of statistics applied to human behavior, where what causes the “$\epsilon$s” is always said to be randomness. Whatever causes any $\epsilon$ to take the value it does, it is not randomness.

I either have in my pocket as I write this my pipe or I don’t. That is, I own a pipe and sometimes smoke it while writing, except when I need both hands I put it in my pocket, or I don’t carry it at all hence it can’t be in my pocket. The proposition of interest is P = “Briggs had a pipe in his pocket when he wrote this proposition.” P is random to you, because the only evidence you have is that which I provided, which is not sufficient for you to form a unique probability. Of course, you can always add evidence which is not provided, but by that maneuver you make probability subject to whim, which is to say subjective. P is not random to me, because I possess enough extra information that the P is an extreme probability, either 0 or 1.

That is, you must judge $\Pr(\mbox{P} | \mbox{Briggs owns a pipe and…})$ whereas I must judge $\Pr(\mbox{P} | \mbox{I have my pipe})$. The former probability is not a fixed number (it may be the unit interval sans endpoints if you consider the tacit premise that the event is contingent; that “I do or don’t have a pipe” is a tautology and provides zero information), but the latter probability is 1 (and would have been 0 if I changed by evidence to “I don’t have my pipe”).

Randomness therefore exists when the probability of a proposition given stated evidence or model is not 0 or 1. That is, randomness applies to the premises (or model) we have and not the outcome. All uncertain events are thus random. An event is random only if it is unknown (in its totality). A state is random if it is unknown. Randomness is thus a synonym for unknown. That, and nothing more.

Statisticians speak, somewhat incorrectly as we have just seen, of random variables. These are mathematical creatures, propositions which contain or represent an unknown quantity. For example, S = “Sally’s grade point average is $x$” where $x$ is unknown, i.e. “random.” S is neither true nor false—it does not have an extreme probability—and can be neither true nor false because there is no premise with which to judge it, except perhaps that “The grade point average will be some number in this set”. But even given that evidence, the proposition has no probability because $x$ is not a number but a placeholder. It’s like saying “The color is _______”. It is an incomplete statement. This seemingly trivial point is crucial to retain. There is no observation of $x$. Once we do observe an $x$, the proposition becomes true with respect to that observation. Thus random variable means a proposition with an unknown quantity (the quantity may of course be multidimensional).

Of course, there is a purely mathematical way to speak of “random” variables, i.e as some kind of measurable function from a probability space into a measure or state space, and so forth. However useful this technique is for computation, and it is, when applying probability to real propositions of interest in arguments, we must not forget that the mathematics are not real. I speak more on the Deadly Sin of Reification which arises from attempts to give equations life in the discussions of modeling.

Chance is identical to randomness in most senses, though it often comes with connotations of unpredictability. Take a “game of chance” such as craps, which is based around a two-dice total, or score. The bounds of the total are deduced from the rules of the game. These bounds are, as is obvious, predictable, so chance does not mean complete inability to predict. There are any number of physical mechanisms that cause each dice total, causes of which we are mostly or completely ignorant. We know the causes must be there, we just don’t know what they are for individual plays. We do know there are many causes: imagine the bouncing rolling dice flopping around, buffeted by this and that. If we knew some of these causes for individual rolls—perhaps we could measure them in some way as the dice fly; say, by noting the walls of the table are cushier and more absorbent than usual—then we could incorporate that causal information and use this to update the probabilities of the totals. A 7, which is a winning score on the come out, might be more or less probable depending on how the information “plays”. The probability changes because the information changes. Incidentally, unless your knowledge of cause is complete, you might not necessarily beat the casino for any single game, but if you have good causal knowledge, you will beat them over multiple games. It is for this reason that casinos ban contrivances that could measure causes or proxies of causes. In any case, chance is unpredictability, which is a synonym of ignorance, which is what random means.

Subscribe or donate to support this site and its wholly independent host using credit card click here. Or use the paid subscription at Substack. Cash App: \$WilliamMBriggs. For Zelle, use my email: matt@wmbriggs.com, and please include yours so I know who to thank.