Class 36: Paradox Lost

From time to time people come across what seems like paradoxes which invalidate probability (as logic). These are excellent devices to sharpen thought, but they all suffer from one of two mistakes: forgotten or missed implicit premises, messing with Infinity.

Uncertainty & Probability Theory: The Logic of Science

Video

Links: YouTube * Twitter – X * Rumble * Bitchute * Class Page * Jaynes Book * Uncertainty

HOMEWORK: You must look up and discover a Research Shows paper and see if it conforms to the conditions given below. Have you done it yet?

Lecture

This is an excerpt from Chapter 4 of Uncertainty, sans references.

Another class of probability problems also exist, such as Buffon’s needle and the so-called marginalization paradox (see Jaynes, Chapter 10 for a gruesome vivisection of this concept), all of which involve making finite choices from infinite sets. These problems also show that probability is conditional. But because messing with infinity is like walking through a raging forest fire and hoping for the best, many folks get burned. Or perhaps it is better to say infinity is like a foreign country; rather, many foreign countries, since there are many kinds of infinities. Mistakes are made when the traveler thinks he has the whole place figured out after only a brief visit. Whenever it is claimed that some “paradox” involving infinity has invalidated this or that philosophy of probability, it is safest to put the claim down to enthusiasm and to continue believing in probability. I say more about infinity when discussing measurement and models. But first a simple example I learned from an unpublished work on probability by Purdue’s Paul Draper. There is nothing unique about the example, which is familiar especially in criticisms of Bayesian theory on assigning “prior” probabilities, though Draper phrases it nicely.

Imagine a factory that spits out tiles anywhere from 1 to 3 inches in width. The so-called principle of indifference would lead to a uniform probability assignment to the widths between 1 in to 3 in. Since the tiles are square, the surface area is anywhere from 1 in${}^2$ to 9 in${}^2$. Considering area under the principle of indifference, says Draper, leads to the assignment of an uniform probability to the areas from 1 in${}^2$ to 9 in${}^2$. But then there is a contradiction. Given the indifference criterion and the evidence we’re provided, there is probability 1/2 for the proposition “The surface area of this tile is between 1 in${}^2$ to 4.5 in${}^2$”, which corresponds to widths 1 in and 2.12132 in. But there is also probability 1/2 for the proposition “The width of this tile is between 1 in and 2 in.” What has gone wrong? Many blame probability. Instead, our ideas of measurement have gone awry.

Any real tile can only be manufactured in discrete increments. Call those increments $\delta$. These do not have to be equal, and one length may even depend on another; the only restriction is that $\delta >0$. For ease, I’ll assume this is fixed. That means the widths can be 1 in, $1+\delta$ in, $1+2\delta$ in, and so on up to 3 in (with the additional assumption that for some $n$, $1+n\delta =3$, but again, this is only for ease). Since widths are fixed, so are areas, which can now only be 1 in${}^2$, $(1+\delta {)}^2$ in${}^2$, $(1+2\delta {)}^2$ in${}^2$, and so on up to $3^2$ in${}^2$. No surface area between 1 in${}^2$ and $(1+\delta {)}^2$ in${}^2$ is possible. Using the statistical syllogism, the probability “The width of this tile is $1+m\delta$ in, where $0\ge m\le n$” is $1/(n+1)$. Obviously, since the surface areas are one-to-one with the widths, the uniform probability applies to them as well. No contradiction. And no restriction, either, because we can let $\delta$ be as small as we like, smaller even than a quark!, as long as it is non-zero. It is only at the limit where these odd contractions pop up, but that is because, as already said, infinity is a strange place, and because the limit here is not only $\delta$ going to 0, but the $n$ going to infinity in some sort of tandem. How you get to infinity matters.

Another example from Draper. There are three balls in a bag, each of which must be either white or black. Given this evidence, what is the probability “All three balls are black”? Drapers says 1/8 because “Consider the following eight statements: all three balls are black, the first two are black and the third white, the first two are white and the third black, etc. One can easily imagine having no more reason to believe any one of those eight statements than any other.” But But then, says Draper, “there are also four possible ratios of black balls to total balls in the urn (i.e., 1, 2/3, 1/3, and 0)…[and] the principle of indifference implies that the probability of the urn containing three black balls is 1/4.” Contradiction! Yet Draper forgets some of his evidence. One of the ratios is indeed 3 out of 3, and another is 2 out of 3. But there are three ways to get 2/3: B1B2W3, B1W2B3, W1B2B3. Likewise, there are three ways to get 1/3, and just one way to get 0/3, That makes 8 total ratios, only one of which contains all black balls; thus, conditional on the full evidence (and notice even Draper started by labeling the balls but then forgot), we’re back to 1/8. Many similar paradoxes resolve in precisely the same way.

Senn (2011) has a similar paradox, one common in Bayesian statistics and which causes consternation, using an argument from continuity. He first defines an “event” which can take one of two values, e.g. success and failure. He then defines the “probability of success” of this event as $\theta$, and says it in turn is equally likely to take any value in $(0,1)$ (sans extremes). Bayesians would call this putting a “flat prior on $\theta$.” He says “Suppose we consider now the probability that two independent trials will produce two successes. Given the value of $\theta$ this probability is ${\theta }^2$. Averaged over all possible values of $\theta$” this is 1/3 (the integral of ${\theta }^{2}d\theta$). The difficulty enters in the next step: “A simple argument of symmetry shows that the probability of two failures must likewise be 1/3 from which it follows that the probability of one success and one failure in any order must be 1/3 also and so that the probability of success followed by failure is 1/6 and of failure followed by success is also 1/6.”

Ignoring the language about “events” and “independence”, this is a seeming paradox. Why? Because of the odd statement “Suppose that we believe every possible value of $\theta$ is equally likely…” What can that mean? Nothing. There are more than a simple infinity of numbers of possible values of $\theta$ when that parameter is continuous, and to “believe” each is equally likely (based on what premises?) is to claim rather shocking, almost omnipotent knowledge. On this view, what is the probability (still with undefined or vague premises) that $\theta$ takes any value? Well, zero. How about after we take some data, consisting of some observed string of successes and failures, what then is the probability $\theta$ takes any value. Bayes’s rule tells us the answer is the same: zero. This $\theta$ is a truly strange creation, a continuous string of numbers: no, they are not numbers at all, $\theta$ is the continuum itself. And it came out of the blue, with premises that just asserted it. To put a probability to every value of the continum is to claim knowledge of those values, which is never something we can directly have. The math works out because we have indirect knowledge in the sense of we know the continuum exists and that it has certain properties, but we can never have knowledge of more than a (limited) finite set of those values. Anyway, as is now easy to see after Draper’s examples, the paradox disappears if $\theta$ is a as-large-as-you-want finite set of values. I’ll leave this as homework for the reader to prove. There are, it is suspected, many things to be learned from investigating the continuum, as infinities always boggle the mind. This is why it is right to suspect our limited understanding of the infinite and not our understanding itself when confronted with a paradox.

Subscribe or donate to support this site and its wholly independent host using credit card click here. Or use the paid subscription at Substack. Cash App: $WilliamMBriggs. For Zelle, use my email: matt@wmbriggs.com, and please include yours so I know who to thank. BUY ME A COFFEE.