There is a discussion of infinity and an argument to return to the discrete and finite in my upcoming book Uncertainty: The Soul of Modeling, Probability & Statistics. Infinity is a wonderful thing, but we can’t know it well enough to assign probabilities to infinite states. Also, all the paradoxes of probability occur because of misunderstanding infinity.
The latter is not surprising because infinity is a big place. So big that you can’t know or even conceive of its full nature.
(Incidentally, I returned the page proofs yesterday, and only learned of Tegmark’s remarks this morning, so that in the book’s second edition, I’ll have to expand my argument. Anyway, the book is good: after setting it aside for a couple of months I was shocked that I had written it: parts of it, and there are weaknesses, positively look like an adult wrote them. I JUST SAW THIS: looks like 22 July is release date.)
Now probability is purely epistemology, a proposition which I prove (as in prove) in the book (and which I don’t have space here to defend). Since epistemology is a matter of our thoughts, and our capacity for thought is not infinite, and though we can speak of the infinite and knows of its existence, or existences, since there are different kinds of infinity, we cannot assign probabilities to infinite possibilities.
Yes, we can write equations that sort of assign probabilities to the infinite. A normal distribution tries, for instance. But a normal distribution has nothing to do with reality (I prove this, too). What we can know is discrete and finite. All (as in all) our measurements are discrete and finite. Thus all our probabilities should be discrete and finite, and the continuum only brought in as an approximation.
In an extremely interesting book, This Idea Must Die, in which many eminent thinkers describe scientific ideas they consider wrong-headed, the physicist Max Tegmark of the Massachusetts Institute of Technology argues that it is time to banish infinity from physics. While “most physicists and mathematicians have become so enamored with infinity that they rarely question it,” Tegmark writes, infinity is just “an extremely convenient approximation for which we haven’t discovered convenient alternatives.” Tegmark believes that we need to discover the infinity-free equations describing the true laws of physics.
Amen, amen, and amen.
Such infinity-free equations exist, and I show some. It turns out in probability that you do not need parameters if you stick with reality-based measurements, which is to say with the discrete and finite; indeed, parameters are a consequence of intruding infinities. With no parameters, there is no need for estimation or hypothesis testing (Bayes or frequentist). These ideas are out the window! Instead, we are left with pure probability. Instead of speaking of the infinite, we speak (in science) of only that what can be observed (in principle, of course), and what can be observed can only be discrete and finite. (I do not say that kinds of infinity cannot exist. God, for instance, is a kind of infinity and exists. I do say we cannot know the mind of God.)
Eliminating infinity eliminates paradox, and it whacks crudities such as “flat” priors over the continuum, which is an infinity much, much, much larger than the ordinary, banal counting infinity, 1, 2, 3, … It’s so large that all we really can do is prove it exists. We can’t even get an understanding of all the numbers stuffed into [0, 1], let alone the entire “real” (ha!) line. So why think we can think of probabilities for each possibility in the continuum?
Answer: we cannot, which is why these things are called “improper” priors, which means everybody knows they’re not probabilities at all. They’re only used because they make the math work out. (And their proofs of utility involve infinite sequences, which I hope you can see are circular arguments.)
Tegmark’s original article is here.
In the past, many venerable mathematicians expressed skepticism towards infinity and the continuum. The legendary Carl Friedrich Gauss denied that anything infinite really existed, saying “Infinity is merely a way of speaking” and “I protest against the use of infinite magnitude as something completed, which is never permissible in mathematics.” In the past century, however, infinity has become mathematically mainstream, and most physicists and mathematicians have become so enamored with infinity that they rarely question it. Why?
The answer (the same in probability) is “infinity is an extremely convenient approximation, for which we haven’t discovered convenient alternatives.” These alternatives, as I said, exist.
Here is where probability comes in, though Tegmark doesn’t mention it directly:
Let’s face it: despite their seductive allure, we have no direct observational evidence for either the infinitely big or the infinitely small…If space is a true continuum, then to describe even something as simple as the distance between two points requires an infinite amount of information, specified by a number with infinitely many decimal places.