Statistics

Infinity & Probability: A Return To The Finite

14569394038_c4e678e113_h

There is a discussion of infinity and an argument to return to the discrete and finite in my upcoming book Uncertainty: The Soul of Modeling, Probability & Statistics. Infinity is a wonderful thing, but we can’t know it well enough to assign probabilities to infinite states. Also, all the paradoxes of probability occur because of misunderstanding infinity.

The latter is not surprising because infinity is a big place. So big that you can’t know or even conceive of its full nature.

(Incidentally, I returned the page proofs yesterday, and only learned of Tegmark’s remarks this morning, so that in the book’s second edition, I’ll have to expand my argument. Anyway, the book is good: after setting it aside for a couple of months I was shocked that I had written it: parts of it, and there are weaknesses, positively look like an adult wrote them. I JUST SAW THIS: looks like 22 July is release date.)

Now probability is purely epistemology, a proposition which I prove (as in prove) in the book (and which I don’t have space here to defend). Since epistemology is a matter of our thoughts, and our capacity for thought is not infinite, and though we can speak of the infinite and knows of its existence, or existences, since there are different kinds of infinity, we cannot assign probabilities to infinite possibilities.

Yes, we can write equations that sort of assign probabilities to the infinite. A normal distribution tries, for instance. But a normal distribution has nothing to do with reality (I prove this, too). What we can know is discrete and finite. All (as in all) our measurements are discrete and finite. Thus all our probabilities should be discrete and finite, and the continuum only brought in as an approximation.

Enter Tegmark:

In an extremely interesting book, This Idea Must Die, in which many eminent thinkers describe scientific ideas they consider wrong-headed, the physicist Max Tegmark of the Massachusetts Institute of Technology argues that it is time to banish infinity from physics. While “most physicists and mathematicians have become so enamored with infinity that they rarely question it,” Tegmark writes, infinity is just “an extremely convenient approximation for which we haven’t discovered convenient alternatives.” Tegmark believes that we need to discover the infinity-free equations describing the true laws of physics.

Amen, amen, and amen.

Such infinity-free equations exist, and I show some. It turns out in probability that you do not need parameters if you stick with reality-based measurements, which is to say with the discrete and finite; indeed, parameters are a consequence of intruding infinities. With no parameters, there is no need for estimation or hypothesis testing (Bayes or frequentist). These ideas are out the window! Instead, we are left with pure probability. Instead of speaking of the infinite, we speak (in science) of only that what can be observed (in principle, of course), and what can be observed can only be discrete and finite. (I do not say that kinds of infinity cannot exist. God, for instance, is a kind of infinity and exists. I do say we cannot know the mind of God.)

Eliminating infinity eliminates paradox, and it whacks crudities such as “flat” priors over the continuum, which is an infinity much, much, much larger than the ordinary, banal counting infinity, 1, 2, 3, … It’s so large that all we really can do is prove it exists. We can’t even get an understanding of all the numbers stuffed into [0, 1], let alone the entire “real” (ha!) line. So why think we can think of probabilities for each possibility in the continuum?

Answer: we cannot, which is why these things are called “improper” priors, which means everybody knows they’re not probabilities at all. They’re only used because they make the math work out. (And their proofs of utility involve infinite sequences, which I hope you can see are circular arguments.)

Tegmark’s original article is here.

In the past, many venerable mathematicians expressed skepticism towards infinity and the continuum. The legendary Carl Friedrich Gauss denied that anything infinite really existed, saying “Infinity is merely a way of speaking” and “I protest against the use of infinite magnitude as something completed, which is never permissible in mathematics.” In the past century, however, infinity has become mathematically mainstream, and most physicists and mathematicians have become so enamored with infinity that they rarely question it. Why?

The answer (the same in probability) is “infinity is an extremely convenient approximation, for which we haven’t discovered convenient alternatives.” These alternatives, as I said, exist.

Here is where probability comes in, though Tegmark doesn’t mention it directly:

Let’s face it: despite their seductive allure, we have no direct observational evidence for either the infinitely big or the infinitely small…If space is a true continuum, then to describe even something as simple as the distance between two points requires an infinite amount of information, specified by a number with infinitely many decimal places.

Yes!

Categories: Statistics

22 replies »

  1. So the answer to Zeno’s paradox is not infinite series, but the finite number of atoms from point A to B?

  2. I guess I am wrong to view the paradoxes as allegory towards adulthood. Math is a wonderful tool that helps me attack problems in the real world. I do not expect math to be able to perfectly predict. The paradoxes remind me to always take a step back from the equations and find the disconnect. In the case of Zeno, it is my nose hitting the wall….

  3. Physical Finitist.

    Eliminating infinity… whacks crudities such as “flat” priors over the continuum, which is an infinity much, much, much larger than the ordinary, banal counting infinity, 1, 2, 3, … We can’t even get an understanding of all the numbers stuffed into [0, 1], let alone the entire “real” (ha!) line. So why think we can think of probabilities for each possibility in the continuum?

    Answer: we cannot, which is why these things are called “improper” priors, which means everybody knows they’re not probabilities at all. …

    What does it mean to understand all the numbers stuffed into [0,1]?

    Because we cannot think of probabilities for each possibility in the continuum, therefore we call (certain) flat priors improper? So, contrapositively, we don’t call (certain) flat priors improper because ___.

    A flat prior is improper because it does not have a finite integral. And its contrapositive statement is: if it has a finite integral, it is not improper.

    The uniform/flat prior over [0, 1] is not improper. In practice, we often consider the probability of an interval, not the probability of each possibility. And I won’t use improper priors to infer anything about probability since we all know improper priors are not probabilities.

  4. Tegmark:

    Not only do we lack evidence for the infinite, but we don’t actually need the infinite to do physics: our best computer simulations, accurately describing everything from the formation of galaxies to to tomorrow’s weather to the masses of elementary particles, use only finite computer resources by treating everything as finite. So if we can do without infinity to figure out what happens next, surely nature can too—in a way that’s more deep and elegant than the hacks we use for our computer simulations.

    Yes, our measuring apparatus are finite and yield only discrete and finite measurements However, those best computer simulations or numerical calculations probably involve mathematics that uses the idea of infinity.

    Anyway, perhaps doing with infinity would be a challenge for Tegmark’s students.

  5. JH,

    As someone who does those sort of numerical calculations, I’m struggling to think of where the underlying mathematics does make necessary use of infinity. There is a continuum theory, which contains integrals conventionally written up to infinity, which is what we try to model. The numerical version automatically introduces a cut off, so those integrals go to pi/a, where we are interested in the limit a->0. But the discrete theory is wholly well defined, and can be constructed from first principles without reference to the continuum theory. There are three possible places I can think of an infinity is needed: a) there is one place where we have a parameter beta which is proportional to 1/g^2, and we are interested in the g -> 0 limit. We conventionally write that as beta -> infinity, but strictly g is the physical parameter. b) There is one place where we need a pseudo-random number distributed according to a Gaussian distribution. c) We need to shift one integral from the real to the imaginary axis. In the continuum theory, this is done via a contour integral, with an integration range heading to infinity. I haven’t really thought about how this would work in the discrete theory, and maybe you need that contour integral to do it.

    But this is the thing: infinities only really show up in one place that I can think of (if an infinity appears elsewhere, we either say the theory is wrong, or regulate the theory, and tweak its parameters so that the parameter we want to make very large cancels out in the final result, or it pops up just as a shorthand for a limit of one over a parameter as that parameter tends to zero): as the limit of an integration. But you can always split up an integral

    \int_0^{\infty} dx f(x) = \int_0^1 dx f(x) + \int_1^{\infty} dx f(x)

    and use a substitution y = 1/x to remove the infinity in the second term. In principle, I think it should be fairly easy to reformulate modern physics to avoid continuous infinities: you can always treat infinity as \lim_{\delta -> 0} 1/\delta, and \delta will cancel out of any final result so you can then take the limit safely at the end of the calculation without invoking infinity. It will just make it rather more cumbersome. Discrete infinities (i.e. series with an infinite number of terms) might be a bit harder to eliminate. There are some key results where my usual proof includes the sum to infinity of a geometric or Taylor series etc. Maybe those proofs can be achieved in a different way, though, which doesn’t need the series.

  6. However, those best computer simulations or numerical calculations probably involve mathematics that uses the idea of infinity.

    In the same manner, we assume frictionless surfaces, perfectly elastic collisions, ideal gasses, and so on. They are simplifications to reality that make the calculations easier. Depending on circumstances, the simplification may give results not too badly different from the exact form. For most moderate values of n and p, the Poisson gives pretty much the same result as the binomial.

  7. Wow, this is a highly relevant continuation of the WMB notes to the “Real Philosophy of Sci..” guest post. Namely questioning “chance” an “randomness” directly leads to the problem of infinity and of course probability as well.
    A sober and practical viewpoint is badly needed, obviously.
    And that applies eminently to entropy discussions where stellar numbers are easily jotted down – on paper and in representations, but so much less and rare in the practical world and experimental physics.

  8. I can only say again that infinity of progression, or regression, or division is just as imaginary as the square root of -1.

    There is a simple refutation of Zeno’s paradox that, for the life of me, I can’t conceptualise to describe just now.

    Anyhow, the observation of someone above that you can’t bang your nose on the wall because no matter how close you get to it you’re still only halfway from where you were is something that all mathemagicians should try to demonstrate.

  9. Zeno’s paradox … no matter how close you get to it you’re still only halfway from where you were

    Zeno’s paradox requires taking steps to the halfway mark between current position and goal. No sane person would do this. As one of my prof’s put it: you eventually come close enough for all practical purposes. Which means in many cases, that 0.999999 is as good as 1. Computers take advantage of this. The value 0.1 can’t be represented in the IEEE floating point standard so an approximation is used.

    The real number line is no less imaginary than the complex plane. Think of i as a unit vector along an orthogonal axis and a complex number as a two variable sample.

  10. Zeno was trying to poke holes in the continuum, or as we might say, exploring the logical consequences thereof. There were several of these paradoxes. Moderns, who have long forgotten what the questions were, do not take the answers seriously. They think he actually believed motion was impossible — like folks who today believe in the “block universe.”

    The thing is, using a mathematical model does not dispel the paradoxes. Mathematics has no causative powers. He was kvetching about the physical problem of how motion occurs, what motion is, not whether you could approximate its quantitative extension conveniently.

    Time is generally thought, by both philosophers and physicists, to have the topology of the real number line. As it is for the continuum of real numbers, so it is for the continuum of time, which is considered to be constituted by a succession of instants totally ordered by the earlier than/later than relation. It is arguable that at least parts of classical mechanics can be modeled using time as merely dense and not continuous, but standard models of both classical and quantum physics take time to be continuous; if it were dense this would not affect the argument to be presented here. Further, no standard theory takes it to be discrete. The question to be considered is whether time is made up of durationless instants.

    The short answer to the question is that it cannot be, for the reason Zeno originally proposed in respect of lines: if a temporal interval or a line segment were composed of durationless instants or points of zero length respectively, then neither the interval nor the segment could have a length greater than zero. The usual approach to the problem now is to invoke the mathematical consistency of proposing finite sums to series of infinite numbers, but this does not dissolve the paradox since infinite series never literally sum to a finite number, they only converge on it as a limit. (For instance, the series 1?2 + 1?4 + 1?8 … converges on 1.) Similarly, when derivatives are invoked to model motion at an instant, they are only ever limits to an infinite series of measurements: instantaneous velocity, for instance, is no more nor less than v = lim (dt ? 0) dx/dt. While it is convenient to ignore talk of limits when calculating using such concepts as instantaneous velocity, and while such concepts may be mathematically consistent, the ontological truth is that limit concepts do not denote actual entities. And an instant, conceived of as durationless (not as a fleeting “chronon” or “instanton” of some physical speculation), is just such a limit concept – it is not an actual something, it is an actual nothing. And no number of nothings can ever make up a something, matter what sorts of mathematical technique are invoked.
    David S. Oderberg, “Instantaneous Change Without Instants” [emph added]

    IOW, the famous dictum of Zeno’s teacher, Parmenides: “From nothing comes nothing.”

    As the famed statistician, George E.P. Box once wrote, “All models are wrong, but some are useful.” The question is always how badly wrong they may be before they cease to be useful. After all, some egregiously bad models have performed usefully well. Copernicus’ mathematical model of celestial motions was polycentric, perfectly circular, and required more than twenty epicycles and still failed to predict the position of Mars. Tycho’s model made all the same celestial predictions as did Copernicus’ model and did a little better on Mars. It also explained why no one had seen any stellar parallax or Coriolis effects, which Copernicus could not do without introducing additional unproven hypotheses. So successful matching of the physical phenomena does not mean that the structure of the model matches the structure of reality.

    Infinite convergent series means that we can calculate a useful approximation to the problem of motion in a continuum. It does not obligate physical reality to ante up.

  11. Zeno’s paradox requires taking steps to the halfway mark between current position and goal.

    But first, you must get halfway to the halfway point; and before that, halfway to the halfway point to the original halfway point; etc. Of course, you could also express this as a series of ever-decreasing additions, but Zeno (or more properly, Aristotle paraphrasing Zeno) worked it backward. It’s less that you can’t reach the goal than that your motion can’t get started. (Remember, Zeno like his teacher Parmenides believed in what we call the ‘block universe,’ and in this motion is not possible. It is only an illusion. Hence, the refutation by Diogenes the Cynic — on hearing the argument, he stood up and silently walked across the plaza — would not impress a Parmenidian.)

  12. BB,

    Computer algorithms for solving math calculations yield approximations to mathematical analysis. For example, an algorithm for square root, involving the first derivative, gives you an approximated value of \sqrt{2}.

    Derivatives are rigorously defined by using limits, which in turn require the notion of infinity. Integral, definite (e.g., \int_0^1 dx f(x) ) or indefinite, also employs limits.

    Even the convergence of an algorithm involves the notion of infinity; but criteria for convergence and non-convergence need to be set up so that the computer stops running, i.e., gets cut off, due to the limited capacity of our computers or our own.

    Cannot escape the notion infinity. The answer to whether infinity is understood varies depending on the context of interest, I think.

    No one yet really knows how or whether there can be discrete physics theories that involving math, but I suspect they can ever be free from the notion of infinity. Well, my suspicion, just like Tegmark’s essay, is vague on alternatives and mute on solutions. 🙂

  13. YOS,

    Depending on circumstances, the simplification may give results not too badly different from the exact form. For most moderate values of n and p, the Poisson gives pretty much the same result as the binomial.

    I was not thinking about using one function to approximate another; see my comments above. The Poisson distribution indeed can be shown to be a limiting case of a Binomial distribution under certain convergence rate assumptions. In practice, such approximations are not necessary nowadays because of the computational power we have.

  14. YOS,
    “the refutation by Diogenes the Cynic would not impress a Parmenidian.”

    But it should imoress us. All reasoning and argumentation is subsequent to a person registering an external object i.e a case of motion. Motion comes first and reasoning afterwards. This is what Father Jaki in his writings stressed on.

    That we are living in a simulation or even the most extreme solipism can not be refuted by arguments does not mean that these ideas are silly.

    Reasoning can only commence in a sane fashion after the reality of external world including motion is affirmed.

  15. I must be missing something, but this post is utterly baffling and I cannot make heads or tails of it.

    “While “most physicists and mathematicians have become so enamored with infinity that they rarely question it,” Tegmark writes, infinity is just “an extremely convenient approximation for which we haven’t discovered convenient alternatives.” Tegmark believes that we need to discover the infinity-free equations describing the true laws of physics.”

    Tegmark can believe in whatever he wants, but until he pinpoints the actual problem and the causal link to infinitary mathematics and / or comes up with something better, this type of proclamations is empty verbiage. Do not like real numbers? Fine. Mr. Tegmark can wash his hands of the bloody thing, vow never to use it again, and then come up with something better (there are fundamental, well-known obstructions to this program, but he is welcomed to try).

    For example, Tegmark suggests that the geometric continuum is one source of problems. There are some reasons to believe that the geometric continuum is indeed *not* a good model for space-time. But all known alternative models are no less infinitary than the continuum; depending on your strictures, some are more palatable in some senses, some are less. Want to get rid of space-time? Once again, great. Show the actual problems with present conceptions and come up with something better.

    There are different senses in which the infinity creeps in physical theories, because there are different senses of “infinity”. Some are “unphysical” (whatever system they are suppose to describe cannot ever be in such a state), some are not and are uneliminable — provably so and in a relatively strong sense.

    “Not only do we lack evidence for the infinite, but we don’t actually need the infinite to do physics: our best computer simulations, accurately describing everything from the formation of galaxies to to tomorrow’s weather to the masses of elementary particles, use only finite computer resources by treating everything as finite.”

    Our “best computer simulations” do not spawn ex-nihilo but are extracted via applied numerical analysis (that pesky infinitary mathematics again) from infinitary models.

  16. Mactoul:
    Certainly, but there are people even here in this comm box who insist the world is a Minkowski 4D block universe. Same thing.

Leave a Reply

Your email address will not be published. Required fields are marked *