Jaynes’s book (first part): https://bayes.wustl.edu/etj/prob/book.pdf
Permanent class page: https://www.wmbriggs.com/class/
Uncertainty & Probability Theory: The Logic of Science
Link to all Classes. Jaynes’s book (first part):
Video
Links:
Bitchute (often a day or so behind, for whatever reason)
HOMEWORK: If you think probability is real, see if you can measure probability itself. Not occurrences of things, but of probability the substance!
Lecture
Due to Circumstances, I’ve had to temporarily move inside to record the lecture. The lighting is terrible. The focus is on and off. I don’t like whiteboards. Too much like my face, which has rapidly lost its summer tan (contrary to rumor, summer’s up here are at least six weeks). However, I thought it was a better idea to put up the lecture rather than delay the Class. If I have to move inside again, I’ll see if I can fix these problems.
Judging by comments and questions across the various platforms where I put up this Class, it seems we need some review and catchup. I promise to get to questions about “randomization” either the next class, or the one after that. But today I want to emphasize the basics about probability.
There are only two main interpretations of probability, with many branches flowing from each. The two are: (1) probability is in your mind, a matter of thought, quantifying or qualifying certainty and uncertainty; (2) probability is in things, a real property of Nature, perhaps causal, like a field, or at least a quality that can be operated on by other forces. We accept (1) and reject (2).
IN YOUR MIND
Probability as logic is as natural as you can get. Every uncertain, and every certain, situation fits in this scheme. This happy fact is not shared by the other theories. Let me prove it to you, again, in what I hope are new ways.
Every probability, or logical argument, of any kind can be represented in this form of “equation”:
Pr(Y|X),
where Y is the “conclusion” of the argument, and X is the evidence or premises, or data, or whatever is accepted as true and considered probative of Y. I call Y the “proposition of interest.” It’s the thing we want to know. X is what we know or assume about Y. This is evidence probative of Y.
Simple as that. Sometimes, as we learned (and proved from scratch), this can be made into math, and a number or limit or some quantification emerges, or sometimes it’s loose and qualitative. If the equation is logical and valid (“All men are mortal,” etc.), with X the premises and Y the conclusion, then the probability of Y is 1, meaning true, or 0, meaning false.
This works for everything. You can’t stump it. This is the replacement for all hypothesis testing, for all parameter estimation—toss out all those old formulas!—for Bayes factors, for everything. Simply state probability, quantified when possible, based on the explicit evidence you accept. Change the evidence, change the probability!
This is the form of all models. Yes, including AI, and all physical models, too. We accept this form in engineering, at least outside academia, and we accept it in daily life. We use it as the basis of judgements in uncertain situations constantly. Should I cross the street now or wait a second? Which line at the grocery store will be the shortest wait? How much meat should I buy? Who will win the presidency?
Logical probability can handle one-off events. It can handle events in a sequence. It can handle theoretically infinite events. Like ordinary logic, it can handle events that are impossible! Which is to say, counterfactual events.
The example I use in the video is sports. Suppose the quarterback didn’t trip on a tuft of grass and got his pass off in time, which connected, and which brought our team ahead; then, given the time left and other knowledge of The Game, it is likely our team would have won. These questions make perfect sense, we engage in them all the time, but it’s only in logical probability they make sense.
This is the replacement system for the morass of inconsistent, bizarre, and even fallacious reasoning using in classical modeling and “testing”. See if you can break it. Propose a situation you think is beyond it, and I’ll show you how it fits the scheme after all.
In Things
In ways nobody has been able to explain, or even tried to explain, probability is in things. It is part of Nature. People say things “have”, or belong to, “distributions.” They will speak of “true values” of parameters in distributions, which they proceed to “estimate”.
One-off events cannot have probabilities, because Nature must “draw” events from distributions, which are only created for multiple events; under frequentism, these must be infinite events. Probability in this way is causal, or is at least operated on by causes. Chance is a force, or is subject to force.
The example I used in the video is Income. You are negotiating a salary with an employer, and will receive some figure. Which, if probability is real, has a life of some kind. The Income will be “drawn from” the “true distribution” of Income. Which distribution is that? The one for jobs will just this Title? Only this country? Only this year? Titles similar to this one? Only for people your age? Plus or minus five years? What?
It has to be something. It has to be very specific, indeed unique, physical description, something exact. We might not be able to identify it, but it must be a real physical event, possessing its own nature because it possesses its own unique probability, which can be operated on by Nature in some way, to change it “randomly”. Or it can operate on its own, using its chanciness. Nobody knows how Nature does this, but it does, and must, if probability is real.
Suppose you, being a white male, receive a lowball amount. Somehow Nature caused the probability to produce this value, maybe not wholly, but in cooperation with other forces. Does the probability force field influence the mind of the HR flak writing your offer? Since your salary is on the low end of the distribution (see the video for a picture), whatever this distribution may be, there must build up in Nature a kind of restorative force to ensure that new values are higher, to keep the “shape” of the distribution intact. Luckily, the next candidate boasts of incredible DIE qualities, and will receive (though nobody yet knows this) a whopper offer.
There’s no way to tell the story of “physical” probability without scenarios like these. Scenarios which, to this late date, remain unexamined in detail. Best we have are measures of things, which are said to possess probability, and that their measure is somehow verification they do! This is a prime circular fallacy, but you simply cannot talk people out of it. I can measure things, too; measuring fits logical probability fine. The burden of proof is on those who claim there is physical probability to show how it adheres to things, or imbues them with causal powers. I have no burden to prove these occult powers do not exist.
Often people believe in physical probability because of the reason that physical things are used in probability equations.
Take the die toss. One of its premises (which is part of our X) is that the physical object has six sides, each with a different number on its face. The tossing is the other part of X. We want, say, Pr(Y=6 shows | X). Since X has definite, real (no joke) physical measures, part of Reality, it seems that probability is physical.
Yet I have a machine (well, it’s not yet perfected) that tosses a die such that it always lands 6. That’s my evidence, or X’; this Pr(Y|X’) = number close to 1. You don’t know I’m using this machine (say), so your information is X, and your probability 1/6. This cannot be if probability in inherent in the die. Everybody must get the same probability.
I often play this trick: I am thinking of a number between 1 and 5. What is the probability it is 3? To me, using my information, which I’ll call X’ again, it is Pr(3|X’) = 0 or 1, depending on whether I am thinking of 3. To you, since you do not know what I know, the probability is not 0 or 1, unless you happen to pick X’ as your information. Most will pick X = “The number can be 1, 2, 3, 4, or 5”. With that, the probability is 1/5.
I usually pick a number like 1.73. Which, my friends, is between 1 and 5.
This emphasizes better than anything I can think of that probability is a matter of information only!
Physicists will insist quantum mechanical outcomes are physically random, because the probability equations have the wave function in them (they’ll also insist, with a chuckle, that “nobody understands QM”). Something like
Pr(Y| ?) = f(?,?),
for some function f() with parameters involving the wave function and whatever else we need (stuffed into ?).
Well so what. The die also had physics in it. But we know the die toss is controlled by causes and conditions. We know the QM outcome is too! We just don’t know what these are. It is the most-often repeated mistake to say QM events do not have causes. Balderdash. All we can say is that these causes are non-local. And that we can’t know them. That’s tough cookies on us, but that does not mean that no such causes exist.
That’s the same fallacy twice. Our knowledge of things is not the things themselves. Repeat that.
Besides all these objections, we have that physical probability cannot abide a counterfactual. But we can. We understand them fine. Logic can handle impossibilities, but Nature cannot, because Nature cannot produce the physically impossible. We can believe impossibilities with utter ease. Have you never heard of elections?
Next week I think I’ll clear up the randomization questions, before we move to “distributions”.
Subscribe or donate to support this site and its wholly independent host using credit card click here. Or use the paid subscription at Substack. Cash App: \$WilliamMBriggs. For Zelle, use my email: matt@wmbriggs.com, and please include yours so I know who to thank.
