Probability Isn’t “Fair”: An Answer To Senn; Part III

Varying degrees of “fairness.”
A quick reminder that we’re trying to unpack the meaning of the “is fair” in the proposition “This die is fair,” and trying to deduce the probability this proposition is true given (and only given) the evidence “This die has been rolled five times and showed five ‘6’s.” See the previous installment for why.

“Is fair” can take one of several definitions. Our predicament arises from not being clear which, and by mixing versions at different stages of the problem.

Meaning 1: In any finite number of tosses, the proportion of observed tosses will match the probabilities deduced from the first example; i.e., the observed proportions will show 1/6 ‘1’s, 1/6 ‘2’s, and so on, or whatever is closest to these if the number of tosses is not divisible by six.

Assuming Meaning 1, and given our evidence, we deduce the probability the proposition is true is 0; it is false. If the proposition were true, we should have seen some combination of five numbers with one missing (e.g. ‘6’, ‘3’, ‘5’, ‘1’, ‘4’); the missing could have been any number between ‘1’ and ‘6.’ (I keep the quotes around the outcomes to help us recall these are labels and not numbers.)

Meaning 2: In any finite number of tosses, the proportion of observed tosses will approximately equal the probabilities deduced from the first example; i.e., the proportions will approximately show 1/6 ‘1’s, 1/6 ‘2’s, and so on.

Assuming Meaning 2, and given our evidence, we deduce the probability the proposition is true is not calculable. The probability is unknown—because “approximately” is not defined. If “approximately” means (and I do not jest) “Leave me alone, I’m tired of playing dice” then the proposition is true, because the observed frequencies are more than close enough for somebody who doesn’t give a damn about dice. If you fail to appreciate this example, you are in for tough times ahead; so pause here and make sure this sinks in.

If “approximately” means “not varying more than 5% from” then the proposition is deduced to be false because, of course, the observed proportions have differed by more than 5%. But if “approximately” means “not varying more than 90% from” then the proposition is deduced to be true, because the observed variations are within this bound.

Who gets to decide what “approximately” means? Well, you do; as does Senn; as do I. Fights start over things like this. What is the one and only definition of “approximately”? There isn’t one! It depends on the situation. As we saw, for some it could mean “Leave me alone”, for others, say casinos, it would be much tighter.

Think this ambiguity bad? It’s even worse than this.

Meaning 3: In any finite number of tosses greater than or equal to 6, the proportion of observed tosses will equal the probabilities deduced from the first question; i.e., the proportions will be 1/6 ‘1’s, 1/6 ‘2’s, and so on, or whatever is closest to that if the number of tosses is not divisible by six.

Given this and our evidence, the proposition is not true or false (1 or 0) but somewhere in between because we haven’t yet reached the limit of 6 tosses. Kind of. If the die were tossed just one more time (for 6 in total), then there is no way the observed proportions could equal the deduced probabilities. The proposition would then be false. But the die hasn’t been tossed just one more time. It could be tossed 100 more times. Who knows? But we still have the feeling, based on the observations, that the future tosses won’t bring the final proportions in line with the deduced probabilities (I keep repeating deduced to remind us these are not subjective guesses nor are they estimates).

Our evidence and assumed definition isn’t proof the proposition is false, especially if we consider it with respect to Meaning 4, which is the same as 3 but with “approximately” put in usual place. Nor is the proposition true. But we also don’t seem in a strong position to quantify the probability. Nothing in the world wrong with that. Not all probability is quantifiable. See the original series for why.

If we insist on asking the original question, we’re left trying to understand what “is fair” could mean. We need to settle on a definite, unambiguous meaning before we can progress. And then even if we do we’re going to be left will all kinds of nagging questions about real dice.

Real dice have weight distributed unevenly. There’s no way to create perfect balance. We can prove this easily: displaying the numbers, which are of different shape, creates an imbalance, however minuscule. It might be possible to engineer a die down to the level of a quark, so that each side is precisely the same number of quarks across, and that the mass of the die is uniform at the Planck scale (except for the surface where the displays are). In practice, for macro-scale dice, this is impossible. But maybe some physicist will figure it out for some tiny thing. Even then, he won’t be sure that the strings which comprise the quarks are the “same length” everywhere and uniformly (if that even makes sense to say).

But even supposing we have this toy, we have the problem of tossing it. How? Onto what kind of surface? From what height? How much spin? With what downward force? In what gravitational field? After all, if we want to discuss tossing a “fair” die, all these things have to be considered. Tossing is part of “fairness.”

It is at this point it dawns on us that we’re on a fool’s errand. If the die were perfect, as we imagine (and as a logical die is in effect), and if the environmental conditions and forces were known precisely, then we’d know—before tossing—exactly what the outcome would be. Indeed, if the forces did not vary, the die would land the same way each time.

Point is, just by our knowledge of physics we know that any real die and its tossing environment isn’t “fair” in any complete physical sense. There’s no point to the original question. No real die (or its tosses) is “fair” in this sense. The proposition is contingent.

We’re asking the wrong question. What we really want to know is if the die is “fair enough”, and to answer that requires, as above, knowing what decisions we want to make regarding the die.

What we can do is to deduce the probability of seeing any arrangement of observations, either before seeing any observations whatsoever, or conditional on our initial knowledge of six-sided (logical) objects supplemented by a set of observations specified by the evidence. (We do this using Bayes’s theorem: see the next Parts.)

In other words, we can then make statements like this, “Given our evidence about six-sided objects and the old observations, the probability of seeing departures of future observed proportions at least as great as X% from the deduced probabilities is Y.” If Y exceeds a threshold, then we act as if the die is not “fair”, but if it is less than this threshold, we say it is. The threshold varies depending on the application. For the person sick to death of dice, X is unimportant and Y is quite low. Casinos want a small X and large Y for obvious reasons.

We’ll never have 100% certainty that any real die is “fair” in this (final) sense that Y = 0 (for vanishingly small X), because we knew before we started that question dealt with a contingent matter, and we are never 100% certain of contingent matters (though we can be 1 – ε certain).

And you’ll notice that nowhere did we confuse the observed proportions—i.e. the relative frequencies—as probabilities. We knew the probabilities and used them to discern whether the relative frequencies were in line with the them; this is what we meant by “fairness.”

We have proved what we set out to show. That we don’t, at least for the kinds of examples that Senn provided, need two kinds of probability. The one kind—probability as logic—was enough.

Yet there is still more to understand. Stick around!

Update We could also form statements like this: “Given our evidence and old observations, in the next n throws, there is probability Y of seeing X ‘1’s” and so forth. In other words, this and the previous example are predictions, statements of uncertainty of the future (or of that which is as yet unseen).

11 Comments

  1. I think we’ve all lost perspective here on what is REALLY important.

    Given that the dragon has already killed the party’s healer and you need a critical strike on the next attack, is it better to utilize a d20 system or d6? And what rituals should be performed to best give the party a chance to survive?

    Really, Briggs, it’s a shameful how you and Senn keep avoiding the important point: How do we utilize all this to maximize our loot and xp!

    See also the comic notes on:
    http://darthsanddroids.net/episodes/0099.html
    http://darthsanddroids.net/episodes/0363.html

    😉

  2. I’d say simply that a fair die is a die for which Prob(1)= Prob(2)= Prob(3)= Prob(4)= Prob(5)= Prob(6)= 1/6. This is enough to calculate Prob(Data|Fair die). If you want to calculate Prob(Fair die|Data), I’d say you’d need some prior distribution on all dice, and even then Prob(Fair die|Data) is going to be zero except for strange (essentially discrete) priors. I don’t see a problem with calculating Prob(Nearly fair die|data) for some pre-determined definition of “nearly” and a well-defined prior on all dice.

  3. I feel like we are going down a rabit hole of irrelevancy.

    1) A die is acutaly one of the most reliable “random number generators” that has ever been devised.

    2) There is no such thing as perfect information. I suggest, that if a good (but not perfect) die is thrown with more than a minimal level of horizontal, vertical and angular force, the result is not predictable, even with the best information available of the initial conditions.

    3) Why does it matter?

  4. Why are people so often obsessed with fair dice? Personally, I think loaded dice are a lot more interesting. What is the Briggsian approach for determining if dice are loaded, and if so, loaded for what?

    And on a slightly different note, I’m convinced that the NSA is remotely manipulating my 6-digit secure ID. How many times do have have to watch it turn over to prove that the digits aren’t random.

    I want applications I can use!!

    P.S. The Lady drinking tea is way more interesting than this dice stuff. You need to use some of your psi-busting examples.

  5. Doug M,

    There is no such thing as truly random numbers in the real universe. There is no best random number number generator because there are no random numbers. There are only number sets whose patterns are complex enough to appear to be random.

  6. For a die to be fair, it must also be impossible to predict the outcome of each throw. A die that always shows numbers in a predictable sequence, for instance 1,2,3,4,5,6, is not fair, even though each number comes up exactly 1/6-th of the time.

    This could very well be a good definition of fairness for dice. If a die is fair, the change that you predict the next throw correctly is 1/6.

  7. Mr. Briggs, I await your response to Senn’s points below.

    I am not criticising informal inference…

    Because it is perfect in theory…

    It does not explain how to become good…

    It is hard to see what exactly a Bayesian statistician is doing when interacting with a client. There is an initial period in which the subjective beliefs of the client are established. These prior probabilities are taken to be valuable enough to be incorporated in subsequent calculation. However, in subsequent steps the client is not trusted to reason. The reasoning is carried out by the statistician. As an exercise in mathematics it is not superior to showing the client the data, eliciting a posterior distribution and then calculating the prior distribution; as an exercise in inference Bayesian updating does not appear to have greater claims than ‘downdating’ and indeed sometimes this point is made by Bayesians when discussing what their theory implies.

    It is necessary, to an extent that is often overlooked, to establish exactly what the basis is for any particular prior distributions being established. This, as indeed are all the difficulties, is related to the first one of temporal coherence. It is important to make sure that one knows exactly what was known when. I am often asked by my clients in the pharmaceutical industry whether they should do a Bayesian analysis. I reply that they should when they wish to make a decision but reporting a Bayesian analysis is not a very useful thing to do. Faced with a series of Bayesian analyses one needs to be able to subtract the prior information first in case it or some element of it is common. It is an important irony that a Bayesian statistician wishing to do a Bayesian analysis will (usually) find it easier to do so if presented with a series of frequentist summaries rather than a set of Bayesian posterior distributions.

  8. Mr. Briggs, if you are to respond with an example, please use real-life, practical statistical data. No marble picking and dice tossing, please. If you need data, I have a set of temperature data that I have downloaded and cleaned up!

  9. JH,

    Answers to Senn’s comments coming. Hint: he’s right about subjectivist Bayes being screwy. But you know I am not in that camp.

    Plus, I only look at dirty temperature.

    If we can’t understand dice and marbles we’ll never get something as hard as temp. But see tomorrow.

    Sander (yours and JH’s were the only comments I read, since they’re on top),

    Your definition of “fair” circular. Read the example closer.

  10. . The maximum likelihood estimate of p would be the value of q that makes that chance largest. We can find that value of q explicitly using calculus; it turns out to be q = x/n, the fraction of times the coin is observed to land heads in the n tosses. Thus the maximum likelihood estimate of the chance of heads from the number of heads in n independent tosses of the coin is the observed fraction of tosses in which the coin lands heads.

Leave a Comment

Your email address will not be published. Required fields are marked *