People say “random” variables “behave” in a certain way as if they have a life of their own. To behave is to act, to be caused, to react. This is reification, perhaps caused by the beauty of the mathematics where, literally, the equations undergo biogenesis. The behavior of these “random” creatures is expressed in language about “distributions.” We hear, “Many things are normally (gamma, Weibull, etc., etc.) distributed”, “Y is binomial”, “Height is normally distributed”, “Independent identically distributed random variables”.
I have seen someone write, “[Click here to] see a normal distribution being created by random chance!” Wolfram MathWorld writes, “A statistical distribution in which the variates occur with probabilities asymptotically matching their ‘true’ underlying statistical distribution is said to be random.” Examples abound.
All of this is wrong and indicates magical thinking. It is to assume murky, occult causes are at work, pushing variables this way and that so that they behave properly. To say about a proposition X that “X is normal” is to ascribe to X a hidden power to be “normal” (or “uniform” or whatever). It is to say that dark forces exist which cause X to be normal, that X somehow knows the values it can take and with what frequency.
This is false. We are only privileged to say things like this: “Give this-and-such set of premises, the probability X takes this value equals that”, where “that” is a deduced value implied by the premises. Probability is a matter of ascribable or quantifiable uncertainty, a logical relation between accepted premises and some specified proposition, and nothing more.
Let S = “Sally’s grade point average is x”. Suppose we have the premise G = “The grade point average will be some number in this set”, where the set is specified. Given our knowledge that people take only a finite number of classes and are graded on a numeric scale, this set will be some discrete collection of numbers from, say, 0 to 4; the number of members of this set will be some finite integer n. Call the numbers of this set g_1, g_2,…, g_n.
The probability of S given G does not exist. This is because x is not a number; it is a mere placeholder, an indication of where to put the number once we have one in mind. It is at this point the mistake is usually made of saying x has some “distribution”. Nearly all researchers say or assume “GPA is normal”; they will say “x is normally distributed.” Now if this is shorthand for “The uncertainty I have in the value of x is quantified by a normal distribution” the shorthand is sensible—but unwarranted. There are no premises which allow us to deduce this conclusion. This is pure subjective probability (and liable to be a rotten approximation).
When they say “x is normally distributed” they imply that x is itself “alive” in some way, that there are forces “out there” that make, i.e. cause, x to take values according to a normal distribution; that maybe even the central limit theorem lurks and causes the individual grades which comprise the GPA to take certain values.
This is all incoherent. Each and every grade Sally received was caused, almost surely by a myriad of things, probably too many for us to track. But suppose each grade was caused by one thing and the same thing. If we knew this cause, we would know the value of x; x would be deduced from our knowledge of the cause. And the same is true if each grade were caused by two known things; we could deduce x. But since each grade is almost surely the result of hundreds, maybe thousands—maybe more!—causes, we cannot deduce the GPA. The causes are unknown, but they are not random in any mystical sense, where randomness has causative powers.
What can we say in this case? Here is something we deduce: Pr(x = g_1 | G) = Pr(x = g_2 | G), where x = g_1 is shorthand for S = “Sally’s GPA is g_1” (don’t forget this!). This equation results from the so-called symmetry of individual constants, a logical principle. The probabilities are equivalent to G = “We have a device which can take any of n states, g_1, …, g_n, and which must take one state.” From the principle we deduce Pr(x = g_i | G) = 1/n.
“Briggs, you fool. That makes GPAs of 0 just as likely as 4. That isn’t possible.”
Is it not? I see you haven’t taught at a large state university. Anyway, the probabilities deduced are correct. What you are doing in your question is adding to G. You are saying to yourself something like “Pr(g_n | G & What I know about typical grades)” which I insist is not equal to Pr(g_n | G). Either way, x does not “have” a distribution.
Homework 1: discover instances of abuse. Homework 2: What’s wrong with the phrase “independent identically distributed random variables”? Hint: a lot.