# Randomness And The “Null” Hypothesis

Last week of teaching!

Randomness is not a cause. Neither is chance. It is always a mistake to say things like “explainable by chance”, “random change”, “the differences are random”, “unlikely to be due to chance”, “due to chance”, “sampling error”, and so forth. Mutations in biology are said to be “random”; quantum events are called “random”; variables are “random”. An entire theory in statistics is built around the erroneous idea that chance is a cause. This theory has resulted in much grief, as we shall see.

Flip a coin. Many things caused that coin to come up heads or tails. The initial impetus, the strength of the gravitational field, the amount of spin, and so on. If we knew these causes in advance, we could deduce—predict with certainty—the outcome. This isn’t in the least controversial. We know these causes exists; yet because we might not know them for this flip does not imbue the coin with any magical properties. The state of our mind does not effect the coin in any, say, psychokinetic sense.

Pick up a pencil and let it go mid air. What happened? It fell, because why? Because of gravity, we say, a cause with which we are all familiar. But the earth’s gravity isn’t the only force operating on the pencil; just the predominant one. We don’t consider the pencil falling to be “random” because we know the nature or essence of the cause and deduce the consequences. We need to speak more of what makes a causal versus probabilistic model, but a man standing in the middle of a field flipping a coin is thinking more probabilistically than the man dropping a pencil. Probabilities become substitutes for knowledge of causes, they do not become causes themselves.

The language of statistical “hypothesis testing” (in either its frequentist or Bayesian flavor with posteriors or Bayes factors) is very often used in a causal sense even though this is not the intent of those theories. We must acknowledge that the vast majority of users of models of uncertainty think of them in causal terms, mistakenly attributing causes to variously ad hoc hypotheses or to “chance.”

Suppose the user of a model of income has input race into that model, which occurs in two flavors, J and K. The “null” hypothesis will be incorrectly stated as “there is no difference” between the races. We know this is false because if there were no difference between the races, we could not be able to discern the race of any individual. But maybe the user means “no difference in income” between the races. This is also likely false, because any measurement will almost surely show differences: the measured incomes of those of race J will not identically match the measured incomes of those of race K. Likewise, non-trivial functions of the income, like mean or median, between the races will also differ.

If the observed differences are small, in a sense to be explained in a moment, the “null” has been failed to be rejected; it is never accepted. Why this curious and baffling language is used is because of Popper’s notions of falsifiability, which we have discussed before. For now, all we need know is that small (but actual differences) in income will cause the “null” to be accepted. (Nobody really thinks in terms of failing to reject, despite what the theory says.) When the “null” is accepted it is repeated that there is “no” difference between the races, or that any differences we do see are “due to”, i.e. caused by, chance.

But chance isn’t a cause. Chance isn’t a thing. There is no chance present in physical objects: it cannot be extracted nor measured. It cannot be created; it cannot be destroyed. It isn’t an entity. The only possible meaning “due to chance” or “caused by chance” could have is magical, where the exact definition is allowed to vary from person to person, depending on their fancy.

Some thing or things caused each person measured to have the income he did. Race could have been one of these causes. An employer might have looked at an employee and said to himself, “This employee is of race K, therefore I shall increase his salary 3%.” Or he might not have said it, but did it anyway, unthinkingly. Race here is a partial cause. This kind of partial cause might have happened to some, none, or all of the people measured. If the researcher is truly interested in this partial cause, then he would be better served to interview whoever it is that assigns salaries and so discover the causes in each case. Assuming nobody lies or misremembers and can bring themselves to proper introspection, this is the only way to assign causes. But that is time consuming and expensive. And researchers have anyway been falsely taught that if certain statistical thresholds are crossed, causality is present. This fallacy is the cause of the harm spoken of above.

Even if the null is not rejected it is still possible that some or even all of the people measured had salaries in part assigned because of their race. There isn’t any way to tell looking only at the measured incomes and races. If the null is accepted, no person, it is believed, could have had their incomes caused partially by their race. Again, there isn’t any way to tell by looking only at the data. But when the null is accepted, almost all researchers will say that causality due to race is absent—replaced, impossibly, by chance. The truth is we have no idea and can have no idea, looking just at measured race and income why anybody got the salaries they did.

When the null is accepted, but the researcher had rather not accept it, perhaps because his hypothesis was consonant with his well being or it was friendly to some pre-conception, he immediately reaches to factors outside the measured data. “Well, I accepted the null, but you have to consider this was a population of new hires.” That may be the case, but since that evidence did not form part of the premises of the model, it is irrelevant if we want to judge the situation based on the output of the model. I have much more to say on this when discussing models. It is anyway obvious, that, to his credit, the researcher is looking for causes. Even if he gets them wrong, that is always the goal.

## 49 thoughts on “Randomness And The “Null” Hypothesis” Leave a comment ›

1. You’re going to hurt the feelings of those who maintain the universe, the planet, life on the planet, etc, all evolved over eons of time due to random chance. Sure, you can say “survival of the fittest”, but that occurs by random chance on where the organism is—the optimal environment, he survives. Non-optimal, he dies. Then there’s that random mutation thing. Some people get very offended when you don’t believe random mutations produced people. Same for the Big Bang—it’s all so chancy! How could that not be the cause??

2. JohnK says:

Yes, but your task is gargantuan, if not actively Sisyphean, as some of your most faithful readers, even guest posters, take the pre-eminent, defining, task of science to be the creation of ‘falsifiable’ hypotheses. With the word ‘falsifiable’ seemingly having a self-evident meaning.

Rather than suggest that you also develop an entire philosophy of science, along with all your other splendid and sparsely-remunerated tasks, I propose that at least you instruct eager readers that, whatever science’s procedure positively is (which is beyond your brief), there are problems with taking science’s procedure to be the creation of ‘falsifiable’ hypotheses.

Of course also, a more objective definition of ‘null’ hypothesis will also work. As you have written many times, there are differences in (real, predictive) probability that can be judged as too small to matter in the particular case. Frequentists, and Popper, hoped to trump otherwise-inevitable social disagreements about what precisely is “too small to matter,” but as you have shown countless times, they failed.

Their intellectual kludges are unsound, and thus social, or personal, or rather, eminently moral argumentation, appears to be an inevitable component of real science. Which is to say, “here I stand” is ultimately a personal and moral statement.

3. Robbie says:

Looked at a description of the ball/peg array demonstration (Galton Board).

“…when there is an equal chance of bouncing left or right, then the pegs collecting in the bins form the classic “bell-shaped” curve of the normal distribution.”

Does that improperly invoke “chance” as a cause? Does anyone have a better description?

4. JH says:

If you know that everything has a cause or causes, then what you think is random would be non-random if you have complete knowledge of the circumstances. For instance, coin tosses have been shown non-random. So, really, if a thing is truly random, it cannot be part of a causal chain.

Well, I have no problem with people using the phrase “caused by chance,” to mean “due to unknown reason.”

5. With respect to randomness and probability, I’m not sure quantum mechanics fits into your scheme. Randomness and probability are not “causes” of getting a particular measurement, but they certainly play a role, over and above what we could possibly know about the system. That is to say, in your example of the coin flip, sufficient knowledge of pertinent variables enables us to predict accurately the landing side; such knowledge would not be sufficient for a quantum mechanical situation. The only way to get a predetermined value for a measurement would be to prepare the system to be in a given state (e.g. passing spin 1/2 particle through an inhomogeneous magnetic field).

6. JH: You have no problem with imprecise language. Interesting.

7. DAV says:

Randomness is not a cause.

No? Then how do you explain the “Down with Random” and “Random is Scrambled” signs being carried about?

“Caused by chance ” is just another way of saying “coincidence”. JH is right. The real problem is that hypothesis testing doesn’t actually address the the model or the hypothesis and therefore can’t address cause,

People have a strong belief that because they can do things by intention (or think they can) then everything else must have been guided by intention. This despite the fact that as we do things intentionally many times unintended consequences are the result.

That everything we see around us could have arisen from mutations caused by chance (unguided coincidence) mangling of DNA given enough time is quite reasonable albeit difficult to prove it actually occurs or is the primary mechanism.

Just like Evolution, there is no real proof that every event has a cause. It’s just reasonable to assume so. For all we know, some things could very well happen for no reason whatsoever. Experience suggests it shouldn’t be the first guess.

8. Bob,

“With respect to randomness and probability, I’m not sure quantum mechanics fits into your scheme. Randomness and probability are not “causes” of getting a particular measurement, but they certainly play a role, over and above what we could possibly know about the system.”

Due to the nature of language you appear to be making a very obvious logical mistake. Stop using the words ‘randomness’ and ‘probability’ and exchange these for ‘unknown causes’. Now change your sentence to read:

“Unknown causes are not “causes” of getting a particular measurement, but they certainly play a role, over and above what we could possibly know about the system.”

Now that your sentence has been re-written I suspect it doesn’t claim what you were trying to express. Which translated basically means, “nothing causes something”.

9. Shack Toms says:

@Bob Kurland
Absolutely correct. The kind of uncertainty in Quantum Mechanics does not reflect a lack of knowledge about a state of affairs, it represents a fundamental lack of determinacy in the state of affairs. The distinction has measurable consequences.

But the point of the article is well taken. In most cases, probability is a matter of uncertainty about the underlying causes and the causal relationships. It requires a subtle experiment to demonstrate that the uncertainties in quantum mechanics are not of this kind. See, for example, Quantum physics says goodbye to reality.

10. “The kind of uncertainty in Quantum Mechanics does not reflect a lack of knowledge about a state of affairs, it represents a fundamental lack of determinacy in the state of affairs”

The “fundamental lack of determinacy” is predicated on assumptions found in classical physics. Which is to say that the universe always behaves in a way that is consistent with local realism. Except QM teaches us it doesn’t. So there is no particular rational reason I can think of to accept this claim as a true. Now, it could be true. There just aren’t any good reasons to believe it as true.

“In most cases, probability is a matter of uncertainty about the underlying causes and the causal relationships. ”

Not most cases, all cases. At least, that position is the more plausible one.

” It requires a subtle experiment to demonstrate that the uncertainties in quantum mechanics are not of this kind.”

There is no reason to believe that QM is an exception, except hubris. If the standard model was a complete theory of reality then there might at least be an argument there. (The counter argument being that we couldn’t be sure it really was a complete theory or the only theory consistent with all observations). But the standard model is not even that.

11. Will, I don’t intend to get intend to get in a slanging match such as you and Rodriguez have been engaged in. I’m not sure I can teach you anything about quantum mechanics in short comments, and I’m positive you can’t teach me anything. I’d suggest you read some background books: to start with, “Quantum Enigma” (Rosenblum & Kuttner– see quantumenigma.com), although this may be a little too horsies and duckies for your taste; Albert Messiah’s two volumes on quantum mechanics (see https://archive.org/details/QuantumMechanicsVolumeI and
https://archive.org/details/QuantumMechanicsVolumeIi)
and finally Bernard d’Espagnat “On Physics and Philosophy”.
After you’ve read those, then we can discuss stuff in quantum mechanics on line.
Or perhaps you’ve already read them (or the equivalent)?

12. Bob you’re the only one imagining there is a ‘slanging match’ going on. I’m have a rational discussion. If you want to criticize Dr Brigg’s position on the nature of causality then you’re more than welcome to do so. But as soon as your arguments are criticized in turn, don’t accuse others of bad faith. Either respond to the counter arguments or acknowledge you don’t know, and be quiet. Nobody is disputing QM. QM is what it is. Interpreting QM philosophically is an open field, however. If you have a preferred philosophical interpretation defend it. If you can’t defend it, perhaps it’s best not to express it.

13. Scotian says:

Bob, not to disparage Messiah’s QM but I have bad memories about that book. It was the text used in my graduate course at McMaster. Tough going but I got through it. Since I never blame my instructors I assume it was a lack on my part due to a weak undergraduate preparation. The two volumes still sit, gathering dust, on my bookself. I preferred Dicke and Wittke:

https://archive.org/stream/IntroductionToQuantumMechanics/DickeWittke-IntroductionToQuantumMechanics#page/n1/mode/2up

which I was assigned as self study the summer before I took graduate Quantum. Sometimes I think that an education is something that one survives, or not, as the case may be.

14. Sheri,

“You’re going to hurt the feelings of those who maintain the universe, the planet, life on the planet, etc, all evolved over eons of time due to random chance.”

This claim is something of an urban myth among religious conservatives. Evolutionary biologists, and their supporters, DO NOT claim that evolution is a process of ‘random chance’ (uncaused causes) but only use the word ‘random’ as short hand. Let’s say I want to find out the best way to cook a steak. I try seasoning, basing, just salt, medium heat, high heat. I freeze my steak and then drop it into the fry pan. I try different types of fry pans. Eventually I work out which method works best. This process is not ‘random’ as such. Yes I tried all kinds of different things not knowing what would or wouldn’t work. So only in that sense (the guesswork) was anything close to randomness involved. That’s why evolutionary biologists use the term ‘natural selection’ to describe the process. It’s not called ‘random selection’ for a reason.

15. Will, it’s clear you have either not read or have misunderstood my original comment in this thread. I have no quarrel with what Briggs has to say about randomness and the null hypothesis. If you read the comment carefully you will see that I said his example predicting a coin toss is not applicable to quantum mechanics. And that is correct, whatever QM interpretation one might prefer. And that’s all I have to say about it… I will not be baited into an extended discussion where there is not a common ground of knowledge of the subject matter.

16. Scotian, Messiah was my third text on QM, read outside of a course trying to learn about angular momentum–that’s the section with which I’m most familiar. The first was Schwinger’s notes, the Second was Dirac. I agree, it’s very dense, but a good prep for going further.

17. Bob,

“I have no quarrel with what Briggs has to say about randomness and the null hypothesis.”

Yes you did, because your claim was of the form ‘I agree but…’ and the but was QM. I criticized that position and now you’re trying to back peddle on what you wrote. Don’t be scared or paranoid if you have one of your beliefs or claims criticized. If you can’t defend your claim, say you don’t know, and be done with it. That’s fine. Maybe that will help you reflect on why you believe what you believe. (Which for most people seems to be based on some style of argument from authority.) To debate and counter debate is what makes for an interesting conversation. If you can’t defend your argument, don’t besmirch or accuse the presenter of the counter argument as nefarious. It’s childish.

18. Will: We’ve been through this before. First, you use the term “religious conservative” as if it has some significance to me. Or maybe to you.

Your example of the steak is based on a goal–the best way to cook a steak. As far as I can tell, there is no goal in evolution. It’s a process of natural selection and random mutations that are then “selected” for. Evolution is not in search of a goal like your steak cooking. It just happens.

(Natural selection is random. Organism A lands in a compatible environment and thrives. Organism B lands in an inhospitable environment and dies. This is random–where they land and if they can adapt. No one can predict anything about future generations, etc. There is utter randomness to this and millions of years required to accommodate the process. Enough random mutations and landing in places where survival can occur and we allegedly land where we are today. I just cannot agree with you on this, I can’t.)

19. Sheri,

“First, you use the term “religious conservative” as if it has some significance to me.”

Remember I am not a Progressive, or at least not a ‘post modern’ Progressive. I don’t view the word ‘religion’ or ‘conservative’ as insult words, so there is no need to be defensive here.

Natural Selection has an element of serendipity but it is not primarily, or even mostly, a random process. It’s more of a ‘brute force’ approach. The climate may get dryer, the climate may get wetter, different food sources may grow or migrate. All of these options are, relatively speaking, rather limited. So what you call ‘randomness’ is also rather restricted. It is true that there is nothing teleological in Natural Selection. But there is nothing teleological in biology or for that matter, in science. Are you happy with your computer? Your car? All products of science and technology that are non teleological. So why be selectively skeptical of Natural Selection?

Let’s say I write an algorithm that is designed to use various strategies to break your email password. If the strategies fail, a brute force approach is taken. The brute force approach will eventually break the code. Is the brute force method a ‘random approach’ ? No. It’s trying, methodically, everything that will work, until it reaches its goal. There is nothing random about it. But it certainly ‘looks like’ the algorithm is trying different ‘random’ passwords until it unlocks the safe, so to speak.

“No one can predict anything about future generations, etc.”

Of course we can. It’s called Artificial Selection. We can make cows produce more milk. We can breed dogs to become smaller and cuter. If anyone wished to bother, we could introduce changes to the natural environment (simulate climate change so to speak), and we would then observe adaption. We don’t di this, because of cost and time, and we don’t need to anyway. Artificial Selection has already made the demonstration.

Anyway, the point of original comment was merely to clarify what you thought people you disagree with, thought about something. Since they don’t think what you think they think, it helps to know this. This will at the very least, make you better able to argue against them, if you at least understand their argument better.

20. Will, you may know what you’re talking about, but I respectfully suggest you don’t read or understand what I have written. And that’s all I have to say…no point in going any further. It’s not a question of interpretation of quantum mechanics, it’s how quantum mechanics is used and calculated. As I’ve pointed out in a post on my blog, arguing on the internet is an exercise in futility when there is not a common area of understanding.
http://rationalcatholic.blogspot.com/2014/02/a-lesson-from-two-homilies-dont-argue.html
You may comment again, and again–I will not be baited anymore.

21. ” It’s not a question of interpretation of quantum mechanics, it’s how quantum mechanics is used and calculated.”

That’s not the question or the point. You are making a mistake and you’re refusing to acknowledge it. Let me summarise, and they if you don’t want to discuss, we leave it at that.

If you wish to preserve the concept of local realism (taken from classical mechanics) then one must necessarily believe in random processes. The argument is, and I am of the view that this is a very good argument, that there are no such things as ‘random’ processes. So is this an ‘attack’ on QM? No more than special relativity is an ‘attack’ on Newtonian mechanics. Both QM and Newton are correct in their respective domains. But you are not saying this: by introducing some flavour of the claim that ‘randomness does not exist EXCEPT for QM’ you are making a philosophical assertion, beyond a mathematical interpretation of QM, whether you understand this or not. QM is non classical. It demonstrates, i.e., entanglement et al., that it does not preserve local realism. You can’t make the claim that QM processes are ‘random’ because randomness is necessary to preserve local realism when we know QM does not obey the rules of local realism anyway! Hence, you and Scotian are making philosophical claims that are (a) speculative and (b) we have good reasons to be skeptical of.

I do not know if Dr Brigg’s is right or wrong. I suspect he is right for the reasons I’ve explained. If you keep posting comments that Dr Brigg’s is right for most things but not for QM, I’m going to keep asking you to defend that argument.

22. Milton Hathaway says:

Will – you lost me on your evolution argument. Are you saying that you believe in the existence of an ‘intelligent cook’?

23. Milton,

Nothing to do with intelligence, nothing to do with randomness either. If I wrote an algorithm to break your email password, you could say the algorithm is ‘goal orientated’ but it’s not. It’s just executing a brute force task. If you say the process is ‘random’ you’re still confused. Sure, it looks random. But it isn’t. There is nothing random in the process at all. Of course, using informal language you might ask, is this algorithm trying passwords randomly until it breaks in? And I’d say, ‘sure, that’s what it’s doing’. But here the word ‘random’ is used as a convenient place holder for something else entirely. But now I’m starting to repeat the point that Dr Brigg’s made. So if you’re still unclear, re-read his article at the top of this page. I also suspect Dr Brigg’s holds certain views about causality because they are consistent with a certain theological world view that he has. If so, that’s fine.That’s not the direction I am coming from, but it’s interesting we still meet at the same point.

24. Larry Geiger says:

I’m the guy in the back on the right. Trying to stay awake. As soon as I figure out what you people are talking about, I’ll get my pencil out and squint at it too.

25. Shack Toms says:

@Will Nitschke

If you wish to preserve the concept of local realism (taken from classical mechanics) then one must necessarily believe in random processes.

I don’t think local realism can be preserved, regardless of whether you believe in random processes.

The reason your idea fails is entanglement, which is the principle that the strict conservation laws hold (strictly) despite the randomness in individual measurements.

You might be able to get away with saying that random processes determine, say, the spin (intrinsic angular momentum) of an electron, but that cannot explain how the total angular momentum of a collection of electrons is precisely conserved, despite the randomness in measurement of each individual electron. With this constraint, if there are two electrons, in a system where the initial total angular momentum is known, then measuring the spin of one will completely determine the spin of the other, thus the randomness within the measurement process goes away to coordinate with another, possibly remote, measurement. That change in the randomness of the process, in itself, would violate local realism.

Depending on your interpretation of the virtual particles in Feynman diagrams you might believe that conservation laws can be violated briefly, but once you sum the whole series of integrals represented by the diagrams it is never violated in the sum—”real” particles don’t violate the strict conservation laws.

Local realism is a more fundamental concept than classical mechanics. Einstein wanted to use local realism to demonstrate that quantum mechanics couldn’t be fundamentally sound. This was more fundamental that saying it contradicted classical mechanics, the contradiction with classical mechanics wasn’t an issue at all.

But the Einstein-Poldosky-Rosen paradox, which presented this objection to quantum mechanics, merely showed that quantum mechanics was inconsistent with local realism. Einstein called this “spooky action at a distance”. This was taken by many as a validation that QM was not fundamental, that the randomness that appeared must arise from some underlying, unknown mechanism (hidden variables). It wasn’t because this contradicted classical mechanics, but because it contradicted local realism, a more fundamental tenet.

It took 40 years before Bell devised an experimental test that would ascertain whether the hidden variable theory was a viable alternative. Aspect did the experiment, and hidden variables failed. The test did not depend on accepting quantum mechanics as a part of the hypothesis. The Bell-Aspect experiment, unlike E-P-R, did not pit quantum mechanics against local realism, it simply tested local realism.

Bell himself thought that “locality” was the part of local realism that failed. In physics, the concept of action at a distance is anathema, akin to saying that “God did it”. Bell, a believer, wasn’t so shocked by this concept.

Later experiments, the ones I referred to were based on ideas by Leggett, found that the problem is, under assumptions that would be even more strange if violated, a problem not with locality but with “realism”, the idea that the real world is in a definite state and that the issue with indeterminacy is a measurement problem.

I believe that the problem is, however, resolved under the many-worlds interpretation, first advanced by Everett in 1957. Under this theory, the indeterminacy is a matter of a fundamental uncertainty about which state of the quantum ensemble we are in, but once a measurement determines that state, then the conservation laws hold, as they do in every one of the states in the ensemble.

In a way, that would justify the initial assertion that it is a matter of knowledge of the states, except that the states within the quantum ensemble can affect each other (e.g. give rise to interference patterns), so again the indeterminacy itself appears to be a part of nature, with measurable effects.

These theories are so wacky that I doubt anyone would take them seriously, except that the agreement with experiment is astounding. Any alternative theory would have to produce equally wacky results in order to conform to the experimental data. The wackiness of the theory is driven by the wackiness of the experimental results.

26. Shack Toms says:

Looks like I left off the end of the blockquote, which should have been after the first paragraph.

27. Will: Apologies. I am used to the use of “conservative” and “religious” used in a negative way.

I’m not skeptical of natural selection. I just don’t view natural selection as a cause that leads to a specific outcome. Again, I see no “goal” in it, though your examples speak of “goals”. Having a goal implies a process with a cause or purpose. Are you saying that if I played around with parts long enough I’d create a computer accidentally? (I do know some inventions are happy chances, but most are not.)

I see in your response to Milton, you are saying your algorithm is not “goal oriented”. Then why would you write in the first place? If there was no goal, it would be a waste of time. Now, the algorithm itself has no goal, but it’s creator did. Is this what you mean?

You are using “predict” in a way I am not. Again, there is cause involved in the cattle–we intervene with purpose. Not all ideas pan out so the predictions can be way off.

I still disagree with the whole idea of natural selection and random gene mutations being the cause of anything in the usual sense of the word cause. This is a really dumb example, so please don’t take offense, but this reminds me of the “give enough monkeys typewriters and enough time and eventually they’ll write like Shakespeare” argument, which made no sense whatsoever. It’s just as likely they will take the typewriters apart. More so, actually. I have the same response to evolution as a “cause”. It’s a process, yes.

After reading over your answer to Milton, you and I may actually be in agreement as you say you and Briggs are. I’m not sure.

28. OK, Will, I’m going to give it one more shot to see if I can teach you something about quantum mechanics, although it’s my impression that you’re more interested in winning arguments than learning..but that may be mistaken.
Here’s Briggs statement in the article that I was commenting on:
“Flip a coin. Many things caused that coin to come up heads or tails. The initial impetus, the strength of the gravitational field, the amount of spin, and so on. If we knew these causes in advance, we could deduce—predict with certainty—the outcome.
The last part “If we knew these causes in advance we could deduce–predict with certainty the outcome” I would bold-face or underlined.
“That is to say, in your example of the coin flip, sufficient knowledge of pertinent variables enables us to predict accurately the landing side; such knowledge would not be sufficient for a quantum mechanical situation. The only way to get a predetermined value for a measurement would be to prepare the system to be in a given state (e.g. passing spin 1/2 particle through an inhomogeneous magnetic field).”
The coin flip has two possible outcomes, each of which is (approximately) equally likely to occur. Briggs is saying (as I understand it) that if one knew all the pertinent physical variables and initial conditions, one could predict whether the coin would land heads or tails.
An quantum mechanics experiment with two possible outcomes is passing a spin 1/2 particle through an inhomogeneous magnetic field. Again, there are two outcomes: the particle can be deflected up or deflected down, depending on the spin direction. (I’m speaking approximately and qualitatively here for the benefit of non-physicists.) Now there is no way of predicting whether the spin will be deflected up or down, no matter what or knowledge of the spin system, unless we have prepared it to be oriented in a particular direction by previously passing it through an inhomogeneous magnetic field and selecting the deflected up or deflected down particles.
The same sort of argument applies for the double slit experiment. There is no way of predicting a priori where the particle will land on the detecting screen.
This scheme is part of the quantum mechanical apparatus–it is empirical and has nought to do with any interpretation of quantum mechanics so your talk of quantum mechanical interpretations is irrelevant.
Now, you can learn or you can argue. I’ve said all I’m going to say and will not engage in any kind of discussion with you in the future.

29. Ye Olde Statistician says:

Does that [a description of Galton’s quincunx] improperly invoke “chance” as a cause?

The causes of the bead ending up in this slot are that first it bounced left off that pin, then it bounced right of this pin, and right again off the other pin, and so on into the slot. That is, it resulted from a series of small causes, no one of which was dominant. Another bead can wind up in the same slot by bouncing differently off other pins. But the results are not uncaused or “caused by chance.” They are caused by the pins (and gravity, and the elasticity and diameters of the beads).

Randomness is a description of the outcomes and has nothing to do with causation. “Caused” does not mean “predictable,” either.

I just don’t view natural selection as a cause that leads to a specific outcome.

Why would anyone expect a generic cause to lead to a specific effect? Causes are proportionate. A generic cause leads to a generic effect. In this case, a population more apt to survive and reproduce in its niche. This motion “toward aptitude” is called ad-apt-ation. There will always be more than one way to survive in a niche.

30. JH says:

DAV,

Statistical modeling (using probability) is atool and can be used for prediction or causal explanation. A strong theory might rely on data and statistical modeling to explain the causal association. Think of drug development, where designed experiments and statistical modeling for causal explanations are must. In this case, with a certain setup, testing the null hypothesis of zero effect, using whatever statistical method, is equivalent to testing of non-causal association. Statistical tools cannot provide the causality, but the accompanying theory can.

Also, further to my previous comment, if every event has a cause, it only appears random as it can be eventually shown to be nonrandom. A truly random event is therefore causeless. But then, doesn’t it also mean “something comes from nothing.” (What causes the lowest-level cause?) A contradiction. Any idea where I went wrong?

31. Shack Toms says:

The “fundamental lack of determinacy” is predicated on assumptions found in classical physics. Which is to say that the universe always behaves in a way that is consistent with local realism.

No, that is just the opposite of what is true. The lack of determinacy may be discovered in an experiment that tests the hypothesis of local realism. The results of the experiment do not presuppose it.

Except QM teaches us it doesn’t. So there is no particular rational reason I can think of to accept this claim as a true. Now, it could be true. There just aren’t any good reasons to believe it as true.

Except for the experiments whose results violate the assumption of local realism, without making any assumption that QM is true.

There is no reason to believe that QM is an exception, except hubris. If the standard model was a complete theory of reality then there might at least be an argument there. (The counter argument being that we couldn’t be sure it really was a complete theory or the only theory consistent with all observations). But the standard model is not even that.

The violation of local realism does not depend in the least on accepting QM, much less on accepting the standard model. So your counter-argument doesn’t address the point.

32. Races “J” and “K”, eh? Jew and Kulaks? We all know how that went down…

33. Shack Toms,

Nothing much I disagree with in your first post except for the minor quibble that there is an assumption you can pull apart local realism from classical mechanics. (I once wrote a paper in uni arguing that you could not.) It’s a bit like claiming you can separate a portrait into the frame and the paint.

In my view many words is a ludicrous attempt to preserve local realism *at all costs*. (It solves the problem of randomness at the expense of introducing infinities. But let’s leave that for now.)

This is where we disagree and why:

“No, that is just the opposite of what is true. The lack of determinacy may be discovered in an experiment that tests the hypothesis of local realism. The results of the experiment do not presuppose it.”

No, you continue to get it backwards, sadly. The lack of determinacy disappears as soon as you jettison the requirement that the “hypothesis” of local realism is true. The experiment determined that local realism was untenable. It did not jettison deterministic causality. This is where you are getting muddled up. In fact, strangely, you noted as much by introducing the many worlds interpretation. This points to a contradiction in your logic. Nothing wrong with that! Most people, even very very smart people, make exactly the same mistake.

“Except for the experiments whose results violate the assumption of local realism, without making any assumption that QM is true.”

Also wrong. Let me try another way to communicate this to you. The claim is:

IF local realism is TRUE, randomness MUST exist.

This is what we’ve learnt from QM. Fine. But since local realism is NOT true, we can draw NO inference from the above premise. Your logical mistake is to continue to draw the invalid inference.

“The violation of local realism does not depend in the least on accepting QM, much less on accepting the standard model. So your counter-argument doesn’t address the point.”

Have no idea where your above confusion arose. I have stated multiple times now that I am perfectly satisfied with QM and also the Standard Model, as far as they go. Whatever point you think I’m addressing is not what I’m addressing. What I’m actually addressing I explained again in my previous paragraph.

Bob Kurland,

Little point commenting on what you wrote as you’re adopting the assumption that I don’t understand QM or I’m “misinterpreting” it or something along those lines. The issue of contention relates to the metaphysics, not the physics, which I assume we all accept. Read what I wrote to Shack Toms as it equally applies to you. If you don’t understand some part of my argument, let me try to flesh it out further. Don’t put words in my mouth, as both of you are doing now. You’re going off in all sorts of odd tangents. Last time you though I was talking about hidden variables, which is irrelevant to my argument also.

And BTW, this is not about winning or loosing debates. We’re just having a conversation. These are philosophical principles we are discussing and nobody knows the ultimate ‘correct’ answers to them. All we can do is discuss why we think the answer might be in one direction instead of another. Nobody is going to win this. But if we can properly understand what we are trying to communicate to each other, then we might all get the door prize.

34. Sheri,

In relation to goals, I think the issue here is that you don’t need someone to have set the goal to produce a particular outcome. Now, this doesn’t mean that someone didn’t set the goal, only that it’s not a requirement. My email password cracker has a “goal”. It’s actively seeking to crack your email. But it’s not consciously aware of the goal. The goal is not found in the algorithm. (Although someone later on might come along and work out what it is.) Now, your argument is, as far as I can determine, that an email password crack algorithm could not exist, unless *someone* wrote it. This is something nature can’t do itself. Someone or some thing has to set a “goal” for nature, otherwise nature would not be able to do anything. I’m assuming the requirement, also, is that whoever provides the “goal” must have consciousness. Let’s say I love fjords and I love them so much I travel to Norway to enjoy them. Fjords have very complex structures. They twist and turn, they have an interesting baroque style. The question is, did someone or something have to set a “goal” for nature to create a fjord or was it an inevitable part of a natural process? Or maybe you don’t have a problem with fjords but you do have a problem with a virus or a single celled organism. At some point you are going to draw a line in the sand and say, “these complex things didn’t require goals but these complex things did”. The interesting question is, did these things past the line require a conscious designer or is this a failure of imagination?

35. Will: Good question. I suppose it could be failure of imagination, but then we’re getting into fiction rather than fact. Suppose all of it required goals. Natural selection works the way it does because it helps reach the goal. I guess to me it’s no different to believe a goal was necessary than to believe one was not. I see no reason for a line–all things could be part of the goal or all things might not be. Black and white, no gray. This seems mostly to be a philosophical question rather than a science one. I admit I’m prejudiced to philosophy and tend to see things through that lens. Maybe I’m just not able to see this through a scientific lens.

36. Sheri, you may be interested in Thomas Nagel’s “Mind and Cosmos: Why the materialist neo-Darwinian conception of nature is almost certainly false”.
(See
http://web.mit.edu/philosophy/religionandscience/nagel4.pdf)
Nagel is a well-respected philosopher who has written another classic: “What is it like to be a bat?”. He is not a theist, so his argument that there is a teleological principle operating for the Universe is not predicated on religious grounds. An echo of Aquinas and Aristotle–a final cause?

37. Shack Toms says:

Nothing much I disagree with in your first post except for the minor quibble that there is an assumption you can pull apart local realism from classical mechanics. (I once wrote a paper in uni arguing that you could not.) It’s a bit like claiming you can separate a portrait into the frame and the paint.

I suppose it depends on what you mean by classical mechanics, but Einstein meant to use local realism to attack QM. An attach on QM that said it violated classical mechanics wouldn’t have been taken very seriously.

I think there are a number of principles of physics that are more fundamental than either QM or CM. Whether you include these principles in those theories is, I suppose, a matter of definition. (That is, to use your analogy, whether you consider CM to be the portrait or the paint.)

In my view many words is a ludicrous attempt to preserve local realism *at all costs*. (It solves the problem of randomness at the expense of introducing infinities. But let’s leave that for now.)

I think Everett actually introduced it to avoid the apparent dependence on consciousness in the Copenhagen interpretation (where consciousness causes “collapse” of the wave function). He wanted to apply quantum mechanics to the early universe, presumably before any consciousness was around to collapse anything.

Personally, I think many worlds is correct as a mathematical model, but I don’t think the worlds in the model are physically realized. The realized state is the state we are aware of (just as the realized moment in time is the moment we are aware of, despite that physical laws describe many times). The infinities of many worlds are thus avoided in the same way that the similar infinities of “many times” are avoided.

There are a number of experiments that are hard to explain without many worlds. A particularly fascinating one is the “Elitzur-Vaidman Bomb Tester”.
The interesting thing about that one is that it sends a photon through beam splitters and gains information about the system under test (the bomb) only in the case that the photon can be known to have not traveled on the path with the bomb in it. So it gains information about the bomb despite going the other way.

The lack of determinacy disappears as soon as you jettison the requirement that the “hypothesis” of local realism is true.

That is a very interesting statement. I am not quite sure I see why you think it is true, but I would like to know. Actually, now that you mention it, I might agree that without local realism, determinacy might be restored.

For example, if the universe were somehow recorded, and then played back, the rerun of the universe would be completely determined, and yet it would violate no laws of physics. However, such a playback would violate local realism.

However, this determinism would not be the kind of determinism that physics relates to. That is, the determinism would not be a determinism that is based on physical facts. I don’t say that as a counter-argument to what you said, just to clarify.

“Except for the experiments whose results violate the assumption of local realism, without making any assumption that QM is true.”

Also wrong.

Where does the assumption that QM is true enter the Bell-Aspect experiment? It was specifically designed to avoid that assumption, in contrast to the EPR paradox.

The claim is:

IF local realism is TRUE, randomness MUST exist.

Who makes that claim? On the contrary, if local realism is true, then it is possible that the claims of QM arise from hidden variables, and thus there is a possibility that the apparent randomness is only a matter of our lack of information about the system.

However, local realism is not true, I think we are on the same page there, although I didn’t realize we were.

38. Shack Toms,

I am more familiar with Deutsch’s many worlds interpretation than Everett’s but I think they get us to the same place. The reason why Deutsch promotes many worlds in his philosophy he is trying to preserve local realism, although it’s an odd form of local realism. (!) Or what might be described as the ‘common sense’ view of the universe. Deutsch is an anthropomorphist. He believes the universe was designed for human consciousness. Now while I happen to think all of that is nonsense, the reason why I mention this is because this philosophy preserves determinacy. The point is not that Deutsch is on the right track with his philosophy, but that a different physical interpretation of QM, completely consistent with the maths of QM, *does* preserve causality.

Therefore, QM doesn’t rule out causality as is commonly assumed, but it does, of course, rule out local realism. Now, as you correctly point out, hidden variable theories might also preserve causality and hidden variables are not ruled out in principle, only hidden variable theories that attempt to preserve local realism. While I am of the view that hidden variables is a wrong headed approach, you are completely correct in that this may be a viable option. At least in principle. If so, I won’t argue as it agrees with Dr Brigg’s position and also my own.

Here is the reason why I am skeptical of theories that claim that randomness is a ‘thing’ in physics. This is because randomness is also an undefined term. It has no mathematical definition. It is essentially a place-holder, or an undefined variable. A magical fix-it to be plugged into an equation to fudge an insurmountable problem. I follow the pragmatic philosophical position that if something can’t be computed, it does not exist. (If you want me to believe that faerie dust is real, tell me what it’s made of, and the answer had better not be ground unicorn horn).

Now, in discussions of this type, someone is always going to jump in and declare that randomness is a ‘thing’ because probability *is* the study of, or at least a description of, randomness. I’m not going to try to disentangle people’s confusion on that point, other than refer them back to Dr Brigg’s article at the top of this page. I’m not suggesting you are confused on that point. I’m only mentioning it because *someone* invariably jumps in and attempts to derail the discussion with that non argument.

I prefer to use the phrase “unexplained variance”;
“random, or “noise” is misleading and pejorative.
also…anyone who claims to be an even barely literate frequentist knows that the null is never accepted…one fails to reject the null.
The leap from rejecting the null to accepting the alternative hypothesis is only as valid as the reasoning underpinning the choice of an explanation for the existence of the alternative.

40. davideisenstadt: Kind of court, where you are found “not guilty” rather than “innocent”?

yes, in fact having been there, I can tell you the comparison is apt, very apt.

42. Will, my Screwtape demon has prompted me to respond to your last comment.
Please tell me where is the mistake in saying one cannot predict the path of a spin 1/2 particle going through an inhomogeneous magnetic field if the particle has not been previously selected to be deflected up or down in a similarly oriented field? I’m talking about the gedanken experiment, not the metaphysics underlaying the experiment.
By the way, I’ve enjoyed the exchange between you and Shack Toms and have learned something new: the Elitzur-Vaidman Bomb Tester.

43. Bob,

You’re not making any sort of mistake. But again I have to refer you to the article at the top of this comment thread. Not being able to predict something is not equivalent to concluding that something is non-deterministic. All it means is that you lack information to make the prediction. Because I’m a computationalist in philsophical outlook I don’t give two hoots about preserving local realism. The math is the maths. The maths is deterministic. The maths doesn’t have to conform to human standards of common sense or our expectations.

I have no basis except suspicion to draw the following conclusion: Dr Brigg’s rejects non causality because the universe was constructed by a divine creator. Therefore there is a plan. The definition of a plan is that it is deterministic. I, on the other hand, reject non causality because nobody can describe non causality. (Other, of course, than declare non causality is “that which we cannot know.”) Since I don’t believe in magic, I don’t believe in non causal things.

44. Bob,

On re-reading my reply I realise it is highly unsatisfactory. It is difficult to get into the specifics without getting into the maths, which may muddy the waters further. I’m going to reach for an analogy which you’re either going to understand, or not. If you don’t follow it, blame it on the deficiencies of my analogy. The analogy requires some technical understanding to grasp. I have not thought of simple way to avoid this difficulty.

Imagine a Turing machine, von neumann architecture. It holds a memory space, which is a 1 dimensional matrix of, say, one quadrillion logical states. (TRUE and FALSE.) As with any von neumann architecture, part of the matrix consists of algorithms and part of the matrix is data. The algorithms generate a three dimensional presentation of a universe similar to ours. Let’s say half of those quadrillion logical states represent the observable states of the universe. (If you think I’m hinting at a holographic projection, you would be on the right track.) Now zoom in on a few trillion or a few billion logical states anywhere in that portion of the matrix. Let’s say you try to ‘snap shot’ the logical states of that portion. What do you see when you zoom in using your microscope? What you see are TRUE|FALSE ( 1’s and 0’s) blinking in and out of reality, for all apparent purposes, completely chaotically. Why do the state changes look totally random when subject to any sort of statistical analysis? It’s a matter of scale. Everything is deterministic and everything is connected to everything else. But if you take a reductionist approach, all you are going to find at granularity is apparent white noise.

45. Will: I think I understand what you’re saying. I actually believe there is a creator, but if there was not, that would not remove causality. You explained it well. Also, your analogy with the Turing machine is very good, or at least it made sense to me!

46. Will, I’ll chew on what your last comment after I read about “Von Neumann architechture”. In your previous comment you mentioned that you believed in “cause” but not necessarily a Creator (if I understood it correctly. You might be interested in Thomas Nagel’s similar view, put forth in “Mind and Cosmos”. A free online pdf is available:
http://web.mit.edu/philosophy/religionandscience/nagel4.pdf

47. Sheri,

I thought of another analogy last night, which doesn’t involve a Turing machine, but it can, as analogies tend to do, detract from the point I was trying to make. If you “get” the Turing Machine analogy, stick with that. Here is my other attempt –

You’re tasked with studying a portion of a busy road in a very busy place. Let’s say Bangkok or Hong Kong, where the roads are packed with vehicles 24 hours a day. Your task is to try to predict when a particular spot on that road will be empty of a vehicle, and when it will occupied. If you attempt that analysis, for all intends and purposes, the state of that part of the road will be completely random. You can, of course, do a probabilistic analysis. You might see a consistent pattern; at 2am in the morning the pattern is different from 7am in the morning, for example. You might be able to understand these states in terms of, for example, waves.

Now, observe that everything on that road is causal. People are travelling in their vehicles for predictable reasons; to get to work, to get home, to deliver goods to stores, etc. But you can’t predict when a particular place on a particular road will be occupied or not occupied. Again, because you’re looking at the problem from the wrong level of granularity.

Bob,

“In your previous comment you mentioned that you believed in “cause” but not necessarily a Creator…”

I’m not sure how can arrive at a Ceation without a Creator while simultaneously arguing that there cannot be effect without cause. What atheists and theists are really debating, usually without realizing it, is not the existence or non existence of the creator, but the nature of the creator.

48. Will: Your road analogy works, too, but I liked the Turing machine better.