The Podcast has been restored! Sorry for the mix up yesterday.
Here’s the story cut short. A woman was browsing a used book stall and came across a volume she owned as a child. The Wall Street Journal asks:
What are the chances? For most of us this is a rhetorical question, expressing our surprise at such a seemingly magical coincidence. But Joseph Mazur, the author of “Fluke: The Math and Myth of Coincidence,” has a more precise answer. The odds that Anne Parrish would stumble upon her book years later in Paris are 3,331 to 1 or, as he puts it, “slightly better than the odds of being dealt a poker hand of four of a kind.” Not so magical, then, after all…
Mazur assigns a probability to each link in the chain of events: the likelihood that Anne would visit Paris in 1929 (0.1, or 10%); the likelihood that she would browse the bookstands (0.3); the likelihood that the book would be there (0.01). The chances that all three would take place and that Anne discover her childhood book are therefore 0.1 x 0.3 x 0.01 = 0.0003, or 3,331 to 1.
This won’t do. Why? Consider this: Briggs assigns a probability to each link: the chance Anne visits the book dealer is 0.987654321; and 0 likelihoods for the rest. Chances she gets her own book are therefore 0.987654321 x 0 x 0 = 0. Hey. Why not? I am a full qualified statistician, holding the coveted posthole digger award.
This means, given my superior prowess at assigning odds, that Anne never got her book, even though she said she did. An investigation is in order.
How did Mazur find different probabilities? He made them up, just as I did. That’s because there are no probabilities “for” this “event.” None, except one. I mean one, as in 1. The probability Anne found her book is 1, because why? Because she found her book. It happened. Conditional on the evidence provided in the story, the probability “Anne found her book” is 1.
Conditional on whatever evidence Mazur made up, the probability is 0.0003. And, to be pedantic, conditional on the evidence I made up, the probability is 0.
All of these probabilities are correct, but only one of them is true. To see that, consider also that the proposition “Anne would have found her book had she tried again” makes no sense, but would have to if we can find conditioning evidence upon which we could agree.
Change the story to this. Anne went to the bodega and bought a lottery ticket, which won. Then, conditional on the evidence of this new story, the probability of “Anne won” is 1. But the probability of “Anne will win” given the evidence she bought a ticket and evidence about the rules of the lottery, the probability can be calculated. That seems like the right answer, which it is but isn’t.
This is because, in both the case of Anne finding her book and winning the lottery, cause is involved. Some (enormous) list of things caused both Anne’s book and Anne to meet at that time and place (and for Anne to have enough money on her). And some list of things caused the lottery to register the numbers that matched Anne’s tickets and some things caused Anne to have those numbers printed on her ticket.
Even though we don’t know the details of the causes of the ticket, we have a fair idea of their nature. This is why we can agree on the evidence needed to form the probability of “Anne will win.” Better evidence exists, however, and that is knowledge of the causes, but we don’t have that (though somebody might, say a physicist in charge of the lottery machine).
We have next to no idea of the nature of the causes of Anne finding her book, which is why we can’t agree, nor should we, on the evidence that gives the probability of “Anne finds her book.”
Anne finding her book feels like a coincidence, yet her winning the lottery doesn’t. Why? Because of our lack of any knowledge of cause in the first instance, and knowledge of the nature of cause (but not precise details) in the second.
We all know that something caused Anne to find her book, but we have no idea what. So it’s hard not to think that some overarching cause guided the event. This might even be true. There is no evidence in the event itself one way or the other. Positing a “guide” is simplifying a complex situation, and that is pleasing in and of itself.
There’s much more to be said about coincidence—which I’ll leave for another day.
Thanks to John Watkins for alerting us to this topic.
It’s very sad what math and science has done to life. Used to be it was simply exciting when someone found a book from their childhood, or a dog was found 3000 miles away from home and returned to the owners, etc. Now we have to compute “odds” (obsessive gambling behaviour?). Can’t it just be awesome or cool or even nice that she found the book? How or why or the odds really don’t matter. It’s the find that is the interesting part. Leave it “number-free”.
Post hole diggers? For goodness sake stop it!
Okay then,
Well there was an Aylesbury called Donald. Donald did unmentionable drake things but he also had a yellow beak, a curly tail and quacked like a drake.
Donald had two friends out of a clutch of seven: Jemima an Aylesbury duck and Daffy, a black Indian Runner, a dude with a ‘tude.
One day, Jemima came too close to a dog and lasted only a week of constant nursing. (Ducks cry because a tear fell from her eye as she sighed her last breath. ( anyway that’s off topic, I hate people who do that)
So, there were two.
Recently, so I’m told, a large white egg appeared in the duckhouse.
Then, on the day when Mr Briggs posted his comments on the third way on probability, which featured a duck and a drake cartoon, another large white egg appeared! The first one was considered a trick, or a stray mallard.
The second? Well, and Daffy is now chasing Donald who is changing his name to Daisy. Daisy still quacks like a boy and has a yellow beak, but has been laying an egg a day ever since.
This story is even more intriguing to me but you have to know the prequel. I’m not telling that.
A case of mistaken identity, probably, it’s not important and I’m not asking for instruction on how to inspect the genitals of ducks, thanks in advance.
Well, there goes the Drake Equation (not Joy’s drake, but this Drake https://en.wikipedia.org/wiki/Drake_equation).
All the uncertain and unknown causes aside, an estimate of likelihood could be made based on a few variables such as the number of books Anne owned as a child, the number of books published during her childhood, and the fraction of those that end up in used book stalls. These could be determined with a little research. We wouldn’t know how accurate the estimate was, but even a semi-informed wild guess might influence how many used book stalls we were willing to visit to search for those beloved books of nursery rhymes our mums read to us.
The following excerpt from a short story “Nexus” currently awaiting editorial judgment may be apropos of this topic:
Dr. Briggs,
So, suppose that you had a job (you’ve said that you don’t) and were the type that liked to place money at risk based on events whose outcome is unknown to you at the time that you place the money at risk, and that you had a WHOLE LOT of it. I.e., you were very rich and liked to gamble.
Anne meets you and informs you that she’ll visit Paris and frequent bookstores. For reasons unknown, Anne offers an even money bet that she’ll find a particular book from her childhood. Do you take it? I’m speculating that you would. On the other hand, were she to demand that you offer 10 billion to one against her finding it, would you not? I’m speculating that you wouldn’t (if you would, substitute a larger number). Somewhere in that range is your “point of indifference.” Does this not inform us as to your estimate, at the time the bet is offered, of the probability that Anne will find her book?
Sheri:
Do you really think that people in earlier times and previous generations didn’t think this way? I doubt that that’s true, it’s just that today’s media enables such speculations can be and are widely communicated. I know that I thought in similar ways as a child in the ’60s.
Rob Ryan…
The postulation that Anne would think to make such a bet is its own level of speculation. Postulating that Anne wanted to find that book is its own bit of guesswork. How many other books would have fit the criteria?
I just got back from visiting Florida. It happened in visiting Florida that I ran into my Aunt. Not a big difference from 1 of seeing here since she was in the plan. During our sojourns to the house that a Mouse built, we didn’t run into anyone we knew. Upon arriving at the airport for departure though, a family showed up that we did know.
This takes us back to the birthday question. “How many people do their have to be in a room for there to be a 50% chance that 2 people have the same birthdate”…
How many people on a vacation do you have to run into to run into someone who lives near you?
Making a bet with Anne ahead of time would require careful calibration…
If you can’t measure it, you can’t manage it.
If you measure it, IT WILL BE GAMED.
Never chase the bid. Always let the bid come to you.
Like Gary (above) my first thought was of the Drake Equation and tghe extreme unhappiness my scoffing at it in school caused the goof professing its significance – but there is a key difference: Anne is an active agent in the discovery process where the quack claim is entirely passive in nature.
Rob Ryan,
Right question but there is no practical answer.If Anne made the wager she likely has reason to think the book may be there so, her wager would lead one to believe the likelihood of her finding it is larger than one would imagine otherwise. In any case, the number of causes leading to Anne finding the book are unknown. This means that it’s not possible to put a number to probability. One could suspect that it is small but in reality can’t even show that.
Rob Ryan: I don’t remember thinking so much in terms of odds in the 60’s—though to be honest, I never thought of the 60’s as having much thought at all, so perhaps that had something to do with it. (Since I’m a bit cranky today, I’ll just stop there. Don’t want to offend.)
This reminds me of Fermi problems. Estimate how many dentists there are in Canada. I estimate 30 M / (2 X 40 X 50 / 2) = 15 k. Remarkably close.
To the various repliers: It’s easy enough to concoct scenarios where Anne isn’t scheming. But never mind that, simplify. Is it a coincidence when I bet on 14 on the roulette wheel and, by golly, up it comes? The physics of the result are currently beyond hope of calculating to that degree of accuracy (though various folks have sought bias or calculated to within sectors with reasonable accuracy). Of course it’s true that, after she found the book or the ball dropped in 14, the probability was 1 of that occurrence and 0 for a book she didn’t find and for 15 coming up.
But I’m in New Orleans at a symposium, flying back to California tomorrow. It would be a coincidence if I were to meet my client on our largest project waiting to board the same flight. And, in fact, I DO need to talk to him and will be in touch with him upon my return. I have no knowledge that he’s in New Orleans, no knowledge that he flies into John Wayne Airport, no knowledge of his personal life beyond the facts that he’s married and likes golf. I suspect that, were I to offer someone on this board an even money $10 bet where I win if I meet Bob at the gate tomorrow and the first responder bets that I don’t, I’d get a taker. I suspect that, if I offered $10 if I meet him to $1,000,000 if I don’t, I’d get no takers (arbitrarily assuming that people have that level of risk capital readily available). Throw aside suspicions of my surreptitiously offering to split the money with Bob or that I dishonestly actually know that he’ll be there. Somewhere in there is the “point of indifference” that represents your estimate of the probability that some guy in the construction industry who lives somewhere in Southern California happens to be in New Orleans and flying to John Wayne on my flight. I claim that that event, were it to happen, would be a coincidence. I further claim that the event has a probability and that that probability is greater than 0 and less than 1.
Sheri: I didn’t become an alcoholic and drug addict until the ’70s (I got clean and sober in 1979 and have been since). In the ’60s I was an introverted, reclusive math and science nerd. I was born in 1954.
Two further points:
1. I claim that the probability estimate is defined by the point of indifference since, obviously, a series of trials cannot take place.
2. To prevent the $ numbers becoming a red herring, let’s instead posit a game with points to wager with some wonderful prize for the winner of the most points after a long series of wagers (or some humiliating degradation for the loser or both). Everyone on the board agrees to put in, say, $1000 and is given 1,000,000 points to wager.
None of these are likely to be implemented, they’re only thought experiments but I think that they show that a probability exists though my estimate and yours may differ. Thus the wager.
Rob:
As I said…
If you measure it… You will be gamed.
If Anne wins the lottery twice, one may feel that it is a coincidence. Since the word “feel” is used, in this case, I think, coincidences are the results of expectation and surprise (cause?). Why is someone surprised? It may have something to do with one’s knowledge. For example, the well-known birthday coincidence. (https://en.m.wikipedia.org/wiki/Birthday_problem). If people feel the probability of an event (quantifiable or not, correct or not) seems unlikely, they probably would feel the occurrence the event is a coincidence.
Another example, (Yes, NPR!)
http://www.npr.org/sections/health-shots/2012/07/26/157444649/two-more-nearing-aids-cure-after-bone-marrow-transplants-doctors-say.
No one expected that the HIV virus would disappear after born marrow transplants. What a coincidence!? Feels like it! Now, what’s interesting is that researchers are studying the cause of the virus disappearance.
I adopted two children born in different countries. They have the same birthday (same year). Is it a coincidence? Yes, it feels like it. If I correctly understand what “the nature of cause” Briggs means, I believe I could quantify or approximate the probability of having the same birthday using some demographic statistics. Still, it is quite a coincidence that they have the same birthday.
Feeling is a weird thing.
Rob Ryan: I really didn’t mean to appear to be asking about your history before, during or after the 60’s. My comment was about the 60’s in general. I was always the science nerd from age 3 or 4 on up, which is probably why I was not impressed with the 60’s.
I’m not sure how this post became a discussion of gambling bets. I suppose money is present in virtually everything, so it had to happen. Whether or not something is called a coincidence or not depends mostly on whether we have any way to figure out the causes. A coincidence is an occurrence we did not expect and could not have expected based on our current knowledge of a situation. If gambling, betting on a coincidence would be foolish—one needs some background on the occurrence or one is just throwing money on the table and seeing what comes. Not to say that people don’t do that, of course.
This is one of those posts that leaves me scratching my head trying to work out if I’ve just read something really smart or really dumb. We can’t calculate the probability of a particular coincidence happening because it happened, so has a probability of one. Aren’t you doing yourself out of a job?
Or is the argument that the probability can’t be calculated because there are too many unknowns (like the Drake equation mentioned in the comments) for a meaningful calculation?
“Positing a “guide” is simplifying a complex situation, and that is pleasing in and of itself.”
That’s God right there!
Swordfish trombone
Yes you read something that is describing the reality of probability. If it happened it isn’t probable it’s moved into the realms of reality! That’s funny.
I often have this argument with my Dad. Coincidence, he says, that’s all.
If there wasn’t expectation about how things work out then there would be nothing interesting or amazing when things turn out differently or surprising than what would be predicted or expected. Probability is expectation. Fraught with bias and inexactitude.
Posthole diggers are prescient tools. Pile drivers? not so much.
Swordfish:
Feelings… nothing but feelings…
When I read this, I got the same feeling I get when I try to explain similar things that are completely different but still exactly the same. Pointing directly at the elephant in the room does not illuminate the elephant. For some reason it causes people to fight back against the desire to see the elephant. Point to the secondary indicators of the elephant (vibrations, honking, stomping, tusk holes), and folks become very good at attributing other causes (garbage trucks, Uncle Svens Trans Am, Televisions, and angry feet).
But the white elephant is standing right there…
The gun fired because I pulled the trigger. The bullet hit the target because the barrel was pointed in such a way that they velocity vector of the bullet passed through the target. The barrel didn’t explode because it was made out of a material that is both strong enough and ductile enough to handle the pressure of the explosive charge that sat behind the bullet. If that explosive charge had a few more grains, the bullet may have left the barrel going slightly faster. There is an upper limit to the number of grains I can put behind the bullet. I like to avoid that upper limit by a comfortable margin.
I maximize the probability of the bullet hitting the target by stacking the front and rear sights. Calling that probability though is a misnomer. If I don’t line up the barrel properly the bullet will not hit the target. If I don’t pull the trigger properly (drawing the barrel out of alignment with the target), the bullet won’t hit the target. Probability IS NOT INVOLVED.
Except it is, because maybe the powder used in the bullet is part of a bad batch. I don’t necessarily know that the powder is good. I assume the powder will kill me though so I make sure the barrel never lines up with my body or anyone else’s body. Sometimes people screw up.
I check both ways before crossing a street. It greatly improves the chance that I won’t get hit and become a statistic. I watch the cars around me as I am driving. Sometimes, I back off. I get a feeling. It doesn’t always mean anything, but the integration of the information in front of me causes me to back off. I can attempt to put the feeling into words, but the words are always insufficient. I will never be able to quantify whether the feeling was right or wrong. Backing off made the situation evaporate. I didn’t find a book. I didn’t meet someone I knew. I didn’t have an accident.
It is all the “didn’ts” that don’t get enumerated in the statistics and probabilities because the “didn’ts” are infinite.
We get closer to reality learning how to calculate the chance that we roll at least one 6 in 10 rolls of a die. We know it is not a certainty that a six will come up. We also know that it is certainly unlikely for a 6 to not come up. 5/6^10
But then the physics of a die are still determinant on what comes up and probability is still fictional.
Sheri:
To me it has to become about gambling because that’s the way I reconcile something where multiple independent trials aren’t possible (P[it will rain|current atmospheric conditions] or P[we’ll be awarded this project|circumstances]). So “What is my/your point of indifference on a wager?” is my/your estimate of the probability.
Rob Ryan: Whatever works for you. I tend to think more along the lines that Brad seems to be commenting about, where I don’t actually know what the probability of something happening or not happening, and often I respond to a feeling that something is wrong. Since I back away, I never know the probability. It seems I am using a “feeling” to explain my belief in the probability of a situation. Or a lifetime of experience. Hard to say. (As noted before, I tend to not even think about the probability of an occurance that has happened. If something is unusual or interesting to me, that is enough.)
Sheri:
It’s of practical importance to me and to my firm because we have to estimate revenue for future periods based, in part, on the likelihood of wins in the collection of proposals that we have outstanding. The sum of the estimated revenues times the estimated likelihood of wins for all outstanding proposals is part of what we call our “soft backlog” and it’s essential to have such estimates in order to plan resource deployment, purchases, etc. effectively. When our BD people log a proposal, they are required to an entry an estimate of the “probability” of winning the project. They base it on experience with the potential client, nature of the work and our experience with it, any knowledge of our competitors, etc. Since every one is a one-time event, probability is an awkward concept. Yet it does yield reasonably actionable information. I reconcile the philosophical conundrum with the practical utilization as described.
Rob Ryan: I do not envy you your job. It sounds very fascinating, but I don’t think I could take the pressure. It does explain why you’re interested in probability. Thank you. Now I understand.
(The only time I worked in a field where probability mattered was a very brief period in seismic processing, and I wasn’t in charge of figuring out if the plots indicated oil or not. The rest of time, I’ve have occupations that involved working with the public directly in service industries, etc.)
@Joy: “If it happened it isn’t probable it’s moved into the realms of reality!”
So it was the “it happened so it’s got a probability of one” thing?
@Brad Tittle: “Probability IS NOT INVOLVED. Except it is, because maybe the powder used in the bullet is part of a bad batch.”
So it was the “too many unknowns” thing?
Don’t envy the medicine man, then, for that is exactly what the medical art is about.
Pingback: This Week in Reaction (2016/04/17) - Social Matter