Outcome 3. This sound like good news for you, since all treatment Bobs got better and no placebo Bobs did. Congratulations! The treatment might have caused these cures.
Or again the treatment might have done nothing and the placebo (somehow) blocked the natural cure. It is after all only a guess, with no evidence, the placebo has zero effect on the Bobs. Assuming placebos have no effect, then we might be tempted to say we know with certainty the presence of the treatment was needed for a cure in people exactly like Bob.
But it’s no so, because it could be that something which was not the treatment happened only to the treatment and not placebo Bobs and this something is what caused the cure. For instance, maybe the treatment Bobs coincidentally got zapped by a cancer-curing ray which missed each placebo Bob.
Yet we can rule this out because we assumed both sets of Bobs behaved and were situated identically after receiving their meds and up to the time of measurement. This is a tricky assumption because it means the treatment can only work if it does not cause any changes in behavior, that it only works by shifting stuff inside the treatment Bobs’ bodies, stuff that does not in any way contribute to behavior. Behavior includes thinking. So we exclude cases where the treatment is itself inactive except that it causes the treatment Bobs to think they received the treatment and perhaps enhancing the placebo effect (we are told this happens with real drugs).
Another possibility: the treatment itself did not work, meaning it was not an active agent, but was instead a catalyst which either unlocked some beneficial or blocked some harmful biological process which led to a cure. A catalyst could be a change in behavior (such as removing the desire to listen to NPR thus causing a reduction in stress).
We have ruled these out for a very important reason. If there is one or more behavior difference between the groups, any one of these differences, or some of them in combination, could have been the true cause of the cure. We wouldn’t know, not with the evidence we have, whether it was the treatment or the behavior which caused the cure.
And even if the treatment worked as advertised, we can only say it did so in Bobs. What about not-Bobs? Recall what we said about the closeness of not-Bobs to Bobs: that it is always an arbitrary measure. We just don’t know whether the treatment would have any effect on not-Bobs; not with the evidence we have.
Outcome 4. Bad news for your plans of going public. Your treatment either did no good or it caused harm, or possibly blocked a natural cure. The placebo might not have done anything either, except getting out of the way of a natural cure. Or it might have caused the cure, either by chemical means or by “releasing” the placebo effect. The latter is only possible if the treatment simultaneously held back the same placebo effect. Which of these combinations actually occurred? You get it by now. We can’t know, not with the evidence we have.
Everything said in Outcome 3 is the same here, but with the drugs reversed.
Analysis
We’re almost in a position to figure what all this means to the proposition “My treatment cures cancer of the albondigas.” But first two terms.
A necessary truth says of a proposition that it must be true, that it could not possibly be false. In particular there is no observation which could refute a necessary truth. Truths which are necessary are true even if you don’t want them to be. Examples: it is a necessary truth that “Necessarily true propositions exist”1 and that “1 + 1 = 2”. The latter is so even if you collected two objects which, upon bringing them together, resulted in no objects (think of an electron plus positron). Mathematical propositions have nothing (directly) to say about real objects; they are entirely metaphysical.
It may not be obvious that the proposition “1 + 1 = 2” (or any proposition) is necessarily true. In these cases, the necessary truth might be the end result of a chain of reasoning, as mathematical proofs are, which ultimately rest on indubitable axioms, which are propositions which are true but are true for no other reason than we’re aware of their truth. All the truths in math, logic, and philosophy are like this.
This last sentence is not equivalent to the one which says “All the propositions in math, etc…” because there are many false propositions in these areas. Note too that a necessary truth, as used here, does not mean a formal logical truth, though there will be overlap, because we haven’t any interest whether any particular proposition can be shoe-horned into some schema which somebody might have shown has a counter example (think grue). Each proposition/argument is and must be taken on its own account.
Next time: the second term and complications.
——————————————————————————————————–
1Roger Scruton said something to the effect that people who state the opposite of propositions like this are inviting us not to believe what they say, and that we should take them up on it. (I have to dig this quote up.)
Pingback: On The Evidence From Experiments: Part I | William M. Briggs
Could it be that there are no scientific “truths” except those that are supported by spiritual truths?
Might have, perhaps, maybe, another possibility, possibly, might not have…
Yes, a smile could melt a cold heart, and behavior might cause the cure. What kind of behavior might that be? ^_^
There, we’ve got the final conclusion!
Well, if I were a yellow SpongBob with a particular cancer that turns me green and found out that a treatment had ridden the green caused by the cancer for the identical yellow SpongeBob #1, I would have taken the treatment without any hesitation. Yellow is prettier than green.
On a more serious note, the outcome from a patient (a particular made of identical Bobs) can sometimes result in a great discovery. For example, http://www.npr.org/blogs/health/2012/07/26/157444649/two-more-nearing-aids-cure-after-bone-marrow-transplants-doctors-say
Dear Mr. Briggs, does the quote by R. Scruton apply to you?
Nonesense. In your gedunken, it seems all Bobs do have the cancer. You have not suggested they might be misdiagnosed.
The goal of the treatment was to cure the cancer. In outcome 3 all who received treatment were cured; all who received placebo were not cured. Given the goal, the definition of “worked” is “patient was cured”. The definition of “not worked” is “patient was not cured”.
For those who were given the treatment: The cancer was cured. It worked. Applying the definition of “worked” in this situation, the treatment worked even if you don’t know why it worked. In addition, the placebo did not work. It did not work even if you don’t know why it did not worked.
You are conflating whether we understand why those who took the treatment were cured with whether or not it resulted in a cure.
Possibly, if those doing the experiment had used 3 Bobs giving Bob1 the treatment, Bob2 the placebo and Bob3 nothing at all, we might learn whether doing nothing would have “worked” (i.e. resulted in spontaneous remission or self-curing) or “not worked” (i.e. Bob3 continues to have cancer.)
Obviously, if doing nothing ‘worked’ on Bob3, that treatment would be cheap– and under the rules of your gedunkan, we should just go ahead and do that. Would we know why he was “cured”? No. Maybe it turns out that when refused treatment, all Bobs enter pacts with Satan, who agrees to cure them provided they agree sell their souls but somehow, we the people doing the experiment can’t observe that Bobs do that so we don’t know the cause of the cure.
But that doesn’t mean the treatment didn’t “work”. Treatments that prompt the system (in this case the patient) do do something that results in the positive outcome (i.e. “working” which in this case is being cured) “worked”. They worked because that is the definition of “worked” in this context.
Our ignorance about the underlying causal mechanism that resulted in the positive outcome does not put in doubt the fact that in the case observed, the treatment “worked” i.e. resulted in the desired outcome.
Lucia,
Always a danger to quote partially. “Another possibility: the treatment itself did not work, meaning it was not an active agent, but was instead a catalyst which either unlocked some beneficial or blocked some harmful biological process which led to a cure.” Notice “work” was defined in that sentence, just as it is defined differently elsewhere. Meaning “work” is ambiguous, as you say. So I agree: pays to be careful.
JH,
Applies to everybody and anybody, my dear, even you, if who would state a proposition like this.
Which R. Scruton quote? Or is it all of them.
The reason I keep coming back to the amazing Briggs is because he appears to be attempting to paint with his experience the same frustration I have tried to paint. I think I can reduce it to “Science has incredible uncertainty but the world seems to see it as certain”. I feebly point to the hidden errors thinking that the wiser heads around me already see it. I am very slowly realizing that it is incredibly wrong of me to assume that the experts do see it. At least the expert that are good at getting into the media.
In this particular story, we have the challenge of cures. Does the cure work? I may be inflecting my own experience into our esteemed hosts words, but he seems to allude that even if we have perfect outcomes, that the cure cured in all that got the cure and didn’t cure in all the people who didn’t get the cure, we still have lots of uncertainty.
Practically speaking, if I had cancer of the albondigas and the cure had been shown to work the way it did in outcome 3, I would be inclined to chase it down. This though is a philosophical discussion about extreme examples and their implications hoping that NOT of the examples be more visible.
RE: “Could it be that there are no scientific “truths†except those that are supported by spiritual truths?”
No.
“You can’t know, not with the evidence you have.”
Matt
You are assuming that I did not notice that because I did not quote or engage it. But my view is you are merely defining “work” to mean something it does not mean and consequently, your discussion approaches “equivocation”. I realize you might think you have an ‘out’ because you stated a definition– but that doesn’t really work because the ordinary meaning will always remain lodged in people’s minds. In short: Of course if you define “work” to mean something silly, then one can say that the observing the “cure” or “positive outcome” doesn’t tell us whether the treatment “worked”. But if all you want to say is you don’t know the cause for the cure or positive outcome, say that. Avoid trying to create a tortured definition of ‘worked’ and then saying we don’t know if it “worked”.
That treatment resulted in a positive outcome: By any sane definition of “worked” it “worked”.
Lucia,
It’s astonishing what can get people worked into a bad humor, isn’t it? Even seemingly “dry” philosophical discussions.
Perhaps you mean to say that any definition except the one I gave that “work” means “work”? And that there is indeed a difference between actively working/causing and being a catalyst? Or do you argue tossing the word “catalyst” for the sin of redundancy? I’m up for it.
Here’s another example. Suppose one of the Bobs says he will slap not-Bob, but he will only do so if Cindy is in eyesight. Bob sees Cindy and slaps not-Bob. Did Cindy cause Bob to slap not-Bob? You’d have to say yes, I suppose.
But in the spirit of amity and universal friendship, I’ll accept your definitions. We still have the other provisos, yes?
I have to agree with Lucia, Matt. I don’t know what you’re trying to get at here, other than play word games. How on earth can you honestly contend that the treatment “unlocking a benefitial biological process” does not constitute “working”? If I had cancer of the Ablodongal (or whatever), I’d consider that “working”.
Mike B.,
Very well, like I told Lucia, let’s concede this one small, minor, inessential point (but how about the Cindy example) and push on. And what about those other explanations, eh?
Pretty much everybody can grasp the basic concept that in any single test, however well performed, seemingly obvious cause-effect findings might be erroneous for a host of reasons, coincidental, etc.
Why carry on about the obvious? Or, it ought to be obvious if one doesn’t get bogged down in details before starting at the broad generalizations & working in from there….
Consider the FDA’s official methodology, explained in general at: http://www.fda.gov/Drugs/ResourcesForYou/Consumers/ucm143534.htm
Multiple tests to assess multiple facets of a treatment (e.g. safety/side-effects, effectiveness, etc.). Proper/differing test dose is an obvious one for why something might work in one case & not another…
Is this really so hard a concept to grasp it requires so many words?
Agree with you Ken. Controlled trials are imperfect; there is always the very real chance of biases, imbalances between prognostic factors even after randomization or whatever process you choose to match arms etc. The issues raised by Briggs today too are well known by informed researchers. Now there are many examples of bad trials, with the shortcomings Briggs points out. But, in the real world decisions have to made. She we just use intuition and common sense rather than research evidence? If I had cancer of the Ablodongal, and I saw the results from the trial with outcome three, I would be very happy to take the medicine. Controlled trials are not perfect, but they bring as close to the truth as we need to get to make real decisions. I wonder if Briggs uses modern medicine when he is ill, and what he says to his GP when the GP recommends one drug over another!
As I said, the issues Briggs raise are well known in epidemiology (despite many bad papers published), but don’t take my word for it, page through this textbook http://www.amazon.com/Modern-Epidemiology-Kenneth-J-Rothman/dp/1451190050/ref=pd_bxgy_b_img_y
This entire argument is rather specious. Nothing is easier than killing cancerous cells. Pour gasoline on them and set them on fire. Done.
The problem with curing cancer is NOT killing the cancerous cells, but rather killing the cells while NOT killing other things that are important to keeping the patient alive.
This is important. You can’t just arbitrarily concoct a hypothesis.
Another example of faulty reasoning in your hypothesis is that if the Bobs are truly identical (something which is absolutely impossible) then why conduct tests on multiple Bobs? Why not just one? Is causality a sometimes thing?
Change Bobs to steel widgets. Manufacture widget Bobs to ASTM standards and then test them. If there is a deviation, then it IS possible to determine what caused the deviation. But according to how your argument is shaping up, that would be impossible.
One crucial difference between “widget” Bobs and human “Bobs” is that humans are all significantly different and governed by internal feedbacks which also differ from patient to patient. Humans are never identical, so why would you posit them as such?
I’m amazed at the comments on this thread. You know what the problem is Briggs, you are putting too much effort into being clear. With a little more obfuscation there would be less nit picking.
Matt:
I guess my problem is that my inner industrial engineer is struggling with such an unrealistic example where the experimenter is trying to eliminate all “noise”, which is both impossible (perhaps what you’re trying to demonstrate?) and inadvisable, seeing that the world is a noisy place, and treatments have to work in a noisy world (perhaps another point you’re trying to make).
If a lubricant supplier came to my can-making company and said he had a new formulation that would reduce bodymaker jams, I would never dream of testing it in one plant, on one line, on one bodymaker, during one shift, using just one roll of steel. It has to work system wide, or it’s no use to me.
Pingback: On The Evidence From Experiments: Part IV | William M. Briggs