The Sorrow And The Pity Of P-values

Thanks to the Wrath of Gnon, we have this report: Sorry, wrong number: Statistical benchmark comes under fire, from Phys.org.

Earlier this fall Dr. Scott Solomon presented the results of a huge heart drug study to an audience of fellow cardiologists in Paris.

The results Solomon was describing looked promising: Patients who took the medication had a lower rate of hospitalization and death than patients on a different drug.

Then he showed his audience another number.

“There were some gasps, or ‘Ooohs,'” Solomon, of Harvard’s Brigham and Women’s Hospital, recalled recently. “A lot of people were disappointed.”

One investment analyst reacted by reducing his forecast for peak sales of the drug—by $1 billion.

What happened?

The number that caused the gasps was 0.059. The audience was looking for something under 0.05.

I’ve said it precisely three thousand four hundred and seventy seven times, give or take a million: p-values are magic. They encourage magical thinking. Science results based on p-values become so much mumbo jumbo.

Get a 0.049 and people start doing Harry Potter imitations, zapping spreadsheets with their magic wand fingers. Which would be only just tolerable if it weren’t for the manic giggling that accompanies these dreadful performances. Get a 0.051 and they start sharpening their seppuku blades. Tears ain’t in it: the lamentations of grown men cursed by the Wee P gods is a pitiable sight. You’d sooner gaze upon Cambodian dogs in a cage being readied for the fryer.

Something must be done. And I know what that something is.

Now I have a whole series of papers, linked here, which contain a dozen or two knock-out death-blow killer guts-ripped-out-and-roasted-over-a-spit-until-charred arguments proving the illogicalities of p-values, proving they should never be used, proving that every practical use of them involves a fallacy.

I showed these to many people. From Judea Pearl, no p-value fan, I got a blank stare. From Deborah Mayo, the palsgravess of p-values, I got the cold shoulder. From many others I got “Who’s this Briggs guy?” “Nobody. Ignore him.”

Now this is only right and natural. I do not complain. Approvingly quoting a guy like me can make a person take on my characteristics. Not the handsome courageous strapping, manly 6’2″ 200 pounds of steal characteristics. No. What they end up with is the smear of associating with a racist sexist homophobic transphobic anti-Semitic Islamophobic xenophobic abelist. Once you’re accused of even one of these, clearing yourself and proving your social justice bona fides is like trying to rid yourself of super-glued double-sided sticky tape one-handed.

There are some academics out there, some even quoted in that piece, who know of what I speak. But I dursn’t out them, because I don’t want to get them into any trouble, and because what I want is not additional fame—I reside already at the pinnacle, unbudgeable—what I want is for people to stop using p-values.

So we’re going to have to go about this in another way.

I suggest plagiarisation. Take the arguments in those papers and claim them as your own. You will not hear any whining from me. Use them and show the world how it errs. Bruit them around at academic conferences. Say “Here’s an argument that proves what you’re doing is wrong.” Be haughty. Take credit. I will celebrate your success and shake your hand, and, if you catch me on the right day, I might even buy you a beer.

Why such awe-inspiring laudable praiseworthy award-eligible generosity from such a humble person like myself?

Because even after you whack p-values, you’re still screwed.

You can swap p-values for Bayes factors, if you have a mind, or even confidence or credible intervals. But you will still be talking about fictions. You’ll still be focused on unobservable non-existent parameters inside some ad hoc model. You won’t be right about cause, except by accident, and you won’t be right about the uncertainty in the observable. You’ll still be wrong.

By which I mean, you’ll be right about the wrong thing.

Why not instead give people what they ask for. Some poor sap goes to a statistician and says, “What happens to the Y if I change this X?” And that statistician says, “I don’t know. But what I can tell you is that if repeated your experiment an infinite number of times, exactly the same each time, except each is randomly different, where nobody knows what random means, then the chance of you seeing a test statistic larger (in absolute value) than the one you actually got is 0.051. Please don’t kill yourself.”

Whereas a statistician focused on Reality would say, “There’s a 10% chance of Y.”

Switch to the predictive way.

17 Thoughts

  1. “I suggest plagiarisation.” I heartily approve. Having figured out humans are too stupid to think and go by name-recognition (explaining why it was so easy to convince the stupid that Che Guevara was some kind of god and not a demon), if someone with great name recognition says it, it might take hold. Of course, they will apologize a day later, but the Dems have shown that has ZERO effect and the stupid always believe what they heard first. Announce an alien invasion today and no amount of retraction, apologies, etc will convince people there’s no invasion. Such is living in the wide world of stupid……

    The “p” value is far too easy to fake, meaning it’s here to stay. In a world of science lies, hard facts have no place. And the personal injury demons can get rich off the stupidity of the process, so it’s a win-win. Sure, a few million die here and there, but the worship of science has always had a cost….

  2. So I’m rereading the post from Oct. 1, 2013:

    “There was some talk that if Harkonen had just admitted more uncertainty in the press release—using the verb ‘suggest’ rather than ‘demonstrate’—he might have avoided prosecution.”

    Why isn’t that a thing for Climate ‘Science'”? They’d be doing Climate Science from our prisons!

  3. @John B()

    WHAT! You mean a precedent exists for prosecuting climate activists pretending to be scientists?! That day can’t come soon enough.

  4. Yonason

    The take away, I think, is that the rule applies to corporate scientists and public relation people, not government ‘scientists’, politicians and their lackeys.

  5. I wanted to see a reference to John Brignell when I clicked on the link. I never quite found it.
    I might construe this article as suggesting that we should all take this remarkable drug. I don’t think that our beloved host would bend that way though.
    I tried to tell my Nurse Practitioner that only 1 in 100 people were helped with the use of statins. I hit a wall. Because I said that, he knew I was a stubborn ass. I didn’t figure out till later the gap that existed between “help” as in reduced cholesterol levels and “help” as in actually staved off heart related challenges.

    I forgive him. He just started as an NP. All I needed from him was a signature to let the Scouts BSA** camp let me show up and wander around. I also got his signature on a prescription that makes it so i breathe when I sleep. I really like breathing when I sleep. My apologies, but I praise the CPAP, I despise the statin. Which is going to save my life? I will spend $800 every 4 or 5 years for a new cap without batting an eye. Statins can go sit in the corner and pout. The NP threatened to stop being my practitioner if I didn’t listen to him about statins. We never made up, but I will probably get his signature again. He sees me for 20 minutes a year and thinks the numbers on a lab report mean something about my 5 year prognosis. I really shouldn’t have let him do the labs.

    Whatever this drug is. That it had a p-value next to it implies that it is really close to the statin in importance to keeping me alive.

    Water, food, sanitation, sleep, exercise represent how much of the pie?
    The sun represents how much of the climate? Hmm. What happens when the sun disappears from the equation…. What happens if I stop eating, sleeping, and drinking?

    ** There is nothing wrong with girls in scouting except that boys and girls are different and OMG is there a difference between the chaos that is twenty 11-16 year old boys interacting and the same boys with 3 females mixed in. k

  6. John B() – November 14, 2019 at 8:25 am

    I’m well aware that in the swamp its denizens “win” by design. But if there’s a precedent for it, one could in all fairness theoretically apply that to swamp critters as well. And they’d deserve it, too.

    Best, of course, is that the nonsense be overturned entirely, and never be applied to anyone again. After all, egalitarian application of a ridiculous law, while less unjust, still isn’t the optimal endpoint. (especially when they seem to have distorted the law out of recognition, if not actually making it up – like they have done and are doing against so many others they hate)

    Attempt at make funny = fail. So sorry.

  7. @

    “I tried to tell my Nurse Practitioner that only 1 in 100 people were helped with the use of statins.”

    What? You, a mere patient, telling a highly trained professional what to do? The nerve!
    https://youtu.be/inwfSkSGvQw

    Been there. Done that. And I keep it up, sometimes just to be annoying, and at others probing to see who I can trust.

    And when we find out how dangerous the statins are, it’s even worse than the fact that they don’t work.

  8. @ John B()

    “Maybe you can work something out”

    Remember, I was just agreeing with your…

    “Why isn’t that a thing for Climate ‘Science'”? They’d be doing Climate Science from our prisons!”

    I have no more solution than you, but also wish there were one.

    ALSO – See my post above on Statins for an e.g., of companies selling a compound that may help prevent heart attack in 1 out of 100 (which Lipitor maker Pfizer parleys into 36% prevention), while causing more harm than help in many, including possibly that 1. Talk about misleading. InterMune gets the shaft, but Pfizer skates. That’s the American justice crap-shoot for you. (Note that Pfizer never technically lied, either.)

  9. Yonason (sorry – I saw “anon” earlier – I guess a 4Chan influence)

    I knew you were in agreement – just thought I’d help by sending the DOJ link

    My wife’s on statins, what’s the best “in a nutshell” link on the stuff?

  10. @ L Ron Hubbard alias John B()

    Hi.

    Thanks for the DOJ link.

    By way of introduction, some anecdotal evidence. My ex-wife was nearly paralyzed by Statins: …couldn’t walk, and had to use a scooter. Didn’t recover until quite a long time after stopping Statins. As to myself, I got ill after taking them for 2 days. I subsequently learned about them and low carb diet, and since I’ve been on that my blood numbers and pressure have been better than ever, which is consistent with this material by Dr, Diamond.
    https://www.youtube.com/watch?v=yX1vBA9bLNk

    I’ll see if I can put more together. I’ve been working at optimizing this for several years, with some success, and some frustration. It’s extremely complex, and I don’t think any one person has it right yet, but here’s another link you may find helpful.
    https://doctorstevenpark.com/gominak2018

    Basically, what I’ve learned is that, in addition to low carb & high animal fat (low vegetable fat, except perhaps for coconut oil), there’s also the issue of eating for the health of one’s gut microbes, as well as what Dr. Gominak discusses in the last link, above.

    One size doesn’t fit all, but I hope the material I’m including is helpful, and I do wish you best success!

    DISCLAIMER – I’m not a physician, and even if I were I couldn’t diagnose anything online. I’m just providing information that has helped me, and may be a good starting point for others trying to get a reliable compass reading on how to proceed. Best would be to find a patient friendly doc who’s open to new ideas by competent researchers. There are plenty out there who get as much wrong as they do right, so use good judgment and proceed with caution.

  11. Justin,

    To prove is used in the mathematical, logical sense above. Meaning now that proofs are available, it is futile to work for false propositions.

    To solve all problems requires the predictive approach. See the class in the books page. Work through the examples.

  12. @Yonason
    “…Basically, everything we’ve been told about a proper diet appears to have been, and continues to be, wrong…”

    The fact that people were told something doesn’t mean they followed it. That famous food pyramid that supposedly caused all ills was just a suggestion, not a mandate. Just look up the US sales of meat/fat/dairy/barbecue gadgets/everything during the ‘reign’ of, oh so cruel pyramid, and then see if you can blame the oatmeal :-)))

    The lack of evidence in one direction is not the proof the effect is in the other direction. It just means there is no sufficient evidence in the direction X. To claim the opposite, you must present the data in the opposite direction.

  13. @ Kalif

    Thank you for validating what I wrote.
    “…Basically, everything we’ve been told about a proper diet appears to have been, and continues to be, wrong…”

    Just because we don’t have to follow those guidelines doesn’t mean we haven’t been lied to, or that most people don’t follow them, to their detriment.

    “Just look up the US sales of meat/fat/dairy/barbecue gadgets/everything during the ‘reign’ of, oh so cruel pyramid,…”

    While I’m at that, why don’t you just look up the volume of fast food sales in that time, as well as the junk food those folks eat with their bbq. Also, how many know not to use high sugar bbq sauce on their healthy ribs? …or not to accompany them with lots of chips, potato salad, high sugar soda and bread?

    Most people do believe the lies they’ve been told about nutrition, as attested to by the number of fat and unhealthy people trying to improve their health by eating what’s bad for them, because they don’t know any better.

    And for those who know better, where will they get all the food that’s good? Groceries only carry what their distributors allow, and the distributors only allow what sells, which is for the most part low fat high carb junk. And what sells is what people are told is good for them.

    So, thanks for just giving an anecdotal alternative to their lies. But please note that it doesn’t prove they haven’t lied, or that that their deceit hasn’t had a disastrous effect. And it isn’t just America. The problem is worldwide.
    https://youtu.be/ceFyF9px20Y?t=346

    You don’t need a “mandate,” if enough people fall for the propaganda. And choices are appropriately limited (I can’t remember the last time I saw just plain yogurt in the grocery), even those who know better will be hard pressed to find real food.

    I think we may agree more than not, but from your tone it suggests you might also be a recovering victim? I’m pretty ticked off at them, myself. If that’s where you’re coming from, I can relate.

Leave a Reply

Your email address will not be published. Required fields are marked *