Skip to content
August 16, 2008 | 5 Comments

Wall Street Journal: Better than a statistics textbook.

On Thursday 14 August, the Wall Street Journal had two excellent articles, which expertly described the statistics and uncertainty of their topics. Several readers have wrote in asking for an analysis of these articles.

1. The first was by Thomas M. Burton: “New Therapy for Sepsis Infections Raises Hope but Many Questions.” Sepsis is a nasty disease that often is the result of other trauma or infection, and is often deadly. Curing it is difficult; usually a third or more of the patients who contract it die. So when a study published by Emanuel Rivers, a doc in the emergency medicine department at Henry Ford Hospital in Detroit, appeared with a new therapy that seemed to have a higher cure rate than traditional therapy, doctors were excited. (Incidentally, I made my first appearance at that same hospital.)

But a few were skeptical. The “questions” in Burton’s title hinge on the statistical methods used in the journal article—which was published in the most prestigious medical journal. Turns out that Rivers did not use all the patients he had entered into his study in the actual statistical analysis. “Statisticians were especially concerned when they noticed that a relatively high proportion of the other 25 — those not included in the final analysis — were either conventional-therapy patients who survived or patients on aggressive therapy who died.”

Why were these patients left out of the analysis? Well, doctor judgment: these 25 patients were not evaluated, at the time, to be as sick, so they were left out. In medical statistics, there is a concept called intent-to-treat, and it means that you must analyze the data putting all patients into the groups that you first put them in no matter what. This procedure is meant to guard against the experimenter effect, which is the boost in results got by the researcher when he, consciously or not, fiddles the patient rolls to get his desired result.

Why wasn’t the original paper criticized on these grounds? A peer-reviewed paper, I should emphasize. Are we saying it is possible that published research could be wrong?

2. Thanks to reader Gabe Thornhill for pointing out another excellent piece by Keith J. Winstein for his article “Boston Scientific Stent Study Flawed.” You might remember Mr Winstein was the only reporter to get the story about boys’ and girl’s mathematical abilities correct.

The story is that Boston Scientific (BS) introduced a new stent, which is an artificial pipe support that is stuck in blood vessels to keep them from being choked off by gunk, called the Taxsus Liberte. BS did the proper study to show the stent worked, but analyzed their data in a peculiar way.

Readers of this blog might remember Chapter 14 of my book: How to Cheat. In classical statistics, an excellent way to cheat, and a method you can almost always get away with, is to change your test statistic so you get the p-value you desire. For any set of data there are dozens of test statistics from which to choose. Each of them will give you a different p-value. For no good reason, the p-value has to be less than 0.05 for a success. So what you do is keep computing different statistics until you find the one which gives you the lowest p-value.

This trick nearly always works, too. It carries a double-bang, because not only can you nearly always find a publishable p-value, nobody can ever remember the actual definition of a p-value. Smaller p-values are usually accompanied with the claim that the results “stronger” or “more significant”. False, of course, but since everybody says so you will be in good company.

Actually, Mr Winstein has two definitions in his piece that aren’t quite right. The first:

Using a standard probability measure known as the “p-value”, it said that there was less than a 5% chance that is finding was wrong


[S]cience traditionally requires 95% certainty that a study proved its premise.

Pay attention. Here is the actual definition of a p-value, adapted to the stent study. For the two sets of data, one for the BS stent, one for another stent, posit a probability distribution which describes your uncertainty in the measures resulting from using these stents. These probability distributions have parameters, which are unknown unobservable numbers that are needed to fully specify the probability distributions.

Now, ignore some of these parameters, and concentrate of just one from each distribution (one for the BS stent, one for the other) and then say that one parameter for the BS stent is exactly equal to the parameter for the other stent. Then calculate a statistic. From above, we know we have the choice of several—and Mr Winstein has an excellent graph showing some possible choices. Here comes the p-value. It is the probability that, if you repeated the same experiment an infinite number of times, that you would see a statistic as larger or larger than the one you actually got given those two parameters you picked were exactly equal.

Make sense? Or is it confusing? I’d say the later. One thing you cannot say is that, for example with a p-value of 0.04, there is a 96% chance that the two stents are the same (BS sought to say their stent was equivalent to its competitor’s). Nor can you say there is a 4% chance you are wrong. All you can say is that there is a 4% chance that if you repeated the experiment many times, each time calculating the same statistic, than one of those statistics would be larger than the one you got (again, given the two parameters are exactly equal).

Whew. A lot of work to get to this point, I agree. But this is it, because nobody—even professorial classical statisticians, which we’ll see in a moment—can actually remember this definition. This is what makes it possible to cheat.

Boston Scientific used something called a Wald test, which is way to approximate the p-value, because often p-values cannot be computed exactly. It is well known, however, that this method gives poor approximations and often gives p-values that are smaller than they should be. However, all this is conditional on the test statistic used being correct, and on the probability distributions chosen for the observable data being correct, and on the parameters you ignored to set up the p-value being ignorable, always huge assumptions. This is why it is strange to see, near the very end of the article, a professor of statistics say that the imperfect Wald method is commonly used but that

Most statisticians would accept this approximation. But since this was right on the border [meaning the p-value was barely under the magic number], greater scrutiny reveals that the true, the real, p-value was slightly more than 5%

The true, the real? The problem here is there is no true or real p-value. Each of the p-values computed by using the different statistics is the true, real one. This is one of the main problems with classical statistics. Another is the persnickety insistence on exactly 0.05 as the cutoff. Got a p-value of 0.050000001? Too bad, chum. Have a 0.0499999999 instead? Success! It’s silly.

Obviously, misinterpreting p-values is a big problem. But ignore that. Winstein and the WSJ have done a wonderful job summarizing a difficult topic. Are you ready for this? They actually got the data and recomputed the statistical tests themselves! This is real science reporting. It must have taken them a lot of effort. If only more journalists would put in half as much work as Mr Winstein, we’d have eighty percent less junk being reported as “news.” In short, read Winstein’s article. He has quotes from Larry Brown, one of the top theoretical statisticians alive, and comments from officials at the FDA about why these kinds of studies are accepted or not.

August 13, 2008 | 20 Comments

Extremely fit have larger carbon footprints than do couch potatoes: scientific study

The following is a scientific study:

Extremely fit have larger carbon footprints than do couch potatoes


Dr H. Harrister, BS, MS, PhD, OBWAG


Ever since the Supreme Court has (wisely) ruled that carbon dioxide is a pollutant, the number of people who have been made exceptionally nervous has increased nearly exponentially (exponentially is a mathematical term). It is up to science to discover ways of reducing this vile gas, to root out its sources, and suggest interesting ways of scientifically punishing environmental malefactors.

Thus, it follows that it is the duty of every single person to reduce their carbon footprint in every conceivable way, and to do so in the shortest amount of time humanely possible before disaster strikes.

The purpose of this scientific paper, therefore, is to bring to mind a particular activity that had previously been assumed virtuous but under the unerring eye of science has proved to be pernicious. That activity is exercise.


We accept, as we must accept after the highest court in the land said so, that carbon dioxide (CO2) is a pollutant. Many common people—unfortunately, those without sufficient educations—are not aware that each time they exhale they are adding to the excessive burden of CO2 in our precious atmosphere. This is because humans inhale oxygen into their lungs with each breath—oxygen that is forever depleted from the air, because that oxygen is processed into CO2.

Each exhaled breath contains a certain amount of CO2. The exact amount is a function of total lung capacity, residual lung volume, and vital capacity, as well as related to other measures of pulmonary function. In this paper, we make the reasonable approximation that variations in the amount of CO2 exhaled per breath are negligible, that is, the amount of exhaled CO2 is fixed for all people, except in two ways which we note below. It is important to recognize, however, that most of this human CO2 outgassing takes place in the troposphere (troposphere is a meteorological term), in the boundary layer. Shockingly, this is also where most people live.

The main source of variation of exhaled pollution (CO2), the intra-person variability, is due to respiration rate (RR). Higher RRs mean more breaths per hour and therefore more pollution added to the atmosphere, and therefore the more likely we are to experience runaway greenhouse effects. Therefore, people with lower RRs have smaller carbon footprints (or lungprints as we should properly say) than those people with higher RRs.

Let LS = lung size. The average resting breath for an adult North American human male contains about LSrest = 325 ml of air. This expands to twice this under aerobic stress (exercise), to about LSstress = 650 ml of air. There will be variations in these values, of course, but we scientifically dispense with this uncertainty.

Because all tissue in the human body has to be oxygenated lest it turn sour, it implies that the fatter (larger) a person is the more oxygen they consume. This is mitigated somewhat because those at that highest scale of fatness, the couch potatoes, tend to engage in very little movement. And since movement means using muscles, and muscles rely on oxygen as part of their fuel, less movement means less oxygen usage (or utilization, if you prefer).

Since the humans under our consideration are assigned to have equal lung volume and other pulmonary functions, more oxygen usage is equivalent to higher RRs. As explained above, more oxygen usage is directly proportional to more CO2 creation. To be explicit: higher RRs give rise to higher amounts of CO2 degassing to the atmosphere. Not all of the air inhaled in processed into CO2. Let the fraction of each breath that is converted to pollutants be fco2.

The RR of couch potatoes is likely to be even and vary little throughout any 24-hour period because of their habit of remaining as stationary as possible. This rate will be slightly higher than thin, non exercising people, and higher than exercising people at rest because couch potatoes’ larger bodies need more oxygen. Studies have shown that this rate is RRCP = 14 breaths per minute. Thus, we present equation 1, the amount of pollutant added (PA) to the atmosphere per day for this class of individual:

Eq. 1    PACP = RRCP x LSrest x fco2 x 60 minutes hours-1 x 24 hours day-1

This equation is unlikely to be wrong because it has been written in the proper mathematical format. It also uses scientific notation, and your author has a PhD.

Equation 1 can be contrasted to a similar equation for those who engage in excessive exercise, which is defined as 22 hours of calm followed by two hours (not necessarily contiguous) of frenetic bursts of ludicrous activity. The RR for exercisers at rest is known to be RREx:rest = 12 breaths per minute. While engaged in stressful activities this rises to RREx:stress = 30 breaths per minute. Thus, the daily amount of PA for exercisers is

Eq. 2    PAEx = RREx:rest x LSrest x fco2 x 60 minutes hours-1 x 22 hours day-1 + RREx:stress x LSstress x fco2 x 120 minutes

Substituting the precise values into equations 1 and 2 gives

Eq. 1b    PACP = 14 x 325 x fco2 x 60 x 24 hours

which equals

Eq. 1c    PACP = 6552000 x fco2


Eq. 2b    PAEx = 12 x 325 x fco2 x 60 minutes hours-1 x 22 hours day-1 + 30 x 650 x fco2 x 120 minutes

which equals

Eq. 2c    PAEx = 5148000 x fco2 + 2340000 x fco2 = 7488000 x fco2

which makes the ratio of PAEx to PACP equal to

Eq. 3    PAEx/PACP = 1.142857

Conveniently, we do not need to know the exact value of fco2, as it cancels in the equation.


Through the strictest scientific procedures, the same as those used in a myriad of studies of this type, we have conclusively proven that those people who exercise have a carbon foot (or lung) print 14.29% higher than those who, altruistically it turns out, lie around on the couch. Future studies will examine the additional benefits of progressing to a drunken stupor, a state in which minimal oxygen usage is obtained.

It might be argued that CPs eat more, thus they increase the amount of CO2 added because of their increased food intake. But this argument is specious when contrasted with the habits of the very fit, defined as people who typically motor to Whole Foods in their SUVs to buy only “organic” comestibles, the creation of which produces far more CO2 than does, say, manufacturing bags of Cheesy Puffs, the chosen snack food of most couch potatoes. Plus, of course, those who exercise more actual consume a greater amount of food than do lazy slobs (this is scientifically true).

The policy implications of this study are obvious: people must be discouraged immediately from exercising. They should be taught the immorality of it, how their narcissistic habits unnecessarily add to carbon burden of the atmosphere, thus endangering the fragile climate system, and therefore the future for our children.

If we can each stop just one jogger from donning his multi-colored, garish shorts and trotting through the neighborhood, we will have done the Earth a tremendous service.

Editor’s note: please help disseminate this scientific study as widely as possible. Inform journalists of the perilous and frightening nature of its conclusions. The time to act is now.

August 11, 2008 | 3 Comments

Surely, you wouldn’t go and see An American Carol?

Yes, I would. And don’t call me Shirley.

David Zucker, the guy who brought us Airplane! and Naked Gun, will be back on October 3rd with An American Carol. Stephen Hayes of The Weekly Standard has a long interview with Zucker, in which he discloses there are actually—shhhh—conservatives in Hollywood.
American Carol
The movie is based around the experiences of fat slob documentarian Michael Malone, who’s last offering was Die You American Pigs. Malone, who has recently joined the left wing because he has sympathy with their goal of banning the 4th of July, is visited by three ghosts of America’s past: George Washington, George S. Patton, and John F. Kennedy.

According to Hayes, “Dennis Hopper makes an appearance as a judge who defends his courthouse by gunning down ACLU lawyers trying to take down the Ten Commandments.” In the movie, there is a rally at Columbia University, where students chant:

“Peace Now, We Don’t Care How!” Some of their protest signs are ones you’d find at any antiwar rally. Some are not. “9/11 Was an Inside Job,” “Kick Army Recruiters Off Campus!” “End Violence–War Is Not the Answer!” “End Disease–Medicine Is Not the Answer!” “It’s Too Dark Outside, The Sun Is Not the Answer!” “Overpopulation–Gay Marriage Is the Answer!”

Apparently, Rosie O’Donnell has claimed that “radical Christianity is just as threatening as radical Islam in a country like America where we have a separation of church and state,” so they incorporate a scene where fat slob actress Rosie O’Connell introduces here new movie The Truth About Radical Christians. A clip from that movies shows

a pair of priests walking through an airport–as seen from pre-hijacking surveillance video–before boarding the airplane. Once onboard, they storm the cockpit using crucifixes as their weapon of choice. Next the documentary looks at the growing phenomenon of nuns as suicide bombers, seeking 72 virgins in heaven. A dramatization shows two nuns, strapped with explosives, board a bus to the cries of the other passengers. “Oh, no! Not the Christians!” O’Connell’s work ends with a warning about new threats and the particular menace of the “Episcopal suppository bomber.”

In answer to the idiotic slogan “War is not the answer” there is a scene where

David Alan Grier plays a slave in a scene designed to show Malone what might have happened if the United States had not fought the Civil War. As Patton explains to a dumbfounded Malone that the plantation they are visiting is his own, Grier thanks the documentarian for being such a humane owner. As they leave, another slave, played by Gary Coleman, finishes polishing a car and yells “Hey, Barack!” before tossing the sponge to someone off-camera.

Over at somebody claiming to have been an extra in the movie describes a scene where Michael Malone “accidentally rips, burns and then stomps on an American flag, surrounded by angry military soldiers.”

Zucker can be a funny guy. If you’ve never seen the original Naked Gun television series, well, I can’t help but feel very sorry for you. Who has seen it can ever forget where Frank Drebbin was invited into the Japanese Garden…to find several well dressed Nihonjin standing in enormous pots. Or the line uttered by the nerdy Mr Science-CSI guy, “Come back next week, Katie, and I’ll explain to you why cows look forward to getting milked.” Or from the movie: “It’s true what they say: Cops and women don’t mix. It’s like eating a spoonful of Drano. Sure, it’ll clean you out, but it’ll leave you hollow inside.” And “Mrs. Nordberg, I think we can save your husband’s arm. Where would you like it sent?”

Who doesn’t know by heart whole snatches of Airplane!: “Joey, do you like movies about gladiators?”, “Looks like I picked the wrong week to quit sniffing glue”, “Nervous? Yes. First time? No, I’ve been nervous lots of times“, Oh, how I wept!

Even better, the blog sites of the Sputtering Left are already abuzz with disdain, e.g. this one or this.

Zucker isn’t always hilarious. For example, he also made the last couple of Scary Movies. And he has been making commercials picking on Iran. Here’s one of them, which is sort of cute, but mostly goofy, resembling a college “film” project:

Still, I have hope.

“Surely you can’t be serious.”

“I am serious… and don’t call me Shirley.”

August 8, 2008 | 12 Comments

The B.S. octopus

Jonathan Bate, of Standpoint, recently wrote an essay “The wrong idea of a university”:

It used to work like this. Dr Bloggs, the brilliant scholar who had solved the problem of the variant quartos of Christopher Marlowe’s Dr Faustus, was one of the most boring teachers on God’s Earth. Mr Nobbs, who never got around to finishing his PhD on the image of the sea in English Literature, let alone publishing any academic articles, was an awe-inspiring teacher: he had read everything and could instil in his students a passion for the subject that would stay with them all their lives. All the Head of the English Department had to do was give Nobbs a heavy teaching load, which delighted both him and the students, and Bloggs a light one, which also delighted the students and gave him more time alone with his textual collations. The department was a happy place.

But then along came the RAE. Bloggs’s work was just the stuff to bring the department the money that came with a five-star rating. Nobbs, to the distress of the students, was pensioned off as “non-returnable”. The next generation of academics learnt the lesson. They finished their PhDs and started up new journals in which to get their work published. They developed more and more specialised areas of expertise. (The RAE is Research Assessment Exercise, an attempt to quantify academic quality in England.)

In proof that university politics have not changed in the 100 years since William James wrote his now-famous essay “The PhD Octopus“, there is this quote:

Some years ago, we had at our Harvard Graduate School a very brilliant student of Philosophy, who, after leaving us and supporting himself by literary labor for three years, received an appointment to teach English Literature at a sister-institution of learning. The governors of this institution, however, had no sooner communicated the appointment than they made the awful discovery that they had enrolled upon their staff a person who was unprovided with the Ph.D. degree. The man in question had been satisfied to work at Philosophy for her own sweet (or bitter) sake, and had disdained to consider that an academic bauble should be his reward.

His appointment had thus been made under a misunderstanding. He was not the proper man; and there was nothing to do but inform him of the fact. It was notified to him by his new President that his appointment must be revoked, or that a Harvard doctor’s degree must forthwith be procured. (Be sure to read the rest of this essay.)

It will probably always be thus at universities: no PhD, no respect. Good thing nobody told Einstein, however, who had his miracle year long before he actually had the credentials to do so. Good thing, too, that nobody told the editors of the journals where he submitted his papers, nor did anyone notify readers of those journals of the lack of Einstein’s bona fides.

Actually, in areas like physics, chemistry, math, and so on, lack of credentials is still not a barrier for authors to gain consideration. Anybody is free to submit a paper, and due to the blinded, or ever double-blinded, refereeing policies at these journals, the paper will get something like a fair hearing. If the paper is published, still nobody will know that the person who wrote it lacks certification (unless, of course, somebody knows the person).

Incidentally, medicine is an exception to this rule. Every paper in medical journals list the authors’ credentials after their names. Papers are festooned with MDs, MPHs, DDS, EDDs, DOs, PhDs, and every other possible combination of letters. I have even seen some with the lowly BS. I have been unable to find any other field—I have looked in English, sociology, history, and so on—that maintains this silly practice. The argument of Appeal to Authority remains strong in medicine. Too, these authors like to see their degrees displayed prominently. Well, who didn’t know that physicians have large egos? (When medical co-authors ask me for my letters, I tell them “HS”, which I almost got away with once until a journal editor caught it. HS = High School.)

Although anybody is free to submit papers regardless of their formal education, few to none will actually work for a university as a professor without the actual blessing. This is so well know as to be unexceptional. It is not usually the professors in the department who employ a teacher sans PhD that care. I have known two exceptional men who were welcomed, more than welcomed, in their departments in spite of their lack of letters. It is usually the administration who insist. They see the spreadsheet before them with a blank column by the man’s name and they balk, unable and unwilling to grant the title “professor”, regardless of the teacher’s ability, unless that column can be filled in. But, however, this has been the way of the world for at least the last century.

What is more pernicious, is that the desire for credentials has spread to nearly every area in society. People used to be able to get jobs with nothing more than high school educations. While it’s probably true that the content of a high school education nowadays is less than it used to be, it is still sufficient to allow somebody to, for example, be an assistant manager of Jamba Juice. That company, we learn from, is soliciting applications for the position, advising applicants that a “Bachelor’s” degree is preferred. Do you really need a BS or BA to learn how to prepare and pour a smoothie? Like many job postings, this one merely says “degree wanted” and is indifferent to the field of study. Proof that the “degree” is not a necessity.

It is true that, generally, more knowledge is better than less, and that colleges attempt to give students more. But it is not clear that what colleges attempt to teach is the sort of knowledge that is useful to being a manager at Jamba Juice. Nor is it even close to true that the only or best or ideal way to gain knowledge is by attending college, especially to acquire job-specific knowledge.

A typical answer from students about why they are attending college is “to get a degree.” Note carefully that this is not the same as “to learn all about biology” or physics, or English literature, or whatever. Or to learn how to be a better citizen or lead an examined life or become, as the hackneyed phrase has it, “well rounded.” Some will say they are at college to “get an education”, which is synonymous with “get a degree”, because, as I hope you know, education is not equivalent to knowledge.

Getting a degree, and not necessarily gaining knowledge, is a rational thing for students to do. This is because they know, as we have just seen, that employers explicitly require “degrees.” It is true that some employers also require field-specific knowledge, but this is not stressed strongly or at all for entry-level candidates. Businesses will teach people what they need to know to do their jobs once they get there. Except in certain highly technical areas, where some competency with computers and an extensive numeracy are expected. Students can gain these skills in college, but they could just as easily have attained them in a trade school in half the time at half the cost. But more and more, non-technical, non-complex jobs require Bachelor’s degrees, mainly because, well, because businesses have convinced themselves “degrees” are needed.

On the whole, employers—and civilians, too—view a Bachelor’s “degree” as something magical, imbuing its holder with special powers—but not necessarily special knowledge. You’ll have heard stories of some person, wholly competent in her job, who is paid a low salary because she has not yet attained her “degree.” Once she comes by it, she is immediately given a raise, because people with “degrees” of course rate a higher salary. Or you might know of another person who everybody agrees should be promoted and given extra responsibility, but, sorry, no degree, so the promotion cannot be given. Everybody is heartily sorry for it, of course, but what can they do? It’s a degree we’re talking about, after all.

It is true that knowing whether a person has advanced educational credentials helps predicts whether they will be to accomplish some task. But it is not wholly predictive, and not even mostly predictive. People are fooling themselves by weighing the evidence of letters after a name too strongly. It has also been observed that the more education a person has the less likely that person will admit a mistake or ignorance on any subject.

A host of “experts” have exploded into public life over the past twenty or thirty years. There is a credentialed expert for any subject imaginable, ready to be drug out and placed onto television to say why this or that is so. Businesses regularly host expensive consultants with “MBAs” from “good schools” to tell them how to do their jobs. Government routinely taps academia to justify or give blessing to what it wants to do. The letters after the name of the expert are enough for most people to accept what is uttered unquestioningly. Having somebody make decisions for you is also comforting and easy. Objecting too what an expert says, unless the dissident is at least as credentialed as the expert, is seen as distasteful, and even in some cases immoral (see two posts back). There is much more to say on this subject, but for now we can note that the old rule that the best argument wins has been lost.

I don’t think anything can stop or reverse this trend of the increasing hunger for degrees. Jobs that used to require nothing except intelligence now require a Bachelor’s. Some of them prefer a Master’s. Soon, a PhD will be the minimum. But we should all remember the words of Frank Mundus, the famous (uncredentialed) shark hunter whose life partly formed the basis for the fisherman Quint character in the book and movie Jaws. Some “PhDs” once took exception to a belief he espoused about a certain behavior in sharks. Mundus was proved right and the “experts” wrong. In reply he said, “A PhD don’t mean shit.”

Amen, brother.

By the way, in case you were nervous, your author (me) has a PhD in statistics from Cornell, so you know I know what I’m talking about.