William M. Briggs, Statistician

First listen to the appalling Chuck Schumer responding to a question about the proposed Fairness Doctrine (link from Unfair Doctrine):

Let’s summarize. He said:

• I think we should all try to be fair and balanced, don’t you?
• [Radio broadcasts]: It’s not like printing a broadside…Do you think we should allow people to put pornography on the air? Absolutely not.
• The very same people who don’t want the Fairness Doctrine, want the FCC to limit pornography on the air.
• But you can’t say “Government, Hands off” in one area to a commercial enterprise, “But you’re allowed to intervene in another.” That’s not consistent.

Schumer is treasure trove to people like me who are always on the lookout for examples of appallingly bad reasoning to use for teaching students logic. Almost any Schumer speech can be milked for at least one lesson—you could probably get half a semester from this bare minute.

Now, nobody knows what any new Fairness Doctrine might be since it is now in its “trial balloon” phase. But we can look to an earlier, abandoned incarnation of it for some clues. We can also glean hints from Schumer’s words.

Schumer thinks we should try to be “Fair & Balanced.” A fine thing, but not something that can be mandated. This is not a question of opinion or morality. For example, supposed on some matter the truth is A (where this is some argument or proposition about a decision we have to make). I set up a newspaper to tout A. Another group, unhappy with the reality of A, says “B is better because it shows we care.” But since A is true, it is absurd for me to publish anything else. It is even more absurd for the government to threaten me with criminal liability for my refusal to explain the merits of B.

Of course, we don’t often know the truth of some thing, but we can make a rational guess. It might be, conditional on some evidence, that A is nearly true, or more than likely true, and that every other alternative to A is less likely to be true. Again, it is absurd for me to publish anything else, and equally or more absurd for the government to intervene.

Can the government ban certain opinions from being published? The answer is yes. In certain circumstances, it is rational to proscribe behavior. Some examples: calls for armed insurrection, pleas for murder or other crimes, for sedition and so on. It is not only right the government should ban these, but it is its duty to do so. The exact limits of opinion that can and should be banned are, of course, unknown, and will be, in some cases, flexibly defined. But in no case does it make sense for the government to say, “Ok, make your plea for murdering the president, but you also have to allow Mr X 5 minutes to offer his counter opinion.” The ludicrousness of any such an argument is apparent. In short, either an idea is banned or it is allowable (a trivial tautology, but one that bears mentioning).

It does not follow that because the vast majority of Americans want to ban or limit pornography from being broadcast, that the government can ban, limit, or regulate any other opinion. Whether or not it is right to ban or limit certain opinions, or what constitutes the definition of those opinions, it does not follow—it is idiotic to propose—that the government should allow airing of the controversial opinion but then require the broadcaster provide time for counter opinions. If that were the case, then we could have a station air Deep Throat followed by a plea for proper dental hygiene.

Proper dental hygiene? Why not “The evils of pornography”? Why not, indeed. Now comes the easiest refutation of any implementation of a Fairness Doctrine. Suppose I say “A is true!” The government wants to say, “You may say A is true, but I mandate that you allow fair time for opponents of A. You shall also bear the expense of this.” Who are the legitimate opponents of A? Those that say B? C?, D, E, F…?

This is the meat of it, friends. Pay attention. In order to enforce any “Fairness” Doctrine, the government will be forced to define the opposite of A. Because, for any matter that is uncertain, there are an infinite or certainly an enormously huge number of alternatives to A. You cannot, in finite time, broadcast every alternative to A even if you wanted to. The only way to mandate broadcasting alternatives to A is by the government dictating—and dictating requires a dictator—what those alternatives are.

For example, in the earlier incarnation of this naked power grab, a prominent person who was “attacked” on the air was to be allowed time to offer his defense. What defines an “attack”? Does any negative opinion about the Great Leader in power constitute an “attack”? The Great Leader proposes a tax increase, and a broadcaster says, “This will negatively effect credit and so make it more difficult to get home loans.” Is this an “attack”? Who can say? The government wants to say. In fact, it must say.

There is no way around this fact: the government must get into the business of defining what an “attack” is, what are its limits, and so on. There is no alternative if you require a Fairness Doctrine. There must come into an existence an office to administer Fairness (I propose “Ministry of Truth”).

Of course, many, like Schumer, would like nothing better than to be in the business of defining what are the limits of opinion on political matters. The reason for this is obvious as it is odious.

Congress shall make no law respecting an establishment of religion, or prohibiting the free exercise thereof; or abridging the freedom of speech, or of the press; or the right of the people peaceably to assemble, and to petition the Government for a redress of grievances.

It is impossible for any Fairness Doctrine to be consonant with those words. It is not a debatable point: it is logically impossible. Unless, as Schumer and other advocates of the “living constitution” want to do, you change the meaning of the plain-English words “Congress shall make no law prohibiting the free exercise of the press.” They must interpret this to mean “Congress shall make no law prohibiting the free exercise of the press unless that law allows us to respond to people who hurt our feelings or otherwise pick on us, or that the speech printed or broadcast is hateful.” This is so absurd that I am shocked that anybody but an academic could ever think it.

Well, that’s enough. I’m already sick of this. There are no subtleties involved in this argument, not anywhere. To see these power-hungry politicians licking their chops over the possibilities due to them because of their recent electoral victory is truly frightening.

Sigh. I didn’t even get to the obvious logical absurdity in Schumer’s phrase “But you can’t say…” I’ll leave that for homework.

Be sure to read through page 2, where you will read:

Conservatives should never say to voters, “We can lower your taxes.” Conservatives should say to voters, “You can raise spending. You, the electorate, can, if you choose, have an infinite number of elaborate and expensive government programs. But we, the government, will have to pay for those programs. We have three ways to pay.

“We can inflate the currency, destroying your ability to plan for the future, wrecking the nation’s culture of thrift and common sense, and giving free rein to scallywags to borrow money for worthless scams and pay it back 10 cents on the dollar.

“We can raise taxes. If the taxes are levied across the board, money will be taken from everyone’s pocket, the economy will stagnate, and the poorest and least advantaged will be harmed the most. If the taxes are levied only on the wealthy, money will be taken from wealthy people’s pockets, hampering their capacity to make loans and investments, the economy will stagnate, and the poorest and the least advantaged will be harmed the most.

“And we can borrow, building up a massive national debt. This will cause all of the above things to happen plus it will fund Red Chinese nuclear submarines that will be popping up in San Francisco Bay to get some decent Szechwan take-out.”

Thanks to Arts & Letters Daily once more.

Michael Crichton, as you will have heard by now, is dead. Unfortunately.

The Wall Street Journal today reprinted an excerpt of a speech Crichton gave called “Aliens Cause Global Warming.” Regular readers of this blog will know Crichton’s opinion on the certainty of man-made catastrophic climate change. Just a reminder (from his speech):

No longer are [climate] models judged by how well they reproduce data from the real world — increasingly, models provide the data. As if they were themselves a reality. And indeed they are, when we are projecting forward. There can be no observational data about the year 2100. There are only model runs.

This fascination with computer models is something I understand very well. Richard Feynman called it a disease. I fear he is right. Because only if you spend a lot of time looking at a computer screen can you arrive at the complex point where the global warming debate now stands.

Nobody believes a weather prediction twelve hours ahead. Now we’re asked to believe a prediction that goes out 100 years into the future? And make financial investments based on that prediction? Has everybody lost their minds?

To explain why he was flummoxed, Crichton first made a point about SETI, the Search for Extraterrestrial Intelligence. A lot of people in that field make reference to the Drake Equation, originated by SETI big cheese Frank Drake. That equation is

• N = R ^* x f_p x n_e x f_l x f_i x f X L

.

We want to solve for N, which is the number of civilizations in our galaxy with which intelligent communication is possible. N depends on the rate of star formation R ^*, the fraction f_p of those stars that have planets, and all those other things you can look up.

Crichton says:

This serious-looking equation gave SETI a serious footing as a legitimate intellectual inquiry. The problem, of course, is that none of the terms can be known, and most cannot even be estimated. The only way to work the equation is to fill in with guesses. And guesses — just so we’re clear — are merely expressions of prejudice. Nor can there be “informed guesses.” If you need to state how many planets with life choose to communicate, there is simply no way to make an informed guess. It’s simply prejudice.

The Drake equation can have any value from “billions and billions” to zero. An expression that can mean anything means nothing. Speaking precisely, the Drake equation is literally meaningless, and has nothing to do with science. I take the hard view that science involves the creation of testable hypotheses. The Drake equation cannot be tested and therefore SETI is not science. SETI is unquestionably a religion.

The fact that the Drake equation was not greeted with screams of outrage — similar to the screams of outrage that greet each Creationist new claim, for example — meant that now there was a crack in the door, a loosening of the definition of what constituted legitimate scientific procedure. And soon enough, pernicious garbage began to squeeze through the cracks.

I agree with him that none of these terms can be known exactly, or even sufficiently precisely to calculate a quantitative answer for N. I also agree that the pursuit of N can take on religious qualities.

But I can’t agree that SETI itself is worthless, nor can I agree that interest in it loosens the definition of “legitimate scientific procedure.” SETI is not just the Drake equation.

Now, I will not attempt to defend even one procedure that SETI workers use, nor will I comment on any statement made by any of its proponents. I cannot say, for example, that searching nearby stars for signals in the hydrogen line makes any sense. But I will say SETI is not the same as religion

I am interested in saying something about the probability of this proposition:

    S = “Intelligent/sentient life besides that on planet Earth exists”

Because we must calculate the probability of S is conditional on some evidence, I offer this blog. Yes, because this blog—because you and I—exist, it means that the universe is set up to allow at least one species of sentient life. Therefore, it is rational to believe that the probability of S given this evidence is greater than 0. I have no idea how much larger than 0 it is. If you are a fan of the reasoning behind the Fermi Paradox, you might say that the probability, while non-zero, is trivially small.

The Fermi Paradox basically says that, since the universe is about 10-13 billion years old, and the one sentient-life example we know of only took about 4-5 billion years to evolve, and since there are plenty of stars and galaxies, there should be sentient life all over the place. That is, SETI should be easy, and since it isn’t, since we haven’t made contact yet, this implies that we are the first or only sentient species. There are obvious subtleties to each stage of that argument that I glossed over, but that’s the gist.

The Fermi Paradox is also conditional on information not articulated. One obvious item is the proposition that all sufficiently advanced civilizations would want to make contact with us. Not just with other species, but with us. That’s a mighty big supposiion. Another hidden assumption is that we ourselves are sufficiently advanced enough to detect messages aimed at us, or have the ability to intercept messages meant for other beings. Pretty big guess, especially with the knowledge that the more efficient a message gets, the more it looks to an outside like noise (basic information theory; deep ties with probability and statistics there), and so civilizations more advanced than us might have communications which are impenetrable to us.

That argument cuts both ways, of course. If the messages are too complex, any search for them is fruitless. And, well, you get the idea. It’s complicated, so much so that it is not an open and shut judgment that SETI is valueless.

Though we have to be careful. Wishcasting is always a danger here, as everywhere. A lot of people—me included—want S to be true and this naturally clouds our judgment.

Got in and just sent out the second batch.

I’m glad I ordered a third batch, because I received more orders this week: those that did order this week will have to wait another couple of days.

Thanks everybody!

And just for a pleasant diversion:

The first batch of books has come in and I’ll very likely mail them out first thing tomorrow.

I had to order a second batch, so those who ordered books since Monday will have to wait a few more days. I expect to have the next set sometime over the weekend or early next week.

I also, just now, ordered a third batch in case anybody else wants one (I also gift some to various VIPs).

Thanks everybody!

It is a cliche, but it is true, that academics must publish or perish. Papers, and more papers, and more papers still, are what makes for a professorial life.

It’s often—it’s very often—not the quality of the papers that counts, it’s the count of the papers that qualifies one for promotion, tenure, and other glories. In many, or even most, places an informal target number exists, saying have this many or it’s out the door you go.

So it should not be surprising that people eek out every ounce of information from a study and try to write it up in as many ways as possible. If you run, say, a clinical trial, and you can only get one paper out of it, then, if you’ll pardon me saying so, there is something wrong with you.

What usually happens, in for example clinical trials, is that a paper is written describing the trial methodology, even if the study design is no different than dozens of hundreds of other studies. Another paper is written with the main results. Then as many as can are done on subsets of the data, or on the data with various “scales” that are added on to pad out a trial. A “scale” is a questionnaire about a subject like quality of life. Any good clinical trial should be able to generate a minimum of eight papers, and a dozen or more is not unheard of.

The only problem with this, is that the civilians are starting to catch on, pace this article in The Guardian.

The author, Ben Goldacre, caught a drug company publishing a trial result twice. There was one main paper, and another tying the main results to a “depression scale”. Goldacre was aghast and said that some people will look to the journals and say, about this new drug, “There are two studies showing its efficacy.”

In other words, they will be more certain than is actually warranted. Goldacre also frets about meta-analysis, and how that strange technique will be fooled, too. Meta-analysis is a tool that gathers studies together to show an effect is real even though the effect was not found in most or any of the individual studies (we’ll talk about this subject another day).

But that drug company was doing nothing unusual. The people who work for it need papers too.

Paper churning—for that is the informal name of the phenomenon—is not limited to medicine. We all have stories of the professor down the hall who has been publishing the same paper for years, here and there adding a small twist to make it seem different. There are so many journals, and new ones appearing regularly, that there is always a market for his work.

This has the result that, in my own field of statistics, there are about 100—yes, 100—monthly or quarterly journals. Each has roughly 10-30 articles. In medicine, there are about 2000-3000 regular journals. There is no way to keep up. It is impossible to read more than a tiny fraction of papers. Most, and it pains me to say this, are not worth reading anyway.

Every now and then, academics will gather and beat their breasts and say “No more! From now on we shall also value teaching and service and not just quantities of papers published!” But their resolve lasts only until the end of the meeting. The next day, they go back to tallying.

There doesn’t appear to be a solution. You can’t limit numbers of papers that people are allowed to publish. You could, I guess, insist that all publications go to open source journals, where the authors are required to pay for “page charges”, that is, pay for publication. Professors aren’t rich and don’t have unlimited research funding, so this would slow the rate of papers. But what about graduate students or independents who have no funding? Forcing them to pay is silly.

The old fashioned filter, allowing the creme to rise to the top—in the form of books, usually—is probably all that will work. Unless we somehow can return to the roots of what a university is meant to be. We’ll leave that for another time.

I recall when Bush filius won his first term. I received emails from several of my lefty friends, whom I love, expressing outrage, shock, horror, etc. One even said, and this is a direct quote, “He is not my president!” There were threats of moving to foreign shores. Opinions were freely given that the American people were stupid. They didn’t get what they wanted and they threw a tantrum.

Well, Mr Obama is my president, and I say Good Luck to him and to us. May he do well and not screw things up too badly. Remember, Mr President, that all your supporters will now come calling with their hands out. Do not forget, whether the story is apocryphal or not: Memento te mortalem esse.

Harry, too bad. I was looking forward to that olive oil.

I’ll get to the wishcasting presidential forecast data, I hope, sometime early next week.

A few months ago we looked at a paper that purported to show that watching TV causes autism. Well, that paper has finally been peer reviewed, and therefore published. It’s making the rounds in the media on this historically slow news day.

Monthly Weather Review chief editor Dave Schultz found this article on the BBC web site. Climate-computer guy Dan Hughes found another at the Washington Post.

The original draft paper is here. If you are in one of the ivory towers, you can download the paper here, at the Archives of Pediatrics & Adolescent Medicine.

The idea is that when it rains it drives kids inside to watch television and that watching TV—never mind how—induces autism. The more hours kids spend in front of the boob-tube, the more cases of autism. Since you can’t measure the number of hours of watching TV, you have to do something else. The authors decided that precipitation would be a good proxy. Here are some of the comments from July:

But how can you tell how much TV all these kids watched? You can’t. There is no way to go back to 1970 and count how many hours each baby watched TV. This is a dilemma, because we would really like to test the dose-response. Perhaps there is a proxy? A proxy is a stand-in variable that is so strongly associated with hours of TV watched that it’s almost as good as the real thing. Can you think of any?

How about precipitation? Sure, rain and snow. After all, when it rains, what else is there to do but watch TV? Actually, lots, and when it snows, there’s even more. But, this is the proxy chosen by the researchers (their Figure 6 will hold some interest for those interested in global warming).

They plotted up maps by county for California, Oregon, and Washington, and colored in counties that had more than median precipitation (from 1990-2001) and then colored those with higher than median autism rates. These colored squares tended to be in the same spot, and is what led them to the conclusion that watching TV causes autism. Case closed.

Mark Lever, chief executive of The National Autistic Society, is properly sanguine about the research. He said, “the latest theory would join a succession of others advanced about the condition and its origins.”

In recent years autism has been linked to factors as varied as older aged fathers, early television viewing, vaccines, food allergies, heavy metal poisoning, and wireless technology, to name just a few.

Some of these theories are little more than conjecture or have been discredited, others seem more promising and are in need of further study. As yet, however, very few have been substantiated by scientific research.

We don’t yet understand what causes autism, although scientists do believe that genetic factors might play a part.

People with autism and their families are naturally concerned to get the right information and there is a lot of confusion and concern over the conflicting theories put forward.

Another guy named Weiss “thinks the results of the study need to be taken with a grain of salt.” To counter that, a man called Lathe said, “Emissions from manufacturing industries, power plants, and from domestic waste incineration generally rise to the troposphere to be diluted into the large volume of the atmosphere. Precipitation can dump this load back on the land, to be absorbed by plants and animals in the food chain.” Not very good meteorology there, because we could equally say that lack of precipitation allows the atmospheric pollution to be worse, causing increases in inhaled ozone, etc. etc.

Overall, there doesn’t seem to be a solid link between rain or TV and autism. The authors of the paper even say “that families more prone to having autistic children may reside in areas with high levels of precipitation, or that such areas might use broader diagnostic criteria for diagnosing autism.” There does seem to have been an increase in the rates of autism, but that increase could very easily be from increased awareness and subsequent diagnoses of the disease.

Nothing has changed between the draft work and the peer-reviewed one to cause me to change my mind about the value of the paper. What I didn’t know before, but I learned from the Post today, is that the lead author and economist Waldman has a son who is autistic. I can therefore understand what motivates him and the desire to find out what happened.

Those are the words of Warren Buffet, who warned of the coming credit crisis. Buffet—one of the very few—had little faith in the “complicated, computer-drive models systems that many financial giants relay on to minimize risk.”

Reader Dan Hughes reminds us of this article in today’s Wall Street Journal, which looks at why AIG did so miserably.

AIG built a lot of models which attempted to quantify risk and uncertainty in their financial instruments. They, like many other firms, tried to verify how well these models did, but they only did so on the very data that was used to build the models.

Now, if you are a regular reader of this blog, you will know that we often talk about how easy it is to build a model to fit any set of data. In fact, with today’s computing power, doing so is only a matter of investing a small amount of time.

But while a model fitting the data that was used to build it is necessary condition for that model to work in reality, it is not a sufficient condition. Any model must also be tried on data that was not used—in any way—to build it.

What happened at AIG, and at other financial houses, was that events occurred which were not anticipated or that had not happened before. Meaning, in short, that the models in which so many had so much faith, did not work in reality.

There is only one true measure of a model’s value: whether or not it works. That it is theoretically sound, or that it uses pleasantly arcane and inaccessible mathematics, or that it matches our desires, or that “only PhDs can understand” it are all very nice things, but they are none of them necessary. Many complex models which are in use are loved and trusted because of these things, but they should not be. They should only be valued to the extent that they accurately quantify the uncertainty of the real-life stuff that happens (climate models anyone?).

What the AIG models failed to account for were the “unknown unknowns”—to use Donald Rumsfeld’s much maligned quotation. They did not quantify the uncertainty of events which they did not know about. They thought that the models quantified the uncertainty of every possible thing that would happen, but of course they did not. Meaning that they were overconfident.

AIG’s failure is yet another in a long series of lessons that the more complex the situation, the less certain we should be.

(A subscription is required to read the full WSJ article.)

I’ll be placing the order for the books tomorrow (Monday) morning. I’ll order a few more in case anybody else orders a signed copy. I imagine I’ll have the books by the end of the week, at which time I’ll mail them out. Look for updates here. Naturally, everybody who ordered will get an email.

Thanks for your support everybody!