William M. Briggs

Statistician to the Stars!

Page 152 of 605

Lord, Save Us From Impact Factors

Love that ‘stache!

Randy Schekman, a Big Cheese in the sciences, is right: people use “place of publication as a proxy for quality of science”. Where a paper is often counts more than what the paper is.

This is from Schekman’s “How journals like Nature, Cell and Science are damaging science: The incentives offered by top journals distort science, just as big bonuses distort banking” in Monday’s Guardian.

Other nuggets:

The prevailing structures of personal reputation and career advancement mean the biggest rewards often follow the flashiest work, not the best.

Or the work that brings in that overhead, baby! Judging by money alone, the major business of colleges are two: squeezing money out of Leviathan and sports. Everything else lags.

While [luxury journals] publish many outstanding papers, they do not publish only outstanding papers.

Steer clear especially of anything hot, current, “sexy”. These papers are likely to be dreck.

The exclusive brands are then marketed with a gimmick called “impact factor” — a score for each journal, measuring the number of times its papers are cited by subsequent research…

It is common, and encouraged by many journals, for research to be judged by the impact factor of the journal that publishes it. But as a journal’s score is an average, it says little about the quality of any individual piece of research. What is more, citation is sometimes, but not always, linked to quality. A paper can become highly cited because it is good science — or because it is eye-catching, provocative or wrong. Luxury-journal editors know this, so they accept papers that will make waves because they explore sexy subjects or make challenging claims. This influences the science that scientists do. It builds bubbles in fashionable fields where researchers can make the bold claims these journals want, while discouraging other important work, such as replication studies.

Amen and amen! Much more time, especially in the vast intellectual backwaters like education and the like, in replicating—to the letter—so-called foundational studies, i.e. those works which everybody believes are true but have never been precisely checked. “Near” checking is not checking, incidentally.

And pay attention: a journal’s “impact” factor—which should measure the force with which it hits the trash can (that’s a joke)—is only a weak, tepid indicator of the quality of its papers. But it’s a pretty good take on how “hot” the journal is. What civilians don’t understand is when a journal’s “impact” factor is on the increase, more scientists start sending papers to it with the mindset, “Hey, might as well try.” It soon becomes the thing to have your work appear in this journal. “Did you hear Jones has a new paper in JASA?” That it appeared in a luxury journal is all that is remembered. What the paper was about is only secondary.

It all goes back to the money, of course. More highly cited papers, the bigger the chance of brining in the overhead. But never mind.

One group I was in scored a paper in JAMA (or was it Lancet?). The egos who run the place not only sent a letter of acceptance but also an invitation to have the front page of the new article bronzed (for a large fee) and framed “suitable for hanging.” I kid you not. Now that’s science.


There is a better way, through the new breed of open-access journals that are free for anybody to read, and have no expensive subscriptions to promote. Born on the web, they can accept all papers that meet quality standards, with no artificial caps. Many are edited by working scientists, who can assess the worth of papers without regard for citations.

This is okay, but what Schekman forgets is that open-access journals are pay-as-you-go: scientists must pay “page charges” for their work to appear. This leaves out folks like Yours Truly who has no grants, no income, and no secretarial support.

Slightly better, and the position adopted (by now by almost 100%) by physicists and mathematicians is to deposit your “pre-print”, i.e. an unedited paper, on arxiv.org. Free to place, free to download, which guarantees a larger readership.

Or one could even launch one’s ideas on some public space, like a blog. About 70,000 people a month drop by here; I’m not even sure 7 people ever read any of the official papers I wrote. Plenty of open, hearty, peer-review here, too: more than from any journal.

What Is And What We Know Of It, Probabilistically Speaking

There are more blue states than red states.

Ontology is the study of what is and what is not. Epistemology is the study of our knowledge of what is and what is not. Though there are obvious points of overlap and dependencies, it would be a mistake or fallacy to confuse that a thing exists with whether this or that person knows the thing existed. This mistake or fallacy accounts for the frequentist interpretation of probability which mislabels the existence of things (relative frequencies) as the probability of things.

This mistake makes sense because sometimes the relative frequency matches the probability. But it does not always do so. For instance, if in a bag there are n objects just one of which is labeled X and just one will be pulled out, the relative frequency of objects labeled X is 1/n which matches the probability that this object is drawn. The existence of the object matches our knowledge of it.

But if our premises are that “All Martians wear hats and George is a Martian” relative to the proposition “George wears a hat”, then there is no relative frequency—there is no ontology (to speak loosely and in the form of computer scientists); there are no Martians wearing hats and therefore no Martians named George. But there is a probability (which equals 1). “All” in the premises may be changed to “Some” or “Just 3 of n” or whatever and the conclusion that no relative frequency but a probability exists remains the same. And (of course) no counterfactual proposition has a relative frequency, but most (all?) can have a probability.

Suppose you don’t know, you have a “blank epistemology”, of the number of objects labeled X in the bag, but you at least know that each object has the possibility of being X. This is akin to suspecting a “loaded” coin, or in “trials” where there will be a defined success (the X) and failure, or whether an elementary particle in this field and measured in that way will show a certain property1, or to any situation where the concern is one thing whose presence is contingent. It is easy to show that there is a still a probability of “drawing” an X (which is 1/2). But while there exists (at least in the here-and-now fixed bag) a relative frequency, it is unknown and therefore cannot be equated with the probability.

The relative frequency in a (say) drug “trial” is trickier. The number of elements (the “sample size”) will be finite. Of course, the relative frequency will eventually be something, say m/n successes, but at no point until we have reached the end will we know what the relative frequency is. Yet at each point the probability is still calculable (in the same way as discovering the initial 1/2), and of course eventually becomes the relative frequency—relative to the proposition “This element in the experiment was X” and knowing only there were m successes and n possibilities. But then the probability is also 1 relative to the same proposition but this time including all the knowledge from the trial (because we know whether each individual was a success or failure).

Now suppose we want to extrapolate what we have learned from the trial—of which everything is known, thus any proposition relative to this knowledge will have extreme probabilities (0 or 1) or any probability (propositions which have no logical relation to the trial but which are contingent will have the interval from 0 to 1). If we want to say something about the next n people before they take our drug, again there will eventually be a relative frequency but it is now unknown. Yet the probability is known (in the same manner as before). And again, once we reach the end of the n, we know everything there is and our probabilities are once again extreme or any probability (the interval (0,1)).

The next abstraction is to assume the trial’s (or even initial) results will be with respect to an infinite population: n goes to the limit, a mathematically desirable state. But nothing changes. We are still able to discover a probability at any point before the “long run” expires. We will, of course, wait for forever and a day before a relative frequency (of the entire set) exists. Once the Trump of Doom sounds and time ends, we will have everything we need to know and the probability and relative frequency will match. But nobody (at that point) will care.

How do we know this? The strong2 “law” of large numbers states that (or, in other words, it can be proved beyond all doubt that):

\Pr\!\left( \lim_{n\to\infty}\overline{X}_n = E(X|V) \right | V) = 1.

which is to say the probability that the growing sample’s relative frequency (the average) equals the “expected” value of an observation given some evidence V is 1, but only at the limit. Notice we have used the limit twice, one time boldly and the other hidden in E(X|V), the expected value. Calculating the expected value thus assumes the probability is known (deduced via V). In other words, the law is right and always has been, and those who use as a justification for calling the relative frequency the probability in finite slices of infinite samples have got it backwards.


1There is a healthy debate whether quantum theory is epistemological or ontological, or a mixture of the two. See inter alia the work of Anton Zeilinger (here or here). Zeilinger has scientist hair, so you know you can trust him.

2The difference between the strong and weak laws for this discussion are negligible.

Is It Okay To Force A Baker To Sell Cakes In Violation Of His Religious Conscience?

If I knew you were coming, I’d’ve baked a cake.

Here’s the setup: a Colorado judge has ordered a Christian man and owner of a bakery that he must bake cakes for homosexual civil ceremonies (homosexual “marriage” is illegal in Colorado) or face fines.

This will please some of you, because you are in favor of so-called same-sex marriage and you feel that anybody who is against it deserves whatever he gets. But I imagine (or hoping) your support does not include the use of fallacious arguments. The question before us is, should a baker be allowed to refuse to bake cakes for those ceremonies which violate his religious practices?

According to the news report, an aggrieved pair denied a cake at a bakery filed a compliant with the government, part of which read. “Being denied service by Masterpiece Cakeshop was offensive and dehumanizing especially in the midst of arranging what should be a joyful family celebration. No one should fear being turned away from a public business because of who they are.”

It is irrational and childish to claim that being denied a slice of cake is “dehumanizing”. This is the complaint of a three-year-old forbidden to lick the icing bowl. If this part of the argument carries any weight with you, then you are lost.

How convincing is “No one should fear being turned away from a public business because of who they are”? Suppose a seven-year-old bellies up to the bar and asks for the shot of the water of life. Turning him away because of who he is would be wrong, if we accept this argument. What if a man ventures to a pharmacy and insists on being sold conception-prevention pills? Or what if a convicted serial child rapist insists on his “right” to wallow in those plastic balls at the local Chuck E Cheese? (And see this.) Clearly, we often and for good reason exclude people because of who they are. The question remains: does the baker have the right not to do business with those he does not wish to.

In steps the ACLU. They state, “While we all agree that religious freedom is important, no one’s religious beliefs make it acceptable to break the law by discriminating against prospective customers. No one is asking Masterpiece’s owners to change his beliefs, but treating gay people differently because of who they are is discrimination plain and simple.”

Whether or not the baker’s refusal is “breaking” the law is the matter before us, and recall same-sex “marriage” in Colorado is illegal. Discriminating against customers is what we have already decided is allowable in the right circumstances. The next statement is a pip: “No one is asking Masterpiece’s owners to change his beliefs…” But the ACLU is asking the baker to change his beliefs. The baker’s belief is that he should not serve cakes for services which violate his religious conscience. The ACLU insists the baker abandon this belief.

Next: “treating gay people differently because of who they are is discrimination plain and simple.” “Gay” people, or those who actual in a homosexual fashion, by their own admission, are different. And again, whether or not the baker can thus treat them differently than traditional, biological couples is the question before us. This argument assumes what it wishes to prove.

The final non sequitur came from the judge who ordered the baker to violate his religious beliefs. The news report summarizes it thusly:

Judge Spencer said Phillips did not demonstrate that his free speech rights had been violated and he said there’s no evidence that forcing him to make a cake for a same-sex ceremony would hurt his business.

“On the contrary, to the extent that the law prohibits Respondents’ (Phillips) from discriminating on the basis of sexual orientation, compliance with the law would likely increase their business by not alienating the gay community,” he wrote.

The judge, as many do, has confused morality with money and believes that more of the latter trumps any of the former. It is true, however, that if the baker sold cakes in opposition to his religious beliefs he would make more money. Perhaps he could charge thirty pieces of silver for each cake.

This argument is a non sequitur because the baker has already insisted that money is not his primary motivation: his religion is. It is obvious to everybody except the judge that the baker was willing to forgo extra money in order to protect his conscience. The judge—we are only surprised he is not from Chicago or Brooklyn—cannot differentiate the two concepts.

We’re left with nothing from this ruling, so we have to re-ask why does this couple’s “rights” trumps the baker’s Constitutionally guaranteed rights of practicing his religion? Should the baker be throw out of business or into jail for simply refusing to bake a cake?

Update Colorado baker stands by his beliefs, says he would go to jail to keep from making cake for same-sex wedding.

Reader Survey

All present and accounted for!

Something fun for everybody: the WMBriggs.com reader survey! Come and tell us about yourself and learn what you only suspected about everybody else. It should only take 20 seconds or so.

This survey will remain up for the duration, accessible at this page or through the top menu bar (“Reader survey”). From time to time I’ll remind us of it.

Please, oh I beg you, please tell the truth, please only vote once, and please do vote on all the questions. Remember, all polls are valid representations of the sort of people who fill them out.

Reminder: the survey is anonymous. I am not NSA. I never use your emails or IPs and I delete all my logs after about a week. I keep various counts, though. We’re at about 70,000 views per month. Pretty good for the obscure subjects which are our focus.

If you have more to say, tips, suggestions, complaints and so forth, please leave them in the comments section.

Merry Christmas!

Update Thanks for the great response, all! Keep ’em comin’!

[yop_poll id=”1″][yop_poll id=”2″][yop_poll id=”3″][yop_poll id=”4″][yop_poll id=”5″][yop_poll id=”8″][yop_poll id=”6″][yop_poll id=”7″]

« Older posts Newer posts »

© 2015 William M. Briggs

Theme by Anders NorenUp ↑