William M. Briggs

Statistician to the Stars!

Page 151 of 408

Interest In On-Line Statistics Courses?

You might have heard what happened to when Stanford professor Andrew Ng put his machine learning (a practical kind of statistical modeling) on line. He expected mild interest. One hundred thousand students signed up.

This was enough to excite even the New York Times, which dispatched Tom Friedman to investigate. He called Ng’s success a “breakthrough”, which it certainly is, in its way.

Friedman also puts us on to a new company called Coursera, which offers courses by professors from well known universities, in much the same spirit as Ng offered his. “The universities produce and own the content, and [Coursera is] the platform that hosts and streams it.” Too, many universities already have on-line courses housed on campus.

Several things. Stanford pays Ng’s salary. This money is only partly for teaching, and more so for Ng to publish papers—which contain the material which will eventually be taught. If Stanford didn’t pay him, he would have little time to think of what new to say. Also, Stanford rightly owns the content to Ng’s lectures. Pay for work, etc. Same thing at Coursera, which hires out the professors for a cut of the pie. All well and good.

Some courses are ideal for the web. The closer the course content is to cookbook recipes, the apter. I mean no disrespect. A courses which shows you how to install and run a certain program is nothing but a series of recipes, a well marked path with milestones and a known destination. Basic machine learning fits this scheme. As do the courses offered at Coursera: Algorithms, Calculus, Introduction to Logic, Vaccines, and Securing Digital Democracy (electronic voting schemes). Recipe does not mean easy.

But maybe not statistics. Unless you want a course in classical frequentist thinking, which is cookbook all the way. Coursera has one just like this. Taught, like it is in many places, by a psychologist (who I’m sure is a nice guy).

Don’t see any courses in history, poetry, literature, high-level philosophy, and the like; all classes which are not amendable to multiple choice testing. These are courses which don’t necessarily have an end, or have different possible ends depending on the mixture of students, or which require students do a lot of writing and talking.

On-line, it’s just as easy for a professor to grade one multiple choice or only-one-correct-answer test as it is to grade 100,000 such tests. But only if the class is recipe-based. One professor could not read through 100,000 essays, or listen to as many presentations. In fact, for these kinds of “free form” courses, he may not be able to handle as many students on-line as he could in person, since the material delivered remotely introduces some level of ambiguity and slows down interaction.

Student interaction for recipe courses, which are (and should be) more popular, is greater, because if the recipe says that “at time T, add X cups,” then it’s likely many students will have this information at hand when they are queried at some forum. This is not as likely in the free form class, where the answers are rarely as firm.

The way I run my introductory class is free form. It is also not a regular course in statistics, but Bayesian from the get go, and in the predictive sense (“all parameters are a nuisance”) regular readers will understand. This marks it an oddity. I get away with it because of a certain rare confluence of events. But it would never fly at most universities, where professors at their professions are more conservative than Rick Santorum (“What! Teach Bayesian probability before frequentist? Never!”). Just ask any professor how easy it is to introduce a new course into the system.

I lecture, but not in contiguous blocks of time. I ask the students lots of questions and frequently. The answers they give provide direction for the course. Students come to the board and work out various matters with themselves, guided by me. I have students collect data (which fits a given paradigm) on any subject which interests them. This a wonderful way to maintain interest, but it limits the use of canned examples, the use of which would free up time.

I also have to spend a lot of time walking people through basic computer tasks, especially R. As a final exam, each student presents a talk on their subject as to an audience presumably unaware of statistics (many use data from their workplace, which they later show their bosses, reportedly to good effect). They must describe their interest, the data, the pertinent questions, show pictures, explain how they quantified their uncertainty, and finally detail how they would check all when new data arises.

Could this work on line? I’m skeptical, but intrigued. Places like Phoenix University push thousands of students through their pipes, and not all the classes are recipe-like, so maybe it can be done. All ideas welcomed.

The last difficulty is “credit.” Some courses earn credit as normal classes. But the free courses universities and Coursera offer come with nothing except a pat on the back or perhaps a letter stating that the student made it through. Of course, I love this trend away from formal “certification” and towards actual love of learning. Seems to work best for recipe courses, though, whose students actually want to bake a cake.

The “free” part doesn’t hurt when attracting students: a fine plan for behemoth institutions; wouldn’t work well for little guys like me.

Ignore below

Test of latex. Should be pretty: sin(x) \ne \int e^x dx, \Pr(x|e) = 0.5[\latex]

Richard Dawkins Turns Gideon?

Today a story which falls under Bets You Would Have Lost. Angry atheist (sounds like a board game) Richard Dawkins said he would have “contributed financially to the scheme” of passing out free King James bibles to U.K. students had somebody contacted him about it. Not only that, but he said he was “a little shocked” that the tots did not already have the bibles.

And not only that, but he, in a rare non-sarcastic voice, hopes that the kiddies actually read the bibles!

All this from a man who likened religion to child abuse. But don’t fret about that judgment or worry about hypocrisy because Dawkins conjoins his religion-equals-child-abuse meme with the belief that rape, child abuse, and other sins against the body are morally “arbitrary.”

By which he means that the raw emotions which tell us these acts are “wrong” were placed in our brains by selfish genes operating under the pressures of evolution, and as such these emotions are able to be suppressed (as he would have religious impulses suppressed) with the application of Reason. Women—let her that readeth understand—should not ride in elevators with Dawkins in case he forgets to switch on his Reason Center.

His stance does make it sound odd that he would have England’s children read the bible to learn about morality, however. He said he would have the bible read because the bible “is not a moral book and young people need to learn that important fact because they are frequently told the opposite.”

According to the Guardian, Dawkins says, “I have heard the cynically misanthropic opinion that without the Bible as a moral compass people would show no restraint against murder, theft and mayhem. The surest way to disabuse yourself of this pernicious falsehood is to read the Bible itself.”

Two things may be gleaned. The first, that Dawkins believes the bible is filled with advice on how to murder, steal, and run amok, or that, while it contains stories on these matters, it does not condemn them. The second is that these actions are all, to him, morally arbitrary anyway, belonging to a non-Enlightened, and possible earlier evolutionary, past.

But never mind! As it is he supports fully the plan created by Education Secretary Michael Gove to distribute the books and to have the children learn from them. Not all his co-anti-religionists agree with Dawkins. The National Secular Society is—are you ready?—outraged (isn’t everybody?) and says the plan favors Christianity. They have (selectively? purposely? in pure ignorance?) forgotten their culture also historically favored that religion.

An old story has a lady saying to a prominent philosopher that she did not see why anybody should read the bible or Shakespeare since they were both filled with cliches. Like this lady, Dawkins is aware that much of our language was shaped by King James’s effort, just as he knows how important that translation is to Western history. He would have the bible read for these reasons.

Dawkins was spot on when he said, “Ecclesiastes…is one of the glories of English literature,” though the New Testament is (mostly) a closed book for him. It’s selective (snarky) quotations from the Old Testament all the way. He even managed yesterday to work in an allusion that ancient Jews were Nazis. Don’t believe this? He said, “It was perfectly fine — indeed strongly encouraged throughout the Pentateuch — to kill Canaanites, Midianites, Jebusites, Hivites etc, especially if they had the misfortune to live in the Promised Lebensraum.”

Ah, Richard. Your selfish memes have taken over what you falsely believed to be your consciousness. Don’t worry. It’s not your fault. There’s no “you” there to carry fault anyway. “You” are just a mass of unthinking proteins, blood, and no guts (Dawkins still hides from a man called William Lane Craig). Besides, given the state of our world, nobody will notice “your” faux pas.

But never mind! We should celebrate that an enemy to religion like Dawkins is now friendly to the idea of kids reading the bible. Some of these kiddies may adopt Dawkins’s imaginatively lurid theology, it’s true, but more (I predict) will take away something better. Some will even come across this verse, which has eluded Dawkins’s glance: “This is my commandment, That ye love one another, as I have loved you.”

Update For Briggs. Remember we are all ladies and gentlemen in the comments. No name calling is allowed.

Scientific Truths Are Not Better Truths Than Just-Plain Truths

One of the key fallacies of scientism, in the sense of being the most destructive to common sense and personal wellbeing, is to suppose that any theory put forth in the name of science is therefore true, or certain enough to believe as true. The posited theory is, after all, scientific and, so scientism says, there is no better recommendation to truth than this.

This fallacy is field dependent, cropping up in some areas much more frequently than in others. It is rare, though still frequent enough in a mild sense, to find the speculations of chemists being refuted each new generation. But it is common as daylight to find the hypotheses put forth by sociologists, economists, and psychologists refuted not a generation after they are published, but often in the next issue of the same journal.

For example, the Weekly Standard’s Andrew Ferguson’s reviews the bad science and scientism behind the recent spate of experimentation “proving” conservatives are dumber/more inflexible/less compassionate/etc. than liberals, theories which are collected in Chris Mooney’s hagiography to scientism, The Republican Brain. (We began a collection of these studies here; please contribute. Also see Mike Flynn’s take on this.)

The studies rely on the principle that has informed the social sciences for more than a generation: If a researcher with a Ph.D. can corral enough undergraduates into a campus classroom and, by giving them a little bit of money or a class credit, get them to do something—fill out a questionnaire, let’s say, or pretend they’re in a specific real-world situation that the researcher has thought up—the young scholars will (unconsciously!) yield general truths about the human animal; scientific truths.

Although he didn’t intend it, Ferguson shows us another fallacy, which is that, in virtue of their being generated by scientists, that “scientific” truths are better than other kinds of truths, say metaphysical or logical truths. Stating it so plainly makes it obvious that if a truth is a truth, then it is a truth, and a truth is not more “truthy” because it comes from a scientist than a truth which comes from (say) a theologian.

The main problem with this summary is that often scientists use the word truth to mean “a belief which is probably but not 100% certainly, no-matter-what true.” That later creature is not a truth at all; it is a conjecture and nothing more. A conjecture which is “almost” true, or “for all practical purposes” true, is still a conjecture and not a truth. A truth is only true when it always is, when it can be deduced, when it arises as the end result of a valid argument. That is, conjectures when they are conjectures are not truths, but conjectures might become truth as new evidence arises.

Physicists make the mistake of confusing truth and conjecture just as often as sociologists and psychologists, only the physicists’ conjectures more often turn out to be truths as that new evidence arrives, so their error is of less consequence. Note that it is an error (a fallacy) to say, given evidence less than deductive, that a conjecture is a truth. The error will turn out to be more or less harmful depending on to what use the conjecture is put. If one is betting that a protein will fold a hypothesized way, one has turned a conjecture into a forecast, which is a term that acknowledges a conjecture is less than a truth.

If the conjecture turns out true, because of new evidence from an experiment, then the conjecture turns into a truth and gains are made. If the conjecture turns out false, we again know this based on new evidence, and loses are suffered. The loses are of particular interest: these are less the closer the conjecture is to the truth. Which is why the loses are greater in the soft sciences: their conjectures are much more often farther from truth.

One reason for the difference is that physicists more often than sociologists test their conjectures against reality. Another reason is that the evidence for a conjecture for the hard sciences is not just statistical, as it often is for soft-science conjectures. And any conjecture which relies primarily on statistics—given, that is, how statistics is practiced today—should not be trusted.

I’ll insert my usual plea that soft scientists act more like their hard (knock) brothers. Do not just assemble a one-time shot of data and compute some statistical model and tell us how well that model fits your data, and then assume because this fit is “good” that therefore your conjectures are true. This is formally a fallacy and is the weakest kind of evidence there is, but (almost universally) the only kind which is offered.

Instead do two things: (1) find very effort to discover evidence which refutes your conjecture (and then tell us abou it). And (2) as hard scientists (often) do, make predictions of data you have never before seen. If you do both these things, then you can ask us to believe your conjectures. Otherwise, keep quiet.

————————————————————————————

Thanks to the many readers who pointed me to Ferguson’s article, including Mike Flynn and Doug Magowan.

What Organic Boors, Swedish Pronouns, And The Exorcist Have In Common

The ExorcistNot much, except to demonstrate that the natural state of modernity is something closely resembling mass lunacy. To explain.

Swedish Pronouns

Via Sam Schulman ‏(Twitter: @Sam_Schulman) we infer that 1984 has not been translated in Swedish. This is an inference, mind, and not a direct claim. But you’ll agree it is a likely one after you learn that in Sweden, “it is now considered a distinct discrimination if one is addressed as a man or woman.” So reports Kopp Online.

Sweden is angling to de-genderify their pronouns so that use of he or she is officially discouraged, to be replaced by something resembling it (hen). Not only does this move strip useful information from its language, the Swedes have made an important step in subtracting from a person’s humanity, since to be called an “it” is to be equated to a chair or a bug. Now that, dear reader, is true equality. And it is under the banner of Equality that these changes are being made.

Ho hum you say? Then consider that “Nyamko Sabuni, currently Minister for Integration and Gender Equality, is now trying politically to free children from the constraints of gender roles.” Sabuni is leading an effort to de-genderify names, so that if he succeeds Sweden will be no longer have the equivalent of Bills and Janes, but will be a nation of only Pats and Chrises.

There is in Sweden a kindergarten with the telling name of Egalia where “gender-free” children are taught the joys of homosexuality and to play house, imagining, for instance, that there are “two or three mothers.” In a separate article, the wardens of this institution justify their experimentation by claiming that “gender” is not something which you are born with, but is something which can be “changed at any time.” This being so, the little tots should learn early how to do this morphing.

Organic Boors

Today reports of new “research” which confirms what everybody who has shopped in an urban farmer’s market already knew (“research” is needed because everybody did not have a p-value to accompany their belief). The news is that organic food boors are often bullies, that they are often self-satisfied “snotty and arrogant” moralists.

“There’s a line of research showing that when people can pat themselves on the back for their moral behavior, they can become self-righteous,” says author Kendall Eskine, assistant professor of the department of psychological sciences at Loyola University in New Orleans. “I’ve noticed a lot of organic foods are marketed with moral terminology, like Honest Tea, and wondered if you exposed people to organic food, if it would make them pat themselves on the back for their moral and environmental choices. I wondered if they would be more altruistic or not.”

Eskine found that “organic people judged much harder” than ordinary humans, that when “it came to helping out a needy stranger, the organic people also proved to be more selfish.” The money quote:

“There’s something about being exposed to organic food that made them feel better about themselves,” says Eskine. “And that made them kind of jerks a little bit, I guess.”

The only surprise is that Eskine found his results surprising: “You’d think eating organic would make you feel elevated and want to pay it forward,” he said.

He (I assume Eskine is a “he” because his first name is Kendall) should have gone to any Whole Foods…

HT Hot Air.

The Exorcist

Stay with me for a long story made short: Health and Human Secretary and self-labeled Catholic Kathleen Sebelius was invited to speak at one of Georgetown University’s commencement ceremonies, which was, or is, kind of, affiliated with the Catholic Church. This was controversial because Sebelius instituted a “mandate” which said that Catholic employers must provide (via “health” “insurance”) their employees abortion-inducing drugs and contraception. Recall poor Sandra Fluke and her plea for somebody—anybody but herself—to fund her birth control.

Abortion and birth control, whether you are for them or not, are against Catholic doctrine, meaning that any Catholic institution had no business honoring a woman like Sebelius who knowingly “mandated” a removal of religious freedom. Sebelius, in a weak attempt to justify her curious behavior and stick in the eye of her critics, in her speech said,

[President John] Kennedy talked about his vision of religion and the public square, and said he believed in an America, and I quote, “where no religious body seeks to impose its will directly or indirectly upon the general populace or the public acts of its officials — and where religious liberty is so indivisible that an act against one church is treated as an act against us all.”

Like many other things in the Obama government, Sebellius got it exactly backwards. There was no fear the Catholic church would foist its views on an innocent and unwilling public, but instead the reality of an omnipotent government forcing the Church to abandon its core beliefs. Shame on her for misusing Kennedy’s words.

Enter William Peter Blatty, author of The Exorcist and alumnus of Georgetown, who is organizing an official protest against Georgetown, which Blatty believes (if I may) is possessed by a spirit of idiocy.

Even if you are not Catholic and earnestly desire abortion for free, Blatty’s fight is yours. Kennedy was right: “an act against one church” is “as an act against us all.” If you let the government encroach upon our freedom because you lust for free contraception, you must know that this will not be the end and the government will soon come for what you treasure.

How To Present Anything As Significant. Update: Nature Weighs In

Update The paper on which this post was based has hit the presses which was used as an occasion for Nature to opine that, just maybe, that some of the softer sciences have a problem with replication. I thought it important enough to repost (the original was on 3 March 2012).

Nature’s article is Replication studies: Bad copy: In the wake of high-profile controversies, psychologists are facing up to problems with replication. The meat is found in these two paragraphs.

Positive results in psychology can behave like rumours: easy to release but hard to dispel. They dominate most journals, which strive to present new, exciting research. Meanwhile, attempts to replicate those studies, especially when the findings are negative, go unpublished, languishing in personal file drawers or circulating in conversations around the water cooler. “There are some experiments that everyone knows don’t replicate, but this knowledge doesn’t get into the literature,” says Wagenmakers. The publication barrier can be chilling, he adds. “I’ve seen students spending their entire PhD period trying to replicate a phenomenon, failing, and quitting academia because they had nothing to show for their time.”

These problems occur throughout the sciences, but psychology has a number of deeply entrenched cultural norms that exacerbate them. It has become common practice, for example, to tweak experimental designs in ways that practically guarantee positive results. And once positive results are published, few researchers replicate the experiment exactly, instead carrying out ‘conceptual replications’ that test similar hypotheses using different methods. This practice, say critics, builds a house of cards on potentially shaky foundations.

Ed Yong, who wrote this piece, also opines on fraud, especially the suspicion that this activity has been increasing. Yong’s piece, the Simmons et al. paper, and the post below are all well worth reading.

Thanks to James Glendinning for the head’s up.

———————————————————————————————————

false positives My heart soared like a hawk when I read Joseph P. Simmons, Leif D. Nelson, and Uri Simonsohn’s “False-Positive Psychology : Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant“, published in Psychological Science1.

From their abstract:

[W]e show that despite empirical psychologists’ nominal endorsement of a low rate of false-positive findings (≤ .05), flexibility in data collection, analysis, and reporting dramatically increases actual false-positive rates. In many cases, a researcher is more likely to falsely find evidence that an effect exists than to correctly find evidence that it does not. We…demonstrate how unacceptably easy it is to accumulate (and report) statistically significant evidence for a false hypothesis.

Preach it, brothers! Sing it loud and sing it proud. Oh, how I wish that your colleagues will take your admonitions to heart and abandon the Cult of the P-value!

Rarely have I read such a quotable paper. False positives—that is, false “confirmations” of hypotheses—are “particularly persistent”; “because it is uncommon for prestigious journals to publish null findings or exact replications, researchers have little incentive to even attempt them.” False positives “can lead to ineffective policy changes.”

Many false positives are found because of the “researcher’s desire to find a statistically significant result.” That researchers are “are self-serving in their interpretation of ambiguous information and remarkably adept at reaching
justifiable conclusions that mesh with their desires” “Ambiguity is rampant in empirical research.”

Our goal as scientists is not to publish as many articles as we can, but to discover and disseminate truth.

I am man enough to admit that I wept when I read those words.

Chronological Rejuvenation

The authors include a hilarious—actual—study where they demonstrate that listening to a children’s song makes people younger. Not just feel younger, but younger chronologically. Is there nothing statistics cannot do?

They first had two groups listen to an adult song or a children’s song and then asked participants how old they felt afterwards. They also asked participants for their ages and their fathers’ ages “allowing us to control for variation in baseline age across participants.” They got a p-value of 0.033 “proving” that listening to the children’s song made people feel younger.

They then forced the groups to listen to a Beatles song or the same children’s song (they assumed there was a difference), and again asked the ages. “We used father’s age to control for variation in baseline age across participants.”

According to their birth dates, people were nearly a year-and-a-half younger after listening to “When I’m Sixty-Four” (adjusted M = 20.1 years) rather than to [the children's song] (adjusted M = 21.5 years), F(1, 17) = 4.92, p = .040.

Ha ha! How did they achieve such a deliciously publishable p-value for a necessarily false result? Because of the broad flexibility in classical statistics which allows users to “data mine” for the small p-values.

Ways to Cheat

The authors list six major mistakes that users of statistics make. They themselves used many of these mistakes in “proving” the results in the experiment above. (We have covered all of these before: click Start Here and look under Statistics.)

“1.Authors must decide the rule for terminating data collection before data collection begins and report this rule in the article.” If not, it is possible to use a stopping rule which guarantees a publishable p-value: just stop when the p-value is small!

“2. Authors must collect at least 20 observations per cell or else provide a compelling cost-of-data-collection justification.” Small samples are always suspicious. Were they the result of just one experiment? Or the fifth, discarding the first four as merely warm ups?

“3. Authors must list all variables collected in a study.” A lovely way to cheat is to cycle through dozens and dozens of variables, only reporting the one(s) that are “significant.” If you don’t report all the variables you tried, you make it appear that you were looking for the significant effect all along.

“4. Authors must report all experimental conditions, including failed manipulations.” Self explanatory.

“5. If observations are eliminated, authors must also report what the statistical results are if those observations are included.” Or: there are no such things as outliers. Tossing data that does not fit preconceptions always skews the results toward a false positive.

“6. If an analysis includes a covariate, authors must report the statistical results of the analysis without the covariate.” This is a natural mate for rule 3.

The authors also ask that peer reviewers hold researchers’ toes to the fire: “Reviewers should require authors to demonstrate that their results do not hinge on arbitrary analytic decisions.”

Our trio is also suspicious of Bonferroni-type corrections, seeing these as yet another way to cheat after the fact. And it is true that most statistics textbooks say design your experiment and analysis before collecting data. It’s just that almost nobody ever follows this rule.

Bayesian statistics also doesn’t do it for them, because they worry that it increases the researchers’ “degrees of freedom” in picking the prior, etc. This isn’t quite right, because most common frequentist procedures have a Bayesian interpretation with “flat” prior.

Anyway, the real problem isn’t Bayes versus frequentist. It is the mania for quantification that is responsible for most mistakes. It is because researchers quest for small p-values, and after finding them confuse them for holy grails, that they wander into epistemological thickets.

Now how’s that for a metaphor!

——————————————————————————————–

Thanks to reader Richard Hollands for suggesting this topic and alerting me to this paper.

1doi:10.1177/0956797611417632

« Older posts Newer posts »

© 2014 William M. Briggs

Theme by Anders NorenUp ↑