Anyway, the real post of the same name is at Quirks, the trade journal of marketers. In it, I outline the most common blunders users of statistics make, at least when it comes to marketing.
I won’t repeat any of the details, but here are the main points:
1. Asking too many questions Questionnaire fatigue is rarely considered but results in beleaguered respondents clicking the same answer for all queries just to get the thing over. Also, Big Data won’t save us all, though it will have successes, which will be just as transient as all sociological findings.
2. Failing to appreciate limitations Many feel that the answer can always be had if only sufficient sweat were expended or enough data collected. If the future were certain, then we’d all be climatologists.
3. Not understanding regression If you read only one point, make it this one. I am always amazed at how many people who routinely use statistical models have no idea of their purpose. We statisticians are probably responsible for this state of affairs because of the way we cherish parameters. Parameters become the reason for models, even though they are entirely invisible, metaphysical creatures.
4. Falling for the latest gee-whiz approach “Can you make my data Big Data?” Yes, actually asked. But more usually, “I just read about technique X. We need to use it on our data.” Even though technique X is no relevance to the question at hand, it sure does sound sexy. As I say, the best analysis often is no analysis at all: simple counts, tables, and pictures give a good feel of the situation and are less likely to lead to over-certainty.
5. Not coming to a statistician (soon enough) A lot of folks have their statistical training from well-meaning, kind, morally upright, entirely praiseworthy people who themselves are not statisticians and who have in turn learned their stats from people like themselves, and so on. Fine for dabbling, but not a course likely to lead to an appreciation of pitfalls and subtleties. The situation is like learning quantum mechanics from garage mechanics because both subjects study movement: it can work—there are lots of capable garage mechanics—but perhaps there is a better way.
We learn quantum physics from “Tim the Toolman Taylor”. He wrote a book.
Sheri,
Is that the same guy who invented plaid paint?
Here is a wonderful example of mistake number 1.
http://www.forcesitaly.org/italy/evidenza/questionario/quest_iarc_98.pdf
I take it then that you don’t buy into the plot of “The Flight of the Phoenix”. Also the way that you talk to marketers is interesting. Did you take a special course for this?
DAV: Not in this quantum reality.
Scotian: I like the way Briggs talks to marketers.
Oh, Scotian: Isn’t the plot of “The Flight of the Phoenix” basically the same as every MacGyver episode out there?
Sheri, I’ve never seen a MacGyver episode and can’t answer your question. Did he design model airplanes?
Scotian: No, it was a television program about a person was always getting into situations where he needed a tool or other item and none was at hand, so he created a solution from the items he could find. If he crash landed, he would revamp the airplane and get it to fly using whatever was at hand. (I think he actually did that in one episode.) People have been known to say someone “MacGyvered” that when someone creates a solution of items on hand.
An engineer once came into our office with a stack of papers and plumped the stack on Mohsen’s desk. Mohsen was our QA statistician. The engineer says to him, “I’ve just run an experiment and I want to know what the data says.”
Mohsen leans over and puts his ear to the stack and listens carefully. Then he looks at the engineer. “They say you should have come to me before you ran the experiment.”
This is a true story.