Today’s post is at Quirk’s, the well known trade journal and marketing research review.
If you want to read the article on-line at Quirk’s, registration is required, but free.
You can also download a PDF copy of the article: Telling the future from the past: predictive versus classical statistics.
If you rely exclusively on classical statistical techniques, then you will often be too certain of yourself. You will derive answers and make decisions in which you are too confident. This is guaranteed. Why? Well, read the article, or stick around. I’ll be writing about this more.
The audience for this paper is working statisticians or marketers and executives who have experience using statistics. Here’s the start:
There is a story about a marketing statistician who was asked by his mother what he was doing. “Modeling for Victoria’s Secret,” he said. “You’re doing no such thing!” she said. She was shocked. She shouldn’t have been, because classical statistics is a lot like a modeling lingerie.
A common experience many readers, especially of the male type, have of the Victoria’s Secret catalog is to marvel at how well the models exhibit their wares. A reader, surely fixated on fashion, might closely examine a photograph and say, “This model appears ideal. Her clothing fits perfectly.” Some especially attentive viewers can tell you the measurements of the garments down to the nearest fraction of an inch. They look at a model and announce, “She must not have got that outfit off the rack because there’s almost no chance a ready-made garment would have fit that well. It must have been made for her.” Yet, knowing this, they still buy the clothing hoping that it will do for them—or a close associate—exactly what it did for the model.
Thanks to Jim Dukarm for providing very helpful comments on an earlier draft.
Henrion, Fischoff, Am. Jour. Physics, 54, 791, 1989, present interesting findings about excessive certainty in physical measurements, and that it leads to the uncertainty bounds on physical constants being far too small — often because the procedures involved in creating them do not explicitly take bias into account. No one seems good at handling bias.
One think I know, however, is that to model lingerie, one needs to be highly certain.
Regarding understated uncertainty: In the bygone days when I had an identifiable waist, I worked in the forecasting department of a large Midwestern utility. We did econometric forecasts of electrical energy sales and peak demands.
One day I had a bright idea: run one of our models on a subset of historical data and use the parameters thus found to predict the outcomes for the rest of the known period, then use the results to show the real uncertainty in the models. I showed the wider uncertainty bands to my boss and suggested that we show these to management along with our usual forecast.
He quashed the idea right away because, he said, if the bosses knew our forecasts had that much uncertainty, they would shut down the department.
Person of Choler,
What you say is true and is, I suspect, the reason why predictive methods have not taken hold. It is so much more comforting to be confident.
Pingback: William M. Briggs, Statistician » What do you guys look like?