There is a belief—a persistent and beguiling, yet false belief—that a formula exists that can predict anything. This formula will differ, it is thought, by the thing predicted, and it is certain that once the formula is fed into a computer, the future is ours to see.
You might have guessed that our experience with, at the least, quantum mechanics and chaos would have beat this notion out of our heads. But you would have been wrong. (Philosophical note: I do not claim that the future is unforeseeable by any means, but I do claim that the standard, mathematical, mechanistic view of the universe is an unproductive route for augury.)
For a symptom of over-confidence, take this example provided by reader Michael Kubat:
Before his sentence, the judge in the case received an automatically generated risk score that determined Loomis was likely to commit violent crimes in the future.
There are no such things as risk scores; they don’t exist, at least not in the sense implied by this sentence, which is taken from the article “This Guy Trains Computers to Find Future Criminals” at Bloomberg. Just like you don’t have “a probability” of being stung by a bee or hit by lightning, you do not have a risk or chance or probability that you’ll break the law.
Why? All probability, all risk, all chance is conditional. In order to have “a” probability, probability itself would have to be unconditional. Put it this way. You have “a” height, which though the units it’s expressed in are relative, is real and measurable (though even height is dependent on conditions; think of measuring yourself near a black hole, where your stature would be diminished).
Here’s another example. What’s “the” probability of drawing the Jack of Hearts? There isn’t one. There is if you assume the conditions there are fifty-two cards in a deck, just one of which is the Jack of Hearts and when drawn only one card will show. But if you change the conditions to there are twenty-two cards, etc., the probability changes (and dramatically).
All this means that it’s possible to predict this Loomis will commit a crime, but only conditional on some ad hoc model. Bloomberg says:
Risk scores, generated by algorithms, are an increasingly common factor in sentencing. Computers crunch data—arrests, type of crime committed, and demographic information—and a risk rating is generated. The idea is to create a guide that’s less likely to be subject to unconscious biases, the mood of a judge, or other human shortcomings.
Algorithms are done on computers!
The “unconscious bias” so fretted about won’t be present in the judge who relies on an ad hoc model, but it will exist in the ad hoc model, or the creators of that model. What happens is that everybody thinks the algorithm is unbiased, but this is simply impossible. The bias are the conditions, and conditions must exist for any algorithm to exist.
Richard Berk from the University of Pennsylvania, “a shortish, bald guy” and statistician, is one of the folks pushing the false view of model prowess. “Berk wants to predict at the moment of birth whether people will commit a crime by their 18th birthday, based on factors such as environment and the history of a new child’s parents.”
Of course, he can make such predictions; anybody can. You can, based on patterns in the scatter from your Fruit Loops. Whether these predictions have any skill (a word I used in its technical sense) is another matter entirely. Bloomberg rightly says Berk’s models “makes people uncomfortable”, which they should. The danger is that they’ll be assumed to be better than they are because they were made by Science on Bias-Free Computers using Machine Learning. Machines that can learn!
Accuracy? “Berk says that in his own work, between 29 percent and 38 percent of predictions about whether someone is low-risk end up being wrong.” Is this from prediction of entirely new data, or from the model fit? A guess is the later. Anyway, these dismal accuracies are not from at-birth predictions, but are contemporaneous. Predictably (get it?), Berk says “focusing on accuracy misses the point”. Yeah, sure it does. Here’s the frightening bit:
When it comes to crime, sometimes the best answers aren’t the most statistically precise ones. Just like weathermen err on the side of predicting rain because no one wants to get caught without an umbrella, court systems want technology that intentionally overpredicts the risk that any individual is a crime risk.
No no no no no no no no no! No. No. Rubbish. Rot. Nonsense.
What is always wanted is, given the conditions (i.e. the model), the actual probability of the event. No two people make the same decisions based on the weather report, and any shading of the probability one way or the other is, in effect, making the decision for somebody. Same goes for predicting pre-crime, where the decisions are (usually) more consequential. Accuracy always matters.
I have much more on these subjects in Uncertainty: The Soul of Modeling, Probability & Statistics.