William M. Briggs

Statistician to the Stars!

Page 146 of 684

Can We Predict The Unpredictable?

Figure 1 (b) from the paper.

Figure 1 (b) from the paper.

The most intriguing thing about the new peer-reviewed paper of the same name as today’s post in Nature: Scientific Reports by Abbas Golestani and Robin Gras is that it is longer than the one word it takes to authoritatively answer the question.

No. We cannot predict the unpredictable.

Nor do we do that good a job at predicting the predictable, as anybody who has ever spent time with models of complex systems like the stock market and global climate system realize (or should realize).

Those who have long been at this game have seen every form of hype and promise come down the pike, each new method touted to fix the shortcomings of the previous wonder. Yet the glory always fades.

Anyway, our authors have devised a “novel” algorithm which they christen GenericPred, which fixes the shortcomings of models such as ARMIA, GARCH, and MLP. Before describing it, at the top of this post is a picture of one of the predictions of the Dow Jones Industrial Average.

We don’t need to understand how forecast “goodness” was defined to be able to see that GenericPred handily beats three other standard methods. Two other pictures show similar performance for other periods of the DJIA. The GenericPred also seems to do well predicting (one version of) the global average temperature anomaly (it predicts an increase of a whopping ~2.5C over the next century).

Looking only at these pictures, it appears GenericPred is hot stuff. Obviously, the old-fashioned methods bit into the seeming increase of the DJIA (pictured above) and were burnt. But not GenericPred. No, sir. It—somehow—sensed that changes were coming and, more or less, nailed the decrease.

I don’t buy it. I may be wrong—yes, dear reader, it is possible that I am mistaken—but I don’t think so. My misgivings flow from the nature of their algorithm and its relation to chaotic signals. But, hey, if I’m wrong and these guys really can predict the stock market two years in advance, they’ll be billionaires in short order and I’ll still be running this penny-ante blog.

GenericPred is based on a good idea, which I’ll roughly summarize. Take a series y1, y2, …, yT and compute some measure of chaos, like a Lyapunov exponent. Now posit new values of the series, yT+1, yT+2, …, yT+m, and then compute the (say) Lyapunov exponent for the augmented series y1, …, yT+m. Pick those values of yT+1, …, yT+m that minimize the distance between the (say) Lyapunov exponent of the original and the augmented series.

This makes some sense because, if the series is indeed chaotic, and no external changes in the causes of this series are expected, then the nature of that series, as measured by various indexes, should remain constant. That there are or can be external changes in causes is obvious, and is why the authors look before and after the last financial “crisis.”

Still with me?

Now if you have had any experience with chaotic time series, you know that, like the DJIA, they are apt to fly off hither and yon. Chaotic signals are those that are caused (as is every time series) but which are sensitive to initial conditions. The weest perturbation at the beginning (or really anywhere) could and does result in wildly different values now. This is why they are so difficult to predict.

Take another look at the prediction picture above. The data before (on or about 1 July 2007) were used to fit the various models, and the predictions came afterward. What would this plot look like if the authors had used date up to, say, 29 June 2007? The old-school models would probably still stink. But it’s not at all clear GenericPred would do as well—because the series is chaotic. Chaotic signals are just as likely to rise as fall. Very curious they got it so startlingly right.

And could something as humble as the Lyapunov exponent (or some other univariate measure) really hold the secret to all possible future values of the stock market?

Every scientist wants to think the best of his creation. Could these researchers have played (in honest earnestness) with the dates to show us the most dramatic discrepancies between their model and the old-school methods? Since these series are chaotic it would be truly remarkable if the method weren’t sensitive to the dates chosen to model and forecast.

The pictures look a little too neat, the points of departure of the old-school models and GenericPred a little too sharp. I do not suggest any nefariousness. But if this new method is to follow the history of all other methods, and prove to be not as exciting as initially promised, it will be because the authors fooled themselves by tinkering one step too many to present their best case.

Anyway, I predict at least a brief GenericPred boom among prediction clients. If the model works as well as promised by these figures, then the authors ought to get rich off the stock market.

Incidentally, all predictions methods should be accompanied by measures of uncertainty. The old-school methods automatically give these, but GenericPred does not. That should be the authors’ next step.

Update One of the authors (Gras) responded in comments below.


Thanks to reader Rich Kyllo for bringing this paper to our attention.

Climate Change Causing Short Peruvians!

The authors' Figure 1.

The authors’ Figure 1.

Who remembers those crank scientists who wanted to genetically engineer human beings so that they would be sprier and narrower and thus have smaller “carbon footprints”? We don’t need ’em!

At least, not according to the peer-reviewed paper “El Niño adversely affected childhood stature and lean mass in northern Peru” in the new and exciting journal Climate Change Responses—it looks like it will prove fertile pickings for posts about bad statistics—by Heather Danysh and a handful of others.

Before we begin, let us warn ourselves against height triumphalism. Picking on short people is wrong and hurtful. We would never use terms like “height deficits” like the cruel authors of the paper do.

To the science! The hypothesis: “Due to severe food shortages and increased incidence of infectious diseases during El Niño, we hypothesized that children born in northern Peru during and after the 1997–1998 El Niño may be shorter for their age and sex than children born in other years.”

(Incidentally, there are more typos per paragraph in this article than in any post of mine. May give some indication of this journal’s quality.)

The authors were keen on height-for-age, or HAZ, and bioelectrical impedance. Impedance is used to guess the amount of lean mass versus water, a method which requires some form of calibration and which always gives an answer which is not certain and thus which should always be accompanied by some measure of uncertainty. This, of course, did not happen here. Keen readers should delve into the paper for their description of a “flood likelihood score” (instead of measuring whether kids lived through a flood, they calculated an index to guess: the Epidemiologist fallacy lurks).

Kids born between 1991 and 2001 were measured. The authors used a “linear mixed model” to explain “height in later childhood” on these “variables”: years between birth date and January 1991, and “the number of years between the child’s birth date and the onset of El Niño for the each child born after the onset of El Niño” in 1997.

Now 1997-1998 was, according to the Oceanic Niño Index (ONI), a strong event. But moderate events were seen in 1991-92, and 1994-95. And 1995-96 was classed as a weak La Niña event, and 1998-2000 moderate to strong La Niña.

So we have a mixture of signals here. What about dose-response? Did kids get taller during La Niñas?

The authors say, “All regression models were adjusted for sex, socioeconomic status (SES) index, and likelihood of living in a flood-prone household, and accounted for clustering using a random intercept by village.” Regression is always abused like this: regression does not prove causality. Say, does that SES index have uncertainty? Skip it.

The top of the post shows the author’s Figure 1. Does it look to you like that 1997-98 event shrunk kids (actually, the HAZ index)? Looks to be like the “rapid urbanization” the authors noted in their study village since 1991 might account for the generally increasing heights, no? And, hey, isn’t this a mean index with no indication of spread (not every kid had the same HAZ), i.e. a graph which is uncertain presented and used in analyses as if it were certain, a move which guarantees over-certainty? I’m just asking.

Actually, the authors’ model acknowledges the increasing (too sure) HAZ! Read that twice. Yet a wee p-value confirmed that kids born after the 1997-1998 event were increasing in HAZ at a rate slightly smaller than kids before it.

So even if the authors’ hypothesis is right, it’s not that El Niño shrunk kids, it only slowed the observed rate of increase. The increase is still acknowledge to be there. Popular accounts like Mother Jones‘ “Another Side Effect of Climate Change and El Niño Events? Shorter Kids” are thus wildly wrong. Who could have guessed?

But there is plenty of doubt about that hypothesis. Somehow El Niño knew only to effect lean and not fat mass of the kids. Wee p-values confirmed “children born after the onset of El Niño have significantly less lean mass than what would be expected if El Niño had not occurred” (forgetting that we didn’t carry uncertainty of the impedance, SES, or HAZ measures forward). Or maybe urbanization produced fatter, better-fed kids?

Now if the authors were to properly carry the uncertainty of all their measures forward, and they properly took into account what kids actually ate, it’s almost certain the wee p-values would become un-wee, thus canceling all headlines.

Yes, the Epidemiologist Fallacy has struck again. El Niño cannot cause shrunken kids. Starvation can—and so can a host of other things. Yet none of these things were measured here. None.

It’s well past the time for anybody to take seriously these Oh-My-God correlation studies. Science should be the search for causes, not the production of goofy statistics.


Summary Against Modern Thought: Of The Divine Perfection

This may be proved in three ways. The first...

This may be proved in three ways. The first…

See the first post in this series for an explanation and guide of our tour of Summa Contra Gentiles. All posts are under the category SAMT.

Previous post.

We’re still on our quest to define what God is.

Chapter 28: Of The Divine Perfection

1 Now although things that exist and live are more perfect than those which only exist, yet God Who is not distinct from His own existence, is universally perfect being.[1] And by universally perfect I mean that He lacks not the excellence of any genus.i

2 [This may be skipped.] For every excellence of any being whatsoever is ascribed to a thing in respect of its being, since no excellence would accrue to man from his wisdom, unless thereby he were wise, and so on. Wherefore, according as a thing has being, so is its mode of excellence: since a thing, according as its being is contracted to some special mode of excellence more or less great, is said to be more or less excellent. Hence if there be a thing to which the whole possibility of being belongs, no excellence that belongs to any thing can be lacking thereto. Now to a thing which is its own being, being belongs according to the whole possibility of being: thus if there were a separate whiteness, nothing of the whole possibility of whiteness could be wanting to it: because something of the possibility of whiteness is lacking to a particular white thing through a defect in the recipient of whiteness, which receives it according to its mode and, maybe, not according to the whole possibility of whiteness. Therefore God, Who is His own being, as shown above,[2] has being according to the whole possibility of being itself: and consequently He cannot lack any excellence that belongs to any thing.ii

3 And just as every excellence and perfection is in a thing according as that thing is, so every defect is in a thing according as that thing in some sense is not. Now just as God has being wholly, so is not-being wholly absent from Him, since according as a thing has being it fails in not-being. Therefore all defect is removed from God, and consequently He is universally perfect…iii

5 Again. Every imperfect thing must needs be preceded by some perfect thing: for seed is from some animal or plant. Wherefore the first being must be supremely perfect. Now it has been shown[3] that God is the first being. Therefore He is supremely perfect.

6 Moreover. A thing is perfect in so far as it is in act, and imperfect in so far as it is in potentiality and void of act. Wherefore that which is nowise in potentiality but is pure act, must needs be most perfect. Now such is God.[4] Therefore He is most perfect.iv

7 Further. Nothing acts except according as it is in act: wherefore action follows upon the mode of actuality in the agent; and consequently it is impossible for the effect that results from an action to have a more excellent actuality than that of the agent, although it is possible for the actuality of the effect to be more imperfect than that of the active cause, since action may be weakened on the part of that in which it terminates. Now in the genus of efficient cause we come at length to the one cause which is called God, as explained above,[5] from Whom all things proceed, as we shall show in the sequel.[6] Wherefore it follows that whatever is actual in any other thing, is found in God much more eminently than in that thing, and not conversely. Therefore God is most perfect.v

8 Again. In every genus there is some thing most perfect relatively to that genus, by which every thing in that genus is measured: since every thing is shown to be more or less perfect according as it approaches more or less to the measure of that genus: thus white is said to be the measure in all colours, and the virtuous among all men.[7] Now the measure of all beings can be none other than God Who is His own being. Therefore no perfection that belongs to any thing is lacking to Him, otherwise He would not be the universal measure of all…vi


iAs the song says, dead puppies aren’t much fun—and they are surely less excellent as puppies than live ones. Here is the crucial idea: the lack of the good is the lack of perfection. Which is to say (in brief), evil is the lack of the good, and the good is perfection. It’s not that God is lacking some thing, but since His essence and existence are the same, He is perfect: nothing could be added to make his essence more perfect. The word perfect does not here retain its popular meaning: it is a technical term.

iiI considered leaving this one out as it is, sort of, a more long-winded repeat of argument 1. But I left it because of this sentence: “…if there were a separate whiteness, nothing of the whole possibility of whiteness could be wanting to it: because something of the possibility of whiteness is lacking to a particular white thing through a defect in the recipient of whiteness, which receives it according to its mode and, maybe, not according to the whole possibility of whiteness.” This is an analogy. If whiteness could somehow per impossible exist on its own, it could not lack in whiteness, right? It would, in the sense of the word above, be perfectly white. Now things which are meant to be white, say a certain flower, which are somehow not white are that way due to some imperfection, an imperfection that cannot be caused by the perfection of whiteness. And the same with God being being-itself.

iiiIsn’t that pretty? Since God is being-itself, He cannot be in any part not-being, therefore he isn’t lacking that which the good of His essence demands. Thus He is perfect—in this sense. We still have lots of work to do to detail the consequences of this.

ivThese last two follow simply from what was proven before.

vCauses must be at least as great as their effects: causes are more excellent than effects. And we’re right back at Chapter 13, which is ever required reading. Don’t skimp here: review, review, review. Book 2, mentioned in footnote 6, is still about 75 chapters off.

viVirtue is what we’re aiming at; most of us, including Yours Truly, are lacking in this regard. But don’t miss the main point: “…since every thing is shown to be more or less perfect according as it approaches more or less to the measure of that genus”. Since imperfection abounds, it’s obvious that inequality is part of the (fallen) system. But since imperfection varies as the distance from the measure of a genus, we have natural goals, things to aim at. It is possible and necessary to measure ourselves against a standard. In other words, though one many may be more virtuous than another, goodness itself is not ultimately relative. We all aim at the same standard.

[1] Sum. Th. P. I., Q. iv., A. 2.
[2] Ch. xxii.
[3] Ch. xiii.
[4] Ch. xvi.
[6] Ch. xiii.
[7] Bk. II., ch. xv.
[8] 3 Ethic. iv. 5; v. 10.

This Is Why Jesus Will Separate The Sheep From The Goats

And all nations shall be gathered together before him, and he shall separate them one from another, as the shepherd separateth the sheep from the goats: And he shall set the sheep on his right hand, but the goats on his left.

Since today is usually the slowest blogging day of the year, we may as well discuss those nasty creatures, goats. The video proves it’s no wonder they butt into so many cheesy jokes. Tasty in tacos, though—and in spiritual metaphors, too.

The sheep-and-goats figure-of-speech reminds me of the latter big experiments in parapsychology during its heyday. A long string of negative results, bust after painful bust, in the late 1970s led some ingenious fellow to birth the theory that skeptics (goats) harnessing evil psi-dampening rays were quashing the psychic elements of the mentally gifted (sheep). How, nobody knew. Oddly, skeptics could do this without even knowing they were so engaged. Peer-reviewed papers were written on the subject. Incidentally, all used statistical arguments.

Reminds you of global warming research, does it not? The closer skeptics look, the faster the CO2 signal recedes into the distance. And so true believers, unable to discern any other cause of their failures, lash out and gibber at their enemies. “Denier!” “Science has spoken!” Are these folks better classes as unruly unpleasant goats or easily led sheep? Have to sort that one out.

The beast at time marker 3:27 reminds me of the ship Surprise’s irascible goat Aspasia, who one day gave Dr Maturin the stink eye and “defecated with intent”. The Surprise is the ship often featured in the greatest novel of (in?) the English language by Patrick O’Brian, written in 20.2 installments starting with Master and Commander and ending with an unfinished holograph owing to the death of the author. I cannot praise this novel too highly. I’m not sure anybody else can either.

And this remarks reminds me of Thursday’s post in which, in respect of Little Big Man, and a sentiment repeated again here, that you should skip the movie, which has almost no relation to the book except for a vague similarity in names of some of the characters. No movie can be a book, but no movie should express the opposite intent and moral of its parent book. The movie had too many anachronisms.

The directors of these films are thus goats. Which reminds me that calling somebody a “goat” is an insult. Strangely, goats are people in the doghouse. Figure that one.

Speaking of being in the doghouse, this reminds me of yesterday’s news that James Watson, winner of the prize which civilians see as the pinnacle of scientific achievement, who announced “he is selling the Nobel Prize medal he won in 1962 for discovering the structure of DNA because he has been ostracised and needs the money.”

Ostracized? Yes, sir. “…[the journalist] somehow wrote that I worried about the people in Africa because of their low IQ — and you’re not supposed to say that.” No, sir, you’re not.

Which reminds me of another goat insult from this curious movie: son of a motherless goat. If anybody ever notices the racial connotations of this movie, watch out. The three actors would become instant goats. Purged forevermore. I’m not sure if any of them have won Oscars, but they’d have to put them into the same auction as Watson’s Nobel.

And this reminds me that the opposite of a goat is a lamb. And thus, given his metaphorical nature, we see the affinity our Lord has with other sheep. Don’t be a goat. Unfortunately, what the World thinks is a goat is often a sheep. Don’t become confused.

« Older posts Newer posts »

© 2016 William M. Briggs

Theme by Anders NorenUp ↑