William M. Briggs

Statistician to the Stars!

Page 3 of 427

Please Don’t Smooth Your (Social Media) Data!

Friends don't let friends smooth.

Friends don’t let friends smooth.

Don’t

Don’t smooth your data and then use that smoothed data as input to other analysis. You will fool yourself. You will make over-confident decisions. It is the wrong thing to do. It is a mistake. It is a guarantee of over-certainty. I don’t know how to put it more plainly. Lord knows I have tried. See below for a non-success story.

Smoothing means any kind of modeling, which includes running means, just-plain-means, filtering of any kind, regression, wavelets, Fourier analysis, ARIMA, GARCH; in short, any type of function where actual data comes in and something that is not data comes out.

Do not use the something-that-is-not-data as if it is data. This is a sin.

Don’t believe me. Try it yourself. The picture is from an upcoming paper I and some friends are writing.

It shows two simulated normal noise time series, with successively higher amounts of smoothing applied by a k-rolling mean. From top left clockwise: k = 1, 10, 20, 30; a k = 1 corresponds to no smoothing. The original time series are shown faintly for comparison. The correlation between the two series is indicated in the title.

More smoothing equals higher correlations. Since there are no causes between these series, the correlation should be hovering around 0, which it is in the first panel. And that correlation stays near 0—for the original real not fake un-smoothed data. But if you calculate the correlation between the smoothed series…the sky’s the limit!

Now it is not true that in each and every and all instances that smoothing will increase the correlation between two smoothed series. It might be that (in absolute value), for your one-time smoothing, correlation decreases or stays put. But it usually will increase, and usually by a lot.

Why? Imagine any two straight lines with non-zero slopes. These two straight lines will have perfect Pearson correlation, either +1 or -1. Regression and other measures will also show perfect agreement. The proof of this is trivial, and I leave it as an exercise (don’t be lazy; try it). Smoothing makes time series data look more like straight lines, as the pictures show. Simple as that.

There are all manner of fine points I’m skipping and would make wonderful Masters projects. Just what kind of data and what kind of smoothing and what statistical measures are affected and by what magnitude? All these questions are quantifiable and will make for fun puzzles. My experience with actual data and actual smoothing and typical measures shows that magnitude is large.

It happens

Now, without betraying any confidences, let me tell you of the latest in a long and growing string of bad examples. Two companies, one internationally known for their quantitative prowess, another even better known for its ability to make vast wads of money. Call them A (stats) and B (client). I did not work for either A or B, but know and advised certain parties.

B advertised and wondered how much of an effect this had on its measure of success. A said they could tell, using sophisticated Bayesian models incorporating social media data.

Social media!

Wowzee! Tell people you have busted open the secrets of social media and they will dump buckets of cold cash on you. Hint: everybody who says they have it figured out is either exaggerating to themselves or to their clients. (Say, that’s a pretty bold statement.)

Anyway, smoothing occurred. And correlations greater than 0.95 were boasted of. I’m not kidding about this number. Company A really did brag of enormous “impacts” of its smoothed measures. And Company B believed them—because they wanted to believe. Sophisticated Bayesian models incorporating social media data! How could you go wrong?

The real correlations, using unsmoothed data, were near 0. Just as you’d expect them to be for such noisy data as “social media” predicting a company’s measure of success. Do you really think Twitter streams contain magic?

I told all involved. I explained pictures like those above. I was emphatic and clear. I stood neither to gain nor lose regardless of the decision. Only two people (at B) believed me, neither of whom were in a position to make decisions.

At least I am comforted that Reality is my friend here. The company’s will eventually realize, but probably never admit, that their measures are spurious. Because they will realize but not admit, these measures will be quietly abandoned…

…As soon as the next computer self-programmed big data machine learning artificially intelligent smart-phone-data algorithm comes along and seduces them.

Summary Against Modern Thought: God ‘s Existence And Essence Are The Same

This may be proved in three ways. The first...

This may be proved in three ways. The first…

See the first post in this series for an explanation and guide of our tour of Summa Contra Gentiles. All posts are under the category SAMT.

Previous post.

Thinking cap times, folks. What God is and that He is are the same thing. I AM WHO AM.

Chapter 22: That in God existence and essence are the same

1 FROM what has been shown above, we may go on to prove that in God essence or quiddity is not distinct from His existence.

2For it has been shown above[1] that there is a thing which exists of itself necessarily, and this is God.i Now necessary existence, if it belong to a quiddity which is not that existence itself, is either inconsistent with or repugnant to that quiddity, as per se existence is to the quiddity of whiteness, or else is consistent or akin thereto, for instance that whiteness exist in some other thing. In the former supposition it will not belong to that quiddity to exist per se necessarily, for instance it becomes not whiteness to exist per se. In the second hypothesis, either this existence must be dependent on the essence, or both of them on some other cause, or the essence on the existence. The first two are in contradiction with the very notion of necessary per se existence: for if it depend on something else, it no longer exists necessarily. From the third supposition it follows that this quiddity is added accidentally to the thing which exists per se necessarily: because whatever follows on the essence of a thing is accidental thereto. Therefore God has not an essence distinct from His existence.ii

5…Further. Each thing exists by its own existence. Wherefore that which is not its own existence does not exist per se necessarily. But God exists per se necessarily. Therefore God is His own existence.

7…Moreover. Existence denotes a kind of actuality: since a thing is said to exist, not through being in potentiality, but through being in act. Now everything to which an act is becoming, and which is distinct from that act, is related thereto as potentiality to act: since act and potentiality are reciprocal terms. Accordingly, if the divine essence is distinct from its existence, it follows that His essence and existence are mutually related as potentiality and act. Now it has been proved that in God there is nothing of potentiality, and that He is pure act.[4] Therefore God’s essence is not distinct from His existence.iii

9…Further. Everything exists through having existence. Therefore nothing the essence of which is not its existence, exists by its essence, but by participation of something, namely existence. Now that which exists by participation of something cannot be the first being, because that in which a thing participates in order to exist, is previous to that thing. But God is the first being, to which nothing is previous.[6] Therefore God’s essence is His existence.iv

10 This sublime truth Moses was taught by the Lord: for when he asked the Lord (Exod. iii. 13, 14): If the children of Israel should say to me: What is His name? what shall I say to them? the Lord answered: I AM WHO AM….Thus shalt thou say to the children of Israel: HE WHO IS hath sent me to you; thus declaring His own name to be: HE WHO IS. Now every name is appointed to signify the nature or essence of a thing. Wherefore it follows that God’s very existence itself is His essence or nature.v

———————————————————————

iChapter 13. Always Chapter 13. Go and re-read it. Don’t be lazy.

iiRepeat that paragraph three times fast. It’s all there, but not put in the friendliest language to us, whose gray matter is accidentally much smaller than Thomas’s. Considering that paragraph’s density is what might have led him to re-write that proof in a happier way, in the Summa Theologica, Book 1, Question 3, Article 4, sed contra:

First, whatever a thing has besides its essence must be caused either by the constituent principles of that essence (like a property that necessarily accompanies the species–as the faculty of laughing is proper to a man–and is caused by the constituent principles of the species), or by some exterior agent–as heat is caused in water by fire. Therefore, if the existence of a thing differs from its essence, this existence must be caused either by some exterior agent or by its essential principles. Now it is impossible for a thing’s existence to be caused by its essential constituent principles, for nothing can be the sufficient cause of its own existence, if its existence is caused. Therefore that thing, whose existence differs from its essence, must have its existence caused by another. But this cannot be true of God; because we call God the first efficient cause. Therefore it is impossible that in God His existence should differ from His essence.

Secondly, existence is that which makes every form or nature actual; for goodness and humanity are spoken of as actual, only because they are spoken of as existing. Therefore existence must be compared to essence, if the latter is a distinct reality, as actuality to potentiality. Therefore, since in God there is no potentiality, as shown above (Article 1), it follows that in Him essence does not differ from existence. Therefore His essence is His existence.

Thirdly, because, just as that which has fire, but is not itself fire, is on fire by participation; so that which has existence but is not existence, is a being by participation. But God is His own essence, as shown above (Article 3) if, therefore, He is not His own existence He will be not essential, but participated being. He will not therefore be the first being–which is absurd. Therefore God is His own existence, and not merely His own essence.

Admit it: that is some pretty writin’! “[N]othing can be the sufficient cause of its own existence, if its existence is caused.” God is not caused, He is the necessary being, as we already learned. Thomas also refers to Article 3, “Whether God is the same as His essence or nature?”, in which he notes that in things composed of matter and form (like us and baseball bats) essence and existence differ, but in things “in which individualization is not due to individual matter”, and since God is not made of matter but does have an essence and since forms or universals don’t exist materially (we are not Plantonists here), God “must be His own Godhead, His own Life, and whatever else is thus predicated of Him.” What is predicated of Him, we are still to learn (Chapter 30).

Now St Thomas continues in Chapter 22, and so shall we, but omitting some of the now redundant material.

iiiThis is redundant, but I like it, because we ever need to emphasize the difference between potentiality and actuality.

ivBy being “first” being, Thomas does not mean God was caused. God’s existence is necessary. It’s not that everything else came “after” God, but that God is necessary for the existence of everything else. God is outside time.

v(Ellipsis original.) We don’t often include scriptural evidence, and even here we are not asking you to rely on it as a separate proof that essence = existence (we already have enough proof). But just look at what this nearly illiterate pre-computerized pre-Enlightenment “desert tribe” came up with! We have deduced from first principles the very same name of God as they knew! I AM = existence. WHO AM = essence. I AM WHO AM is equivalent to existence = essence, and so is HE WHO IS. Ain’t you amazed?! (I’m speaking in the voice of Captain Aubrey.)

[1] Ch. xiii.
[2] Ch. xviii.
[3] Ch. xiii.
[4] Ch. xvi.
[5] Ch. xviii.
[6] Ch. xiii.
[7] vii. 11.
[8] ii.

Humanae Vitae & The Synod: Theories And Predictions

Selfish genes theory does not predict these.

This originally ran 14 May 2014, but since this weekend is Number Six’s big show, I thought it well to have another look. The title is New & Improved! See the Update at the bottom.

Old predictions

Theories are useful to explain and to predict. Any theory can explain, but only true or likely true theories can skillfully predict.

For instance, Uncle Bob explains that your car won’t start because of Gremlins. His theory, which he drags out on all State occasions, does explain. But it is, as I hope is obvious, a theory which is useless to make predictions.

Two obvious examples. If you say the sun will peek above the horizon at 6:32:17 AM because gravitational theory says it will, and the sun does its duty, your theory has something going for it, especially if the theory makes lots of accurate predictions. And if you say, and say each year for two decades, that the planet’s global average temperature will soar to “unprecedented” heights, yet the temperature misbehaves and stays put, you’re theory is likely false.

Now how about these predictions, made in 1968, on what would happen were contraception to be embraced (which, of course, it has been). This embracement will:

  1. “[O]pen wide the way for marital infidelity and a general lowering of moral standards”. Nailed it.
  2. Especially in the young, “[A] man who grows accustomed to the use of contraceptive methods may forget the reverence due to a woman”. Nailed it.
  3. He will “reduce her to being a mere instrument for the satisfaction of his own desires”. Hooked-up nailed it.
  4. He will “no longer considering her as his partner whom he should surround with care and affection.” Nailed it.
  5. “Finally, careful consideration should be given to the danger of this power passing into the hands of those public authorities who care little for the precepts of the moral law. Who will blame a government which in its attempt to resolve the problems affecting an entire country resorts to the same measures as are regarded as lawful by married people in the solution of a particular family difficulty?” Oh my, oh my, is that one nailed.
  6. “Who will prevent public authorities from favoring those contraceptive methods which they consider more effective?” None, that’s who: another hit.
  7. “Should [the government] regard this as necessary, they may even impose their use on everyone.” HHS mandate, anybody? Nailed it again.
  8. “It could well happen, therefore, that when people, either individually or in family or social life, experience the inherent difficulties of the divine law and are determined to avoid them, they may give into the hands of public authorities the power to intervene in the most personal and intimate responsibility of husband and wife.” Smack! Pow! Wow! Capital-N ailed it.

Who is this guy, this prescient sage, who, drawing from some theory, foretold the world of 2014 so accurately? Well, his name was Paul, and as he came from a long line of Pauls in the same Institution to which he was appointed Leader, he called himself Number 6. One thing we know, given his batting average, we should accord the theory which created these predictions pretty high weight.

Right?

Number 6’s theory also explains the popularity of divorce, out-of-wedlock births, and the rise in the belief of individuals’ “unlimited dominion” over “his specifically sexual faculties.” Number 6 didn’t actually specify these as predictions, though, taking them “as read.”

History

Now the most fascinating thing about these predictions is how they came to be made. What happened was this. Number 6’s predecessor called on a Commission of experts, who met and deliberated from 1963 to 1966 (Number 6 boosted Commission membership halfway through), giving a report to Number 6 two years before he made his predictions. Nobody was in any rush.

The Commission was loaded with sober academics and had the support of a good portion of the leadership of Number 6’s Institution. Word leaked out, as word always leaks out, about the Commission’s efforts and opinions, and this excited popular and media support. The Commission, not wanting to be on the wrong side of history, favored contraception. After all, this was a different world than that world which came before this world: or something.

After several years of glowing expectations, most expected Number 6 to endorse the Commission’s report. He did not.

Boy!, did tempers flare. To say the free-for-all crowd were displeased is a massive understatement. Number 6 and his Institution were ridiculed in the press and in academia and, indeed, by some leaders in Number 6’s Institution. One academic member of the Commission called Number 6’s predictions “that horrible document.” A prominent leader in Number 6’s Institution publicly charged Number 6 with “an anti-collegial act”. Ouch.

That fellow was far from alone. Many other leaders and groups of leaders castigated Number 6 openly. These dissidents went so far as to tell the common folk to ignore Number 6 and do what they please. And they did. And where they did do as they pleased, it was found that the Institution lost members.

Of course, it is not often remembered, perhaps willfully, that Number 6’s theory made stunningly accurate predictions, whereas his enemies’ theories made inaccurate ones.

The reason for that digression is important because again Number 6’s Institution will meet to discuss matters pertaining to human sexuality and the family. The meeting will go for at least two years. Experts will be confided in. Reports will be written.

As before, the press and a sizable chunk of leadership is on the side of liberalization; they particularly favor giving the nod to divorce but also to so-called same-sex marriage and perhaps even abortion. The world has changed, these people say, and therefore the Institution must also change—to become something that is not the Institution.

The Institution’s new leader Francis is being groomed by the liberalizers as the man with the plan, as somebody who is willing to set aside the old truths for new ones. These new dissidents are in the habit of parsing every public word of Francis’s to find in them support for their new truths. So adept are they at this that almost before Francis is done speaking, a news item or blog post is up saying, “Change we can believe in is coming!”

New predictions

My guess, working within Number 6’s theory, is whichever leader is in charge after the family synod is over will support tradition. The ban on contraception will be upheld. Marriage will be, as it can only be, declared to be between one man and one woman. Sodomy will still be a sin. Divorce will still be forbidden and not supported in the Institution’s activities. Abortion, if mentioned, will still be condemned.

The howling which will greet the announcement that there cannot be new truths, but only Truth, will be wondrous to behold, especially since, as before, liberalizers expect the vote to go with them. New dissidents will arise who, again as before, will tell people to ignore official proclamations and do what they want.

What will happen to rebellious families is obvious: more of what Number 6 said, a decrease in interest in marriage, increased state control over all things sexual, recognition that children belong to the state and not “parents”, and because of the dissolution of the family, an increase in support of euthanasia.

And people, even seeing the accuracy of these predictions, will still largely disbelieve the theory.

Update 14 May. It’s coincidence day at WMBriggs.com: Are Our Relationships Threatening The State?

Update 18 October 2014. Not that I want to brag, but it looks like the Truth Theory is still holding strong. But what a week!

The question is whether, after the conclusion of next year’s synod, Pope Francis will emulate his brother Number 6, or will he seek more worldly pastures?

I predict the former, in the following sense. I say Tradition holds, whether Pope Francis wants it to or not. It won’t matter what he or what anybody else wants, sin will still be sin. Doctrine will remain unchanged. Now, how will this Great Continuance happen? I have no idea. But I am reminded of the tale of Arius, a bishop who led one of the Church’s earliest heresies, a man whose power of convincing other Church fathers waxed and waned, but which never deserted him, and who, on his way to a final crucial meeting where he might have convinced others of his fallacy, had this happen to him:

It was then Saturday, and Arius was expecting to assemble with the church on the day following: but divine retribution overtook his daring criminalities. For going out of the imperial palace, attended by a crowd of Eusebian partisans like guards, he paraded proudly through the midst of the city, attracting the notice of all the people. As he approached the place called Constantine’s Forum, where the column of porphyry is erected, a terror arising from the remorse of conscience seized Arius, and with the terror a violent relaxation of the bowels: he therefore enquired whether there was a convenient place near, and being directed to the back of Constantine’s Forum, he hastened thither. Soon after a faintness came over him, and together with the evacuations his bowels protruded, followed by a copious hemorrhage, and the descent of the smaller intestines: moreover portions of his spleen and liver were brought off in the effusion of blood, so that he almost immediately died. The scene of this catastrophe still is shown at Constantinople, as I have said, behind the shambles in the colonnade: and by persons going by pointing the finger at the place, there is a perpetual remembrance preserved of this extraordinary kind of death.

The answer thus comes via Peter Kreeft, who is fond of quoting a Southern Baptist minister who managed to sum up the lessons of the Bible in four words. “I’m God. You’re not.”

Global Warming’s Shark Jumping Moment

Weeeeeeeeeeeeeee!

Weeeeeeeeeeeeeee!

Travel day today, so something light and airy—and incomplete.

President Obama, presumably sacrificing a tee time, went to keyboard and typed these words:

This is a big moment in the fight against climate change—stick it to climate change deniers by adding your name: http://t.co/fkCzkiMhFw

Now you may think the problem lies in the words stick it to climate change deniers, but you would be wrong. Mr Obama is a politician and stick it is political language, and typical language at that. It is the weak man who allows himself to be offended easily. Add to that that it probably wasn’t our dear leader but some staffer or other hack who wrote the tweet, then there is nothing unusual with stick it.

No. The shark jump is in adding your name. Add your name to what? A petition.

Climate science by petition! While global warming has always had a thick dressing of ideology, this converts the matter completely from science to pure politics. What we have here is the triumph of pragmatism and democracy, the disastrous idea that truth can be had by vote, finally applied to science. Here’s the petition pleading:

Deniers and deep-pocketed polluters make it pretty hard to get anything done on climate change—but here’s one meaningful way you can fight them: The EPA is collecting comments on President Obama’s climate plan, and it’s our chance to show public support.

This is one of the decisive moments in the fight against climate change. Collecting comments gives the EPA a chance to see what ordinary people have to say about this important issue. (Don’t worry—they hear from the special interests on every day that ends in Y.)

What ordinary people have to say about global warming is, as far the EPA goes, meaningless. What value is it to collect the opinions of those unaware of fluid physics about the value of that physics? Of course, it is of some use to ask people how potential regulations might affect them, but about the science behind the regulations there is none.

Yet BarackObama.com says

The other side thinks they can win this fight simply by shouting the loudest, and they have a lot of money to back it up. What they don’t have is a whole lot of people—genuine voices standing up for what’s right. And we’ve proved time and again that, when we raise our voices together, we can take on even the most powerful interests.

Forget the canard about money (I’m still waiting for my check from Big Oil), and forget the idea that the people currently in power need to fight “powerful interests”. What is the point of “voices”? Will these voices fix model parameterizations? Will crowd wisdom tell us the proper role of cloud feedback? Should we turn to social media to guess proxy temperature reconstructions?

I’m writing in a hurry and can’t develop the idea, but somehow I’m put in mind of those early sci-fi movies, where groups of concerned scientists and government officials would gather around a table to discuss the alien giant bug crisis. Sleeves were rolled up. Uniforms worn. Discussions ranged. But never once was there the idea of putting ideas to a vote among the populace.

Incidentally, if you add your email (I made one up) to the Science Petition, you are asked to give money to Mr Obama to “help defeat the dirty special interests”. Sheesh.

That Curious Synod Relatio

The officiala seala.

The officiala seala.

The extraordinary Synod released into the wild Monday its draft relatio which caused a lot of emotions to soar. The Left cheered, “This is the beginning!”, while the Right said, “This is the end!” (Both could be correct, of course.) Look for a spike in both champagne and whiskey sales as giddiness takes over.

One paragraph (#50) in particular captured the imagination of the world:

Pedophiles have gifts and qualities to offer to the Christian community: are we capable of welcoming these people, guaranteeing to them a fraternal space in our communities? Often they wish to encounter a Church that offers them a welcoming home. Are our communities capable of providing that, accepting and valuing their sexual orientation, without compromising Catholic doctrine on the family and matrimony?

No, wait. Something wrong with the keyboard. My bad. It actually read:

Adulterers have gifts and qualities to offer to the Christian community…accepting and valuing their sexual orientation…

Darn it. This new WordPress dynamic editor kills me. Try again:

Murderers have gifts and qualities to offer to the Christian community…accepting and valuing their furious raging temptations…

That’s not it either. Never mind. Since I can’t seem to get the editor to work, meanwhile you can link over the original, uncorrupted text.

Anyway, I don’t see why all the steam. Of course pedophiles, adulterers, and murderers are people, too! So are tax cheats, arsonists, and members of Congress. Listen, sisters and brothers, we’re all sinners needing the Grace of God, a grace we should beg with fear and trembling. That’s all the paragraph is trying to say.

This is no earthquake. There really isn’t anything to see here. Move along.

Does seem a bit unfair to all the other sinners to highlight the “gifts and qualities” of just one group, though. People are going to feel left out. Right now some little old lady is sitting in the back of the pews thinking, “All I do is gossip about my family members, which I know is sinful and hurtful, and I don’t even get one lousy paragraph?” The only answer we can give her is: Whoever said life was fair?

Just a moment. I think I have it…yes. The editor is fully functional again. Here’s #21.

The Gospel of the family, while it shines in the witness of many families who live coherently their fidelity to the sacrament, with their mature fruits of authentic daily sanctity must also nurture those seeds that are yet to mature, and must care for those trees that have dried up and wish not to be neglected.

Reactionary groups charged that the bishops cribbed this from the inside of a “Just Because” greeting card bought at a ₤100 store back in 1999, and reportedly found earlier this week between the seat cushions of a couch in Cardinal Wuerl’s hotel room. In answer to these serious allegations, Catholic blog watchdog Team Patheos said, “Reactionaries are poopy heads.”

I frankly don’t see it as a big deal. These bishops are busy guys and they only had a day, day-and-a-half to complete the relatio. They were thus in a position of a college student coming off a Thursday night with a term paper due next morning, and there sits before him Wikipedia in all its tempting glory. Besides, it’s the thought that counts, not the source, right?

The greeting-card crib may not have been the only skullduggery. A highly placed anonymous Source and self-described Zed-Head was able to tell your intrepid reporter that he suspects portions of the relatio were inserted by Chinese hackers who broke into the Vatican computers.

“How else do you explain phrases like paragraph nine’s ‘a greater need is encountered among individuals to take care of themselves, to know their inner being, and to live in greater harmony with their emotions and sentiments, seeking a relational quality in emotional life’ and thirteen’s ‘interpreting the nuptial covenant in terms of continuity and novelty’?” he asked.

“What have these to do with Catholic teaching and the salvation of souls? There is no way any faithful, conscientious bishop would have written these things. It must be from hackers.”

« Older posts Newer posts »

© 2014 William M. Briggs

Theme by Anders NorenUp ↑