William M. Briggs

Statistician to the Stars!

Page 5 of 758

Trend Models & The Inability To Discover Cause

The plot atop this post is one of an observable, the cause of which is known to be related to a variety of physical, chemical, biological, geographical, and even sociological factors. The data is real, but I’ll disguise its source. All I can say is that it is measured out-of-doors. The points are yearly averages, from 1993 to 2014, of the same observable which itself is measured at hourly intervals (i.e. each point is the numerical mean of about 365 x 24 points; there is the odd missing value, and there are leap years).

It is obvious, I hope, that the values of (let us call it) X have changed. It is equally obvious that some thing or things caused X to change. In fact, since the points of X shown are averages of hourly measurements, there must be many (to say the least) causes of each point.

Thus far, this time series is like myriad others.

Now, it turns out that at some year a policy was changed such that some of the underlying physical, chemical, biological, geographical, and sociological were required to be changed. The human controllable parts, that is; this was a one-time change. The natural question, and a good question, is how much effect did the policy change have?

The answer is: nobody knows and we can’t tell with certainty, nor with anything like certainty. The short reason why we can’t know is because we don’t know the causes of X, therefore we don’t know how the policy changed all the causes of X, therefore we have no solid idea why X changed.

Reversing that, supposing we did know the causes of X, then we (of course) would know why X took each value, and we therefore would know how the causes change due to the policy, and therefore we would know the exact changes in X if the policy were in place or if were it not.

This much should be obvious. But there has developed a tradition in classical (mainly frequentist, but also Bayesian) statistics which purports to reveal the changes in X due to a change in policy. I don’t want to delve deeply into the statistical methods, since our purpose is philosophical. I’ll instead say it was asserted that because there was measured a “statistically significant” upward trend after the policy was implemented, that therefore the policy had worked.

This cannot be so. Here is why. Here is a picture of the difference in trend coefficients from linear regressions, first regressing 1993 versus 1994-2014, then 1993-1994 versus 1995-2014, and so on, up to 1993-2013 versus 2014. The points are colored red if the p-value associated with the difference in trend coefficients is wee, else it is black if the p-value is not wee.

It isn’t quite the same, but if you have heard of “tests for differences in means” (always a misnomer), think of those. For example, the first test is for the mean of 1993 (which is just 1993) versus the mean of 1994-2014, and so on. The difference in means (or thereabouts) are the dots. Take 2000: the red dot says the mean (or thereabouts) of X from 1993-2000 was “significantly” lower than the mean of X from 2001-2014.

Now the mean of 1993-2001 was also lower than the mean from 2002-2014. It was not “significantly” lower. But was it lower? Yes, sir, it was. Some thing or things caused the differences in means in both cases. The interpretation of “significance” is one of two things, both of which are always (as in always) wrong.

(1) A classical statistician will say some real thing caused the change from 1993-2000 to 2001-2014, and he will also say “chance” or “randomness” caused the change from 1993-2001 to 2002-2014. Chance and randomness are not physical, they have no causal powers; therefore, they cannot cause anything. When this is pointed out, the statistician will then say that he does not know what caused the 1993-2001 to 2002-2014 shift. And that is fair enough, because typically he will not know. Yet here we do know there multitudinous causes which, we can infer, must have changed in magnitude between the two time periods. Where or when or how we cannot say.

Yet pause. The statistician is obliged to say some real thing caused the shift from 1993-2000 to 2001-2014. Now all we have in the “data” are years and values of X. It turn out that neither 2000 nor 2001 were the years of the policy change. Because the change is “significant”, the statistician must declare that some thing (or things) changed, but he doesn’t know what. But we have already agreed that some thing (or things) changed between the periods 1993-2001 and 2002-2014, even though these changes weren’t “significant”. What’s the difference between “significant” and not “significant”? Well, there isn’t and cannot be one: the concept is entirely ad hoc, as we see next.

(2) Since the statistician (nor you or I) does not know what causal changes took place between 1993-2000 to 2001-2014 and between 1993-2001 to 2002-2014, but he must say change did occur in the first place but not the second, because the first shift was “significant” but the second was not, he will say the parameter which represents change in the first shift really did change, but that the parameter which represents change in the second shift did not change and is in reality equal to 0. The parameter is the mathematical object in the regression that says how the central parameter of the normal distribution which represents uncertainty X changes when shifts occur. (Regression is about changes in parameters, not observables, unless they are turned into predictive models.)

Consider this carefully. The statistician will admit that a shift took place between 1993-2001 to 2002-2014, and he knows the shift must have been caused by some thing (or things), and he must admit this thing cannot be chance, because chance isn’t ontic, so he will say that the parameter did not change and must be set equal to 0. Now parameters are just as real as chance, so that physical events cannot operate on them. This means the parameters, and the models themselves, are purely a matter of epistemology; they only represent our uncertainty given certain assumptions. Nothing in nature could change the parameter, which calculations show is not equal to 0, so it is a pure act of will on the part of the statistician to set it equal to 0.

And this is fine: the statistician is free to do whatever he wants. The model can be made predictive assuming the parameter is 0 or non-zero. But in no way can the statistician assert that no changes in cause occurred, because of course they did. Causal changes also occurred between 1993-2000 to 2001-2014, and the statistician is free to accept the non-zero value of the parameter. In both cases, the statistician has no idea what happened to the causes, except that they changed.

Result? Probability models cannot ascertain cause, nor changes in cause. If causes were known, then probability models aren’t needed. Probability models cannot ascribe proportional cause, either. If we added terms to the regression, all the model would tell us is if the term changed in such-and-such a way, the uncertainty in X goes up or down or whatever.

To understand cause, we must look outside the data and try and understand the essence and powers of those factors related to X. Finding cause is brutal hard work.

(3) I’m not going to tell you when the policy took effect. But consider that the change-point test would typically only be done for this year. I went the extra step of pretending that the policy changed in 1994, then pretended it changed in 1995, and so on. This extra analysis shows you how over-confidence occurs. After all, look at all those “significant” shifts! As mentioned, the policy was a one-time thing. That means lots is happening here: and it would mean lots is happening even if there weren’t any red dots. If any change occurs, cause changes.

And even if no dot changed, cause still changed. How do we know? Because we know the policy must have done something, given our outside knowledge. But it could be that after it was implemented, counter-balancing causes took place and affected X such that X did not move. Call this the Doctrine of Unexpected Consequences (to coin a phrase).

No. To discover the effects of the policy, we must do the hard work of going every place the policy touched and doing the labor of measuring as many of the causes as possible.

(4) A note on predictive models. Every probability model should be examined in its predictive, and not parametric, form. We almost never, except in rare circumstances, care about the values of parameters. We care about the observables. What we want is to say, given a change in some variable, here is the change in the uncertainty of X. Everything is stated in terms of measurable observable. No statements about parameters are made. Testing is not spoken of. Terms are put in models because they can be measured or because decisions can be made about that, and that’s it.

Much, much, much more on this in Uncertainty: The Soul of Modeling, Probability & Statistics.

Notes: some might suspect correcting for “multiple testing” will fix this. It won’t. Make the p-values as wee as you want, the interpretation is the same. Some might say, “Why I’d never use that model! Everybody knows the Flasselblitz is the best model for this kind of data.” It makes no difference.

Every Result Of Unsupervised Learning Is Correct; Or, All Learning Is Supervised

The real point I wish to make is that there is no such thing as unsupervised learning; or, stated another way, Truth exists; or, stated another way, every solution to an unsupervised learning problem is conditionally correct.

To explain. In machine learning, artificial intelligence, and statistics, too, there are, it is said, two regimes: supervised and unsupervised learning. Let’s state these in terms of classification, since that’s the simplest and always discrete, a form of problem which regular readers will know I love.

In so-called supervised learning, a set of data is observed which have specified classes (more than one, of course). These classes are known, A, B, C, and so on. Male and female are good example. Days of week another. And so on endlessly.

Not only are the classes known, but other measures thought probative of the classes are also measured—these additional measures are a requirement. If all—as in all—we had were labels, we are done. Given that warning, if we’re interested in M/F, then perhaps we have “calories consumed today” and “weight bench pressed”, or whatever.

A model then relates the probative measures to the classes, and in the end we form the prediction

     Pr( class = i | old data, new probative measures, M),

where M are the assumptions that led to our model. Notice we have “integrated out” any parameters of M, since they are of no interest to man nor beast. This form is so familiar that nothing more need be said about it. Except that I dislike the term “learning” applied to it. It doesn’t matter if M is a standard statistical model, or machine learning algorithm, or neural net, or whatever. It’s just a model—unless are in the rare situation where we know the cause of the class given the new probative measure, or have otherwise deduced M from known or assumed true premises.

What about unsupervised learning? All we have are some measures and no classes. For instance, somebody supplies us with a spreadsheet having just “calories consumed today” and “weight bench pressed”. Obviously—and most importantly—these are labels we assign, that we supply meaning to. This must not be forgotten. To the computer which reads these numbers, they are just numbers; and they’re not even numbers, just electrical impulses.

Now we might think to ourselves, “Say, I wonder if these two measures belong directly or indirectly, together or perhaps separately, to different classes? All I have are the measures and no indication of class. How many classes might there be? If there are two or three or whatever, what are their characteristics? It might be that there are k classes, and that it is true that these two measures vary by some set way inside each class. Indeed, if the measures do not vary in some set way between two or more classes, I will not be able to tell these classes apart.”

And so we come to an algorithm that identifies classes. There are many such algorithms. K-means clustering is one of the most popular, ably and succinctly described here (I’ll assume everybody reads this, or already knows about these algorithms).

We start with the assumption this algorithm will find the clusters, or classes, that are there. And we end with that assumption, too!

Put it another way. We started by assuming the algorithm will do its job. We run it and it does its job. Therefore, there algorithm has done its job. Assuming no mechanical or electrical failures in the computations, the algorithm has done what it promised to do. How could it not?

The clusters/classes the algorithm finds are correct—conditional on assuming the algorithm. It’s exactly the same situation as supervised learning. The probability above spits out correct probabilities conditional on the model. Unsupervised learning spits out correct classifications conditional on the model/algorithm.

So why, then, do people look at the output of unsupervised learning algorithms, and (sometimes) say, “The number of clusters/classes is too large. The algorithm is over-fitting,” or “I think these classes are right in number, but they don’t look right”? Why do they express any dissatisfaction, since the algorithm always does what it was designed to do?

Because, as should now be obvious, these folks are using higher-order criteria to judge the algorithms. Meaning the “learning” was supervised after all, but that the supervision wasn’t done inside the computer. Which proves the boundaries of the computing machine/algorithm are artificial, if you like. Or, since part of the algorithm is sans mathematics, not all aspects of the algorithm are quantified or quantifiable, which is also not surprising to long-time readers.

It turns out the algorithms are always set up in such a way that there was prior knowledge of the targets; i.e., supervision. (And given many of these algorithms are used in computer vision applications, the pun is apt.) The picture which heads this post (taken from the clustering/k-means link above) proves the point. Supervision is always there. And it’s always there because we’re always looking toward what we either know or suspect is true.

Technical Notes: All these clustering algorithms are based on two notions: the number of clusters/classes and the concept of variation within and between classes. Something has to guess how many, using some prior criterion, and something has to say what it means to vary and how that variation is measured, also pre-specified. Changing these two notions gives a different algorithm.

There is no such thing as random, and so pieces of algorithms that are said to do this or that “randomly” are always deterministic after all, but with an eye closed to the determinism.

Of course, since the “learning” (the high falutin’ term for estimating parameters in a model) is always supervised, and the problems to which these models are put are important (automatic classification, say), finding better algorithms is just the right thing to do. Obviously, the better we are at knowing the cause or measuring the determinants of classes, the better these algorithms will be. So our ultimate goal, just as in statistical modeling, is always the same: understanding cause.

Summary Against Modern Thought: Animal Souls Are Mortal

This may be proved in three ways. The first...

This may be proved in three ways. The first…

See the first post in this series for an explanation and guide of our tour of Summa Contra Gentiles. All posts are under the category SAMT.

Previous post.

Do not quail. I am sure in the new heavens and earth, God will allow new quail, as well as new cats and dogs.

Chapter 82 That the brute souls of animals are not immortal (alternate translation) We’re still using the alternate translation.

1 This truth [the title] can be clearly inferred from what has been already said.

2 For we demonstrated above that no operation of the sensitive part of the soul can be performed without the body. In the souls of brute animals, however, there is no operation superior to those of the sensitive part, since they neither understand nor reason. This is evident from the fact that all animals of the same species operate in the same way, as though moved by nature and not as operating by art; every swallow builds its nest and every spider spins its web, in the same manner. The souls of brutes, then, are incapable of any operation that does not involve the body. Now, since every substance is possessed of some operation, the soul of a brute animal will be unable to exist apart from its body; so that it perishes along with the body.

Notes And this includes “talking” monkeys and “counting” horses, animals which excel at mimicry but who cannot apprehend universals.

3 Likewise, every form separate from matter is understood in act, for the agent intellect renders species intelligible in act by way of abstraction, as we see from what was said above. But if the soul of the brute animal continues to exist after its body has passed away, then that soul will be a form separate from matter, and therefore a form understood in act. And yet, as Aristotle says in De anima III [4], with things separate from matter, that which understands is identical with that which is understood. It follows that the soul of a brute animal, if it survives the body, will be intellectual; and this is impossible.

4 Then, too, in every thing capable of attaining a certain perfection, we find a natural desire for that perfection, since good is what all things desire, yet in such fashion that each thing desires the good proper to itself. In brutes, however, we find no desire for perpetual existence, but only a desire for the perpetuation of their several species, since we do observe in them the desire to reproduce and thereby perpetuate the species—a desire common also to plants and to inanimate things, though not as regards desire proper to an animal as such, because animal appetite is consequent upon apprehension. For, since the apprehending power of the sensitive soul is limited to the here and now, that soul cannot possibly be cognizant of perpetual existence. Nor, then, does it desire such existence with animal appetite. Therefore, the soul of a brute animal is incapable of perpetual existence…

Notes We skip some simple arguments to bring the best counterargument in the next three paragraphs, and its solution in the fourth.

9 Nevertheless, it would seem possible to show that the souls of such animals are immortal. For, if a thing possesses an operation through itself, distinctly its own, then it is subsisting through itself. But the sensitive soul in brutes enjoys an operation through itself, wherein the body has no part, namely, motion; for a mover is compounded of two parts, the one being mover and the other moved. Since the body is a thing moved, it remains that the soul is exclusively a mover, and, consequently, is subsisting through itself. Hence, the soul cannot be corrupted by accident, when the body is corrupted, for only those things are corrupted by accident which do not have being through themselves. Nor can the soul be corrupted through itself, since it neither has a contrary nor is composed of contraries. The result of the argument, therefore, is that the soul is altogether incorruptible.

10 And, seemingly, Plato’s argument that every soul is immortal comes to the same thing, namely, that the soul is a self-mover; and everything of this sort must be immortal. For the body dies only when its mover departs from it, and a thing cannot abandon itself. That is why Plato inferred that a thing which moves itself cannot die. And thus he came to the conclusion that every soul possessed of the power of motion, even that of brute animals, is immortal. Now, we have remarked that this argument is reductively the same as the preceding one, since, given Plato’s position that nothing moves without being moved, a thing that moves itself is a mover through itself and therefore has an operation through itself.

11 Now, Plato also maintained that the sensitive soul enjoys an operation of its own, not only in respect to movement, but also as regards sensation. For he said that sensation is a movement of the sensing soul itself, and that the soul, thus moved, moved the body to sensation; wherefore Plato said, in defining sense, that it is the motion of the soul through the body.

12 Now, these Platonic dicta are patently false. For the act of sensation is not an act of movement; rather, to sense is to be moved; since, through the sensible objects altering the condition of the senses in acting upon them, the animal is made actually sentient from being only potentially so. However, it cannot be maintained that the passivity of the sense in respect of the sensible is the same as that of the intellect in relation to the intelligible, so that sensation could then be an operation of the soul without a bodily instrument, just as understanding is. This is impossible, because the intellect grasps things in abstraction from matter and material conditions, which are individuating principles, whereas the sense does not, being manifestly limited to the perception of particulars, while the intellect attains to universals. Clearly, then, the senses are passive to things as existing in matter, but not the intellect, which is passive to things according as they are abstracted. Thus, in the intellect there is passivity in utter independence of corporeal matter, but not in the senses…

Notes Aha! Animals are sentient, our good saint says; the opposite thinking is a modern confusion. Senses are not intellection, a phrase so important you ought to repeat it to yourself.

14 There is also the fact that sense is overwhelmed by an exceedingly high degree of intensity on the part of its objects; but the intellect is not, because he who understands the higher intelligibles is more and not less able to understand other things. Hence, the state of passivity brought about in the sense by the sensible differs in kind from that which the intelligible causes in the intellect; the latter occurs without a bodily organ, the former with a bodily organ, the harmonious structure of whose parts is shattered by the pre-eminent power of some sensible objects…

Notes Students take note: more knowledge is better than less.

20 It is, then, clearly impossible for any operation of the brute animal’s soul to be independent of its body. And from this it can be inferred with necessity that the soul of the brute perishes with the body.

Notes So long, Spot.

Resolved: Women’s Suffrage Caused More Harm Than Good

If your first reaction upon reading the title was “Whaaaaaaaaaaaaaat!” or “Frrrpppkwiliq boahh boahh boahh!” or some variant of “%&*&^%$##!”, then you have discovered the meaning of sacrilegious. Congratulations! You hold an idea that you consider beyond possibility of all discussion—just as ideologues do. But you also hold at least one thing sacred—and this is in your favor. If it’s the right thing.

The fun of debates is what can be learnt, hence the form of the resolution. Taking the Pro side is the irrepressible Anne Barnhardt (whom we have been meeting regularly of late):

…Do you know when things really started to go — literally — to hell in this country? When women were given the right to vote seperate and apart from their husbands…

…Up until women’s suffrage, a man was the head of his marriage and his household, and his vote represented not just himself but his entire family, including his wife and his children. When men voted, they were conscious of the fact that they were voting not just for themselves and their own personal interests, but they were also charged with the responsibility of discerning and making the ultimate decision about what was in the best interests of their entire family…

As soon as the 19th amendment was passed, men were effectively castrated, and in many, many cases disenfranchised by their wives. No longer was the man the head of the household. No longer was he responsible for his wife. Now the wife was a “co-husband” at best, or a flat-out adversary at worst. The notion of a man making the final decision about what was best for his wife and family per his God-given vocation as husband and father was now over…

Women are made with a healthy, innate desire to be provided for and protected…women want someone or someTHING to take care of them. For this reason, women tend to lean socialist, and are generally in favor of the expansion of government when the government promises to “provide” for them.

Well, you get the idea. The Government has stepped in and became Man for Woman. What was the name of that woman in the Obama campaign that proves this? Julia? A woman now can “have as many fatherless children as she pleases” and Government will step in and care for them and the woman. There is no requirement family does so. Thus, in a very real way, “Fathering children no longer binds a man to a woman in any way.”

Barnhardt then says “Men didn’t vote to societally castrate themselves, and never would have.” But this is false: men did allow women’s suffrage by vote (in 1919)—and they did not have to. And you would have thought, after December 1917 when both houses of Congress passed a Constitutional amendment to ban alcohol, the result of a movement largely led by women, that they would have seen what was coming.

Banned alcohol? Pause to recall that the United States of America banned alcohol. Via Constitutional Amendment. Banned alcohol. By rewriting its founding document. This is insanity itself. Not the first lapse into darkness, nor the last, but an extremely telling one.

What is true is after female suffrage:

Many, many married couples quickly found themselves voting against one another. The man would tend to vote for the more conservative platform, and the woman would vote for the more socialist platform. When this happened, the effective result was the nullification of BOTH individuals’ votes. What this did was massively reduce the voting influence of the married household, and magnify the voting influence of the unmarried — and the unmarried tend to be younger, and thus more stupid, and thus vote for big government.

Our debate is merely theoretical, as all but the most ardent ideologue will understand. It is only the activist who would read these words and invoke the Slippery Slope and scream that our Democracy is imperiled.

No. As detailed many times on this blog, suffrage has only expanded, and will continue to expand, both in eligibility and in matters. We recede from the Republic and inch ever closer to a true Democracy. And no small debate on an obscure site at the far edge of the Internet will change this. So do please relax and try not to take any of this personally.

« Older posts Newer posts »

© 2017 William M. Briggs

Theme by Anders NorenUp ↑