Skip to content

Author: Briggs

August 13, 2018 | 6 Comments

How To Do Predictive Statistics: Part V New (Free) Software Multinomial Regression

Previous post in the series (or click above on Class). REVIEW!

Download the code: mcmc.pred.R, mcmc.pred.examples.R. If you downloaded before, download again. This is version 0.22! Only the example code changed since last time.

For an explanation of the theory behind all this, which is markedly different than classical views, get this book: Uncertainty.

Mandatory rant

We’ll use another built-in dataset, the Netherlands voting data. Accessed like this:

x = Nethvote

As before, assigning it to x is for simplicity sake. Find out all about the data with ?Nethvote. Essentially, voters could pick one of several parties. And, supposing those parties last until the NEXT election, and given some demographic information, we want the probability that

     Pr(Y | new X, old X&Y, Model & Assumptions)

Which—surprise!—is identical to the same probability we want in every predictive model! The emphasis on the NEXT election cannot be stressed too highly. Why? Glad you asked. Let me shout the answer:

There is NEVER a need to model what happened, only what might happen.

We do not need a probability model to tell us what we saw. We need only use our eyes. If we want to know if more religious people voted for vote (i.e. party) = CDA WE JUST LOOK. There is no need to do a “hypothesis test”, which is insane. Either more religious would have voted for CDA, or they wouldn’t have. AND THAT IT IS.

The classical idea, frequentist or Bayes, p-value of Bayes factor, of ascertaining whether more religious “really” voted more frequently for CDA is nuts. About the future? Well, that’s what model is for. To quantify the probability more religious will vote CDA accepting as an assumption religion is probative. It is our decision whether we choose religion as probative or not; two people looking at the same data, and even same model, can come to different conclusions.

I shout, because these ideas are central to the (old, ancient) predictive approach. They are foreign to the hypothesis testing classical methods, which aim to uncover occult forces in data. We will have none of that. Our concern is only observables and measures.

On to the data!

The real beginning

Because of a limitation (out of my control) of MCMCmnl, we have to keep track of the model formula. So we call the method a little differently than when we did ordinary or logistic regression.

form = formula('vote ~relig + class + income + educ + age * urban')

lv = levels(x[, as.character(form[[2]]) ])

fit = MCMCmnl(form, mcmc.method="IndMH", B0=0, mcmc=5000, thin=10, tune=0.5, baseline='D66', data=x)

Notice form is a standard R formula. This one was chosen to match the one in the native help function. Fool around with others. The object lv holds the levels of the “y” observable. It’s written in a generic way, so that it works with any data set. We could have, of course, just wrote lv = levels(x$vote), but that works only on data frames with vote as an outcome. Notice, too, that we can change the baseline. We don’t have to: it will default to the normal R base level. We keep track of the levels because you’re allowed to change them, and MCMCmnl doesn’t save the model formula. Ah, well.

Predictions are somewhat different than before, too. We have pass in the model formula and levels of the y. We also need, as ever and as core of the predictive method, a scenario. How about this one? Mixing code and output, and ignoring the ‘dist’ measures, which we don’t use.


vote distD66 distPvdA distVVD distCDA relig class income educ age urban
PvdA 2.669695 2.335121 4.109881 6.45008 0 0 1 2 5 1


p = MCMCmnl.pred(fit,form,x[1,],lv)

I get

> p
     D66      CDA     PvdA      VVD 
0.076580 0.067476 0.822900 0.033044 

So, given non-region, class of 0, and so on, the probability a NEW voter will go D66 is about 8%. Your results will vary a bit, since as ever this is a numerical approximation. But they’ll be close. The most likely vote will be cast at 82% is for PvdA for NEW voters of this sort, and the least likely is VVD at 3%. I don’t know Dutch politics, so I offer no opinions on what this means.

The idea, if it isn’t clear, is that you get a probability for each possible category, because why? Because that’s what we wanted!

The form and lv ensure everything is labeled correctly at the end. Pain in the keister. But as yet there are no wrappers for any of these methods to make things easier.

How about all the scenarios in the data? You bet:

p = MCMCmnl.pred(fit,form,x[1,],lv)
for(i in 1:nrow(x)){
  # this preserves the proper names for p's columns
  if(i>1) p=rbind(p,MCMCmnl.pred(fit,form,x[i,],lv))
p =, row.names=FALSE)

for (i in 1:4){
    plot(x$class,p[,i],main=names(p)[i], ylab='Pr(Vote|D,M)',col=x$relig+1)

Notice we stacked the answers one on top of the other, and turned p into a data.frame. The plot is for each category or level of vote, as a function of class (which really does have all those odd values; probably the output of some other model). For fun, I colored the points by religion yes/no.

This is only one possible plot of many. Other obvious ones will suggest themselves to you. Do them as homework.

Everything is more complex because the model itself is more complex. There isn’t any real or general way to make this easy, either. Nor should there be!

“But, Briggs, can’t I do an average probability for each class level, using all the old scenarios? That way I can tell the impact of ”

Sure you can. But why would say impact when you meant influence? Second, it would be fooling yourself. Because your model included all those other things, you have to state probability only with regard to and conditional on all those other things. Otherwise you’re talking weird.

If you want to discuss only class, then build a model with only class.

form = formula('vote ~ class')
lv = levels(x[, as.character(form[[2]]) ])
fit = MCMCmnl(form, mcmc.method="IndMH", B0=0, mcmc=5000, thin=10, tune=0.5, baseline='D66',data=x)

Then you can say what you want about class considered only by itself. Or whatever.

The key lesson is that you specified a model with all those measures, so you can only speak of the model with all those measures. If you don’t want to speak of them, remain model-silent of them.

Mini rant

We are done with multinomial. But not really. It should be used in place of ordinary regression almost always. Why? Because all measures are discrete and finite, thus all Y are, thus all Y are better approximated by multinomials. Now, all Y are approximated by continuity, which is an ENORMOUS assumption, and untrue. No measure can be continuous, and none infinite in actuality.

All data should be transformed into the units of decision. We talked about this before with regard to CGPA data. If you are a dean only interested in counting numbers of students scoring 3 or hihger in CGPA (or whatever), then you have naturally created an analysis were the Y is dichotomous. Or maybe you want 3 or above, which naturally implies under 3s are of interest, and then 4s (to be given special recognition, say). Then we have a trichotom. Multinomial can handle this, ordinary regression cannot.

Two people can have the same data and come to different conclusions about it, as happens all the time in real life. People have different decisions to make, and different consequences to face about those decisions. Therefore, every analysis, i.e. model, should be tailored to the decision at hand. Since every decision, like every measure, is discrete and finite in act, then so should by every model.

“But Briggs, if I quash the data into buckets like you say, then I lose information. I won’t know the difference, in this case, between a CGPA of 2.876543 and 2.876544. I’m losing power or whatever. Besides, I’ve heard discretizing data is bad.”

You heard wrong. I remind you that there is no difference between 2.876543 and 2.876544—not one bit! nor between 0 and 2, or 0 and 2.9—when any decision you make recognizes no difference between these CGPAs! If you are going to make different decisions, then you will have different buckets, and thus a different model, and different results.

This is not a bug, it is a feature. Just like the conditionality of all probability.

Next is Poisson regression.

August 12, 2018 | 1 Comment

Summary Against Modern Thought: A Second Misunderstanding of Separate Substances

Previous post.

Like last week, this chapter exists to squash a subtle, technical errors. Again it requires close reading to grasp. It’s best to at least skim: don’t skip. Reminder: An intellectual substance is an angel.


1 Because there is very great difficulty in Alexander’s opinion, as a result of his supposition that the possible intellect in a condition of habituation is entirely corruptible, Averroes thought that be found an easier way to show that we sometimes understand separate substances. In fact, he asserted that the possible intellect is incorruptible and separate in being from us, as is also the agent intellect.

2 He showed, first of all, that it was necessary to hold that the agent intellect is related to principles naturally known to us, either as agent is to instrument, or as form to matter. For the habitual intellect, by which we understand, has not only this action of understanding, but also another, which is to make things actually understood; indeed, we know by experience that both actions stand within our power.

Now, the action of making things actual objects of understanding is more properly indicative of the meaning of habitual intellect than is the act of understanding, for to make things actually intelligible precedes the act of understanding them. But there are some things within us which are rendered actually understood in a natural way, not as a result of our effort or of the action of our will: such are the first intelligible things.

In fact, to make these actually understood does not depend on the habitual intellect, through which things that we know from study are made to be actually understood; rather, these first intelligibles are the starting point of the habitual intellect. And that is why the habit of these intelligibles is also called understanding by Aristotle, in Ethics VI [6: 1141a 7]. Now, they are made to be actually understood by the agent intellect alone. And by means of them other things are made to be actually understood: these are the things that we know from study. So, to make these subsequent things actually understood is the work both of the habitual intellect, as regards first principles, and of the agent intellect. Now, one action is not attributed to two things unless one of them is related to the other as agent to instrument or as form to matter. So, the agent intellect is necessarily related to the first principles of the habitual intellect either as agent to instrument or as form to matter.

Notes In this sentence is wisdom: “the action of making things actual objects of understanding is more properly indicative of the meaning of habitual intellect than is the act of understanding, for to make things actually intelligible precedes the act of understanding them.” You can learn a fact, but to assimilate it, really own it, is far more difficult. The “first intelligible things” are given by a certain form of induction.

3 In fact, he indicates how this is possible in the following way. Since the possible intellect, according to his theory, is a separate substance, it understands the agent intellect and the other separate substances, and also the first objects of speculative understanding. So, it is the subject for both types of objects. Now, whenever two things are united in one subject, one of them is like the form of the other. Thus, when color and light are present in a diaphanous body as their subject, one of them, namely, light, must be like the form of the other, namely, color. Now, this is necessary when they have an ordered relationship to each other, but not in the case of things accidentally associated in the same subject, like whiteness and musical ability. But speculatively understood things and the agent intellect do have an ordered relationship to each other, since the objects of speculative understanding are rendered actually understood by means of the agent intellect. So, the agent intellect is related to the objects of speculative understanding as form is to matter.

Therefore, when the objects of speculative understanding are united with us through the phantasms, which are in a sense their subject, the agent intellect must also be connected with us, because it is the form of the objects of speculative understanding. Thus, when the objects of speculative understanding are only potentially present in us, the agent intellect is only potentially connected with us. But, when some objects of speculative understanding are actually in us, and some are potentially present, its connection with us is partly actual and partly potential. Then it is that we are said to be in motion toward the aforementioned connection, for, as more things are made to be actually understood within us, the agent intellect becomes more perfectly connected with us. This progress and movement toward the connection is accomplished through study in the speculative sciences, through which we acquire true objects of understanding, and also false opinions that are outside the orderly process of this movement are excluded, just as monstrosities are outside the order of natural operation. Hence, men may help each other in making this progress, as they are of mutual assistance in the speculative sciences.

And so, when all potential objects of understanding have been made actual within us, the agent intellect is perfectly united with us as a form, and then we will understand perfectly through it, just as we now understand perfectly through the habitual intellect. Hence, since it is the function of the agent intellect to understand separate substances, we will then understand separate substances, as we now understand the objects of speculative understanding. And this will be the ultimate felicity of man, in which man will be “like some sort of God.”

Notes The key word is, of course, like. Knowledge is felicity!

4 Now, the refutation of this theory is sufficiently evident from the things that we have said earlier: in fact, it proceeds from the supposition of many points which are disproved in the foregoing sections.

5 First of all, we showed above that the possible intellect is not some substance separated from us in its being. Hence, it will not be necessary for it to be the subject of separate substances, especially since Aristotle says that the intellect is possible, “in that it is able to become all things.” From this we see that it is the subject only of those things that are made actually understood.

6 Again, we have shown above, concerning the agent intellect, that it is not a separate substance, but a part of the soul, to which Aristotle assigns this operation: “to make things actually understood” [De anima III, 5: 430a 14], and this lies within our power. Hence, it will not be necessary for the act of understanding—through the agent intellect to be the cause, for us, of our capacity to understand separate substances; otherwise, we would always understand them.

7 Furthermore, if the agent intellect is a separate substance, it cannot be joined to us except through species that have been made actually understood, according to this theory; and neither can the possible intellect, even though the possible intellect is related to these species as matter to form, while, conversely, the agent intellect is as form to matter.

Now, species that have been made actually understood are joined with us, according to his theory, by means of the phantasms which are related to the possible intellect as colors to the visual power, but to the agent intellect as colors to light: as we see from the words of Aristotle in Book in of On the Soul [III, 5: 430a 15]. But to the stone in which color is present, neither the action of the power of sight as it sees nor the action of the sun as it enlightens can be attributed. Therefore, according to the aforesaid theory, it would be impossible to attribute to man either the action of the possible intellect as it understands or the action of the agent intellect as it understands separate substances or as it makes things actually understood.

8 Besides, according to this theory, the agent intellect is not asserted to be connected with us as a form except by the fact that it is the form of objects of speculative understanding; and it is claimed to be the form of these objects because the same action belongs to the agent intellect and to these objects of understanding, which action is to make things actually understood. So, it could not be a form for us, unless by virtue of the fact that the objects of speculative understanding share in its action. Now, these objects do not share in its operation which consists in understanding separate substances, for they are the species of sensible things, unless we go back to the opinion of Avempace that the quiddities of separate substances can be known through the things that we understand about sensible objects. Therefore, it would not be at all possible for us to understand separate substances in the aforesaid way.

9 Moreover, the agent intellect is related to the objects of speculative understanding, which it makes to be so, in a different way from its relation to separate substances, which it does not make, but only knows, according to this theory. So, there is no necessity for it to be joined to us in its function as knower of separate substances, even if it is joined to us in its function as maker of the objects of speculative understanding. Rather, there is clearly a fallacy of accident in reasoning such as his.

10 Again, if we know separate substances through the agent intellect, this is not accomplished because the agent intellect is the form of this or that object of speculative understanding, but because it becomes a form for us, for in this way we are enabled to understand through it. Now, it becomes a form for us even through the first objects of speculative understanding, according to his own statement. Therefore, immediately at the start, man can know separate substances through the agent intellect.

11 Of course, it might be answered that the agent intellect does not become a form for us, in a perfect way, by virtue of certain objects of speculative understanding, so that we might understand separate substances through it and the only reason for this is that these objects of speculative understanding are not sufficient for the perfecting of the agent intellect in the act of understanding separate substances. But not even all the objects of speculative understanding taken together are sufficient for that perfection of the agent intellect by which it understands separate substances.

For all these objects are intelligible only in so far as they have been made to be understood, while those separate substances are intelligible by their own nature. So, not even the fact that we will know all the objects of speculative understanding will make it necessary for the agent intellect to become a form for us, in such a perfect way that we may understand separate substances through it. Or, if this is not required, then we will have to say that, in understanding any intelligible object, we understand separate substances.

August 11, 2018 | 11 Comments

Insanity & Doom Update XLVII

Item North Carolina Elementary Teachers Struggle To Pass Math Portion Of Licensing Exam

The Charlotte Observer reports that nearly half of newly hired teachers in North Carolina failed to pass the math portion of a licensing exam which is required for them to keep their jobs. Some teachers claim the problem isn’t with them but with the test itself which includes middle school and high school math problems. Last month the Board of Education granted new teachers on an additional year to pass the test…

A 19% pass rate is pretty awful for college educated people.

This, it is thought, is a sample test (pdf). Readers here could pass even without lead in their Dr Grip pencils. We haven’t heard yet (and I haven’t looked) to see criticisms about racism, etc.

Item Adult Swim’s ‘Rick and Morty’ Co-Creator Depicts Children Being Raped, Tortured, and Killed

Nope, not Dan Harman, the co-creator of “Rick and Morty,” who filmed himself raping a babydoll. The other co-creator, Justin Roiland, is now embroiled in his own child rape scandal. Roiland made a cartoon, posted on his website, about raping, torturing, and killing children. It’s so graphic we can’t show it here. If you want to watch it (which I don’t recommend), it’s at this link. The cartoon begins with two young boys who decide to go for a swim in the ocean naked.

It starts in the pits and only goes downhill from there. Which you wouldn’t think possible—since it started in the pits. But it did. Don’t click on the link. I can’t think of anybody who would be saddened to learn Hollywood fell into the sea with a milestone around its neck.

Item I feel pretty — and confident. A straight man’s adventure with makeup.

I wonder if everyone at this bar thinks I look like a geisha — a dude geisha. Are the other patrons sneaking glances at me, judging the guy alone in a booth? I’ve come prepared for a date, dressed in the standard-issue San Francisco uniform: dark jeans, desert boots, crisp button-down. I’ve also added the sheen of makeup on my face.

The girl I’m meeting is late…Is there now going to be a line on the side of my face? Also how does this foundation look in this dim lighting? To my date, will I resemble a 9-year-old girl who got a hold of her mother’s makeup kit, or a responsible 29-year-old man who pays rent and appreciates rye whiskey? I weigh the benefits of verifying that I look fabulous by whipping out my cellphone camera, but what if my date walks in and sees me doing so?

Washington Post, friends. The woman shall not wear that which pertaineth unto a man, neither shall a man put on a woman’s garment: for all that do so are abomination unto the LORD thy God. Unless it’s to feel pretty.

Item The Rise And Rise Of The UK’s Student Sugar Babies

A growing number of students are turning to sex work to fund their studies as they struggle to cope with increasing debt and living costs, a new student finance survey has revealed.

BuzzFeed News spoke to student sex workers who said that they had developed sugar daddy relationships or taken to selling intimate pictures and videos online because their maintenance loans fail to cover their day-to-day expenses.

Save the Student’s national student money survey, released last week, has found that 3% of UK students are financing their studies with adult work, rising to 4% when faced with a cash crisis….

For some students, sugar daddies are less of a necessity and more a means to enjoying a better lifestyle at university, allowing them to travel, go out partying more often, and live in better accommodation than many of their peers. However, others sell sex as a way to survive.

There was an older word used for women who sold themselves for cash to spend on frivolities. Can’t recall it now. Nobody can, evidently.

One woman, who would have in olden days been called this now-forgotten name, for it would have been accurate, as it still is, if we could only remember it

With help from a ‘sugar daddy’, I could go partying every weekend, travel to different states, shop regularly, and eat out in the city three to four times a week if I wanted. I sort of became accustomed to that standard of living and it quickly became apparent that I wouldn’t be able to afford the same thing in London without some ‘aid’.

That darn word. What was it?

August 10, 2018 | 19 Comments

Episcopal Meltdown

A new prayer is being formulated by the Episcopal Church. An early copy was leaked. That copy found its way into my hands. I put it here for your judgement.

Our entity which art, in our judgment, a presence in the universe, why did you assign the gender of male to Jesus at birth, when you must have known this would have caused angst and turmoil to early Twenty-First Century progressives?

Did you not know he—we are forced to use the masculine pronoun, for which we apologize to our listeners—would tell his (again, sorry) followers to pray “Father, hallowed be thy name. Thy kingdom come.”

Father! How could he (again, sorry) know such a thing! When it is clear to us, situated as we are in 2018—-2018!—that you cannot be any gender at all. We know this because we now know, since it is the Current Year, that we can be any gender we like. Even genders never heard of before.

Help us, glorious entity, which may be a projection of our own enlightened selves, to fix this abominable situation. Teach the world of the infinity of gender!

Strong words. Maybe they’ll tone it down a bit in the end. We’ll have to wait and see.

Work is ongoing. Reports are that

The Episcopal Church formed a committee Wednesday to “provide a pathway” toward revising the Book of Common Prayer to include gender-neutral language.

Church leaders called for immediate revisions to correct the “overwhelming use of masculine language” throughout the book, arguing that the language is now a hindrance to spiritual inclusion, according to the Episcopal Church website.

“As long as ‘men’ and ‘God’ are in the same category, our work toward equity will not just be incomplete. I honestly think it won’t matter in some ways,” Wil Gafney, a professor of the Hebrew Bible and strong advocate for the edit, told the Washington Post

The Episcopal News Service shared further concerns from church leaders that the current language has created a “barrier to evangelizing young people.”

Interesting strategy. If young people are turned off by Truth, bend it, or adapt it, or paint over it a bit. That’ll turn them on, and surely bring them, and their donations, surging into the pews. Right?

And what’s more important to the young than pretending to be genders other than those assigned at birth?

The Episcopal Church has hard labor ahead. Because the leaked prayer is right. Jesus was a masculine man. The manliest man. It would, as it was, be impossible to write about this manly man without using masculine language. And it was, as the prayer says, Jesus himself—which is the only pronoun that makes any sense, given Jesus’s obvious blatant and even in-your-face masculinity—that taught us to say “Our father“. Indeed, he often spoke of God as father. Which is a masculine position. It’s in the job description, for crying out loud.

Therefore the only way to avoid hurtful masculine non-inclusive, and therefore exclusive, language, is to change it. Sure, any changes will shift Jesus’s words and meanings. But this is, again as the prayer says, 2018. If Jesus were here now, he’d probably watch his (again, sorry) words more carefully.

Well, you just can’t argue with that kind of logic.

The changes to scripture, which haven’t been leaked as far as I know, will likely be in an effeminate direction. Now effeminate is not the opposite of masculine; feminine is. While there is nothing in the world wrong with feminine language, there is everything wrong with effeminate language. It is squishy, sweet, and lawyerly.

But it is necessary—if you want to claim, as the Episcopal Church is now claiming, that two men (or two women, or two peoples of any gender) can be married to one another. Reports say

The Episcopal Church removed restrictions on same-sex marriage, a move that allows all couples to wed where they worship, even if their bishop disapproves…

Same-sex couples are already allowed to marry across most Episcopal Churches in the United States, but a few U.S. dioceses had not permitted religious wedding ceremonies for this type union.

Friday’s decision overrides previous decisions by local dioceses to not allow the liturgies, which currently includes eight of the of the nation’s 101 Episcopal dioceses…

No one spoke against the resolution during a short debate by the House of Deputies, the news service affiliated with the Episcopal Church said.

Well, and why would they speak against it? Once you gift yourself the ability to change scripture so that it conforms to Current Year thinking, why, the boundless pit is the limit.