William M. Briggs

Statistician to the Stars!

Page 149 of 701

This Partly Explains Why Education Is Such A Mess

An educator facilitating the actualization of inherent learnabilities.

A trained educator facilitating the actualization of inherent learnabilities.

I am away from the computer today.

Alternate title, A Few Moments With The Teacher’s College (Columbia) Record. Many educators are told what to think at this institution before they are released into the wild, so it is of some importance. Jim Fedako first raised our awareness–and what could be more important than that!—of this site. Here are some of the featured articles.

This magazine may be the best argument for homeschooling you will ever discover. Here are a few articles culled from recent entries.

Article 1 Teaching as Jesus Making: The Hidden Curriculum of Christ in Schooling.

Background/Context: The ideas of teaching as salvation and teacher-as-martyr are not new concepts. Prior research, however, has largely failed to explore the historical and cultural religious roots that continue to inform the ways in which teachers are constructed. That is, though prior work has engaged with thinking about religion and thinking about teachers as saviors, little work has been done to uncover the hidden curriculum of teaching that positions teachers as versions of Christ in the public school classroom…

Findings/Results: We illustrate that although we think of teaching as a secular activity and assume that religion has been expunged from public, including teacher, education, the sediments of religion remain present in how the teacher learns to imagine, construct, and enact his or her work as teacher as savior and martyr.

Shorter summary: “I’m not too fond of religion, but I am a god to my students.”

Article 2 The “Two-Way Street” of Having an Impact: A Democratic School’s Attempt to Build a Broad Counterhegemonic Alternative

Background/Context: Critical education studies tries to make sense of the relationship between education and differential power in an unequal society and to what degree schools impact the social order. A premise in this field is that a fundamental aim of critical education is exposing unequal social, cultural, and economic power relations and engaging in social action that transcends the setting of the classroom and school. Counterhegemonic schools are thus generally characterized by an aspiration to be meaningful beyond the school community and a commitment to social transformation…

Conclusions: Many obstacles stand between a counterhegemonic school and being socially meaningful, including sociohistorical and political factors. No less important, however, are the broader structural aspects to creating a space in which transformative schools can succeed. Although bottom-up attempts may push hegemonic forms to incorporate certain aspects of their vision, they cannot have meaningful and widespread impact if unaccompanied by broad support and action at the policy level and if they do not become organic parts of a larger transformative agenda.

Shorter summary: “Not all schools and not all kids are the same.”

Article 3 Dialogue Across Differences of Position, Perspective, and Identity: Reflective Practice in/on a Student-Faculty Pedagogical Partnership Program

Background: Inspired by various conceptualizations of both cultural diversity and cross-role partnership, this discussion challenges the assumption that holds sway in many people’s minds: Differences primarily divide us. The context for this argument is a program that pairs undergraduate students and faculty members in semester-long partnerships to explore and revise pedagogical practices…[Emphasis added]

Research Design: Through systematically documented reflective practice, I draw on audiorecorded conversations, mid- and end-of-semester feedback, and follow-up interviews with student and faculty participants in the program, as well as on my own reflective notes and less formal communication with participants, to identify the ways in which these faculty and students conceptualize differences as resources for learning…

Conclusions/Recommendations: Higher education needs to create more opportunities for students and faculty to engage in dialogue across various kinds of difference…

Shorter summary: “I like when students and teachers talk.”

Article 4 Postmodern Test Theory

…Conclusions/Recommendations: The model-based reasoning that characterizes test theory is useful not because it measures extant traits defined and evidenced in the same way for all students, but because it helps us organize our thinking, marshal and interpret evidence, and communicate our inferences and their grounding to others. A skeptical attitude about models in assessment makes our uses of them more flexible, more powerful, and, ultimately, more effective at meeting and fulfilling the aims of education than they would be if we believed that they accurately captured the totality of the phenomenon. Our understandings of students’ learning and programs’ effects are enriched by multiple perspectives and diverse sources of evidence, some new or previously neglected but others with familiar (albeit reconceived) forms.

Shorter summary: “Tests tell us which kids are doing well and which poor, but I don’t like this.”

How Good Is That Model? Scoring Rules For Forecasts: Part II: Update


Read Part I

Part II

What we’re after is a score that calculates how close a prediction is to its eventual observation when the prediction is a probability. Now there are no such things as unconditional probabilities (see the Classic Posts page at the upper right for dozens of articles on this subject). Every probability is thus conditional on some evidence, i.e. premises. This evidence is called the model. Assuming no mistakes in computation or measurement (which, unfortunately, are not as rare as you’d like), the probability put out by the model is correct.

The probability is correct. But what about the model? If the model itself were correct, then every probability it announced would be 0 or 1, depending on whether the propositions put to it were deduced as true or false. Which is to say, the model would be causal and always accurate. We thus assume the models will not be correct, but its probabilities are.

Enter scoring rules. These are some function of the prediction and outcome, usually a single number where lower is better (even when the forecast is a vector/matrix). Now there is a bit of a complexity involving proper scoring rules (a lot of fun math), but which don’t make much real-life difference. A proper score is defined as one which (for our purposes) is “honest”. Suppose our scoring rule awarded all forecasts for probabilities greater than 0.9 a 0 regardless of the outcome, but used (probability – I(event occurred))^2 for probabilities 0.9 or less (the I() is an indicator function). Obviously, any forecaster seeking a reward would forecast all probabilities greater than 0.9 regardless what his model said. (The analogy to politics is obvious here.) What we’re after are scores which are “fair” and which accurately rate forecast performance. These are called proper.

A common score for dichotomous outcomes is as above: (probability – I(event occurs))^2, or its mean in a collection of forecast-observation pairs. This is the Brier score, which is proper. The score is symmetric, meaning the penalty paid for having large probabilities and no outcome equals the penalty paid for having small probabilities and an outcome. Yet for many decisions, there is an asymmetry. You’d most likely feel less bad about a false positive on a medical test for pancreatic cancer than for a false negative—though it must be remembered that false positives are not free. Even small costs for false positives add up when screening millions. But that is a subject for another time.

The Brier score is thus a kind of compromise when the actual decisions made based on forecasts for dichotomous events aren’t known. If the costs are known, and they are not symmetric, the Brier score holds no meaning. Neither does any score which doesn’t match the decisions made with the forecast! (I keep harping on this hoping it won’t be forgotten.)

There are only main reasons to score a model. (1) To reward a winner: more than one model is in competition and something has to judge between them. (2) As a way to understand particularities of model performance: suppose our model did well—it had high probabilities—when events happened, but it did poorly—it still had high-ish probabilities—when events didn’t happen; this is good to know. (3) As a way to understand future performance.

The first reason brings in the idea of skill scores. You have a collection of temperature model forecasts and its proper score; say, it’s a 7. What does that 7 mean? What indeed? If your score was relevant to the decisions you made with the forecast, then you wouldn’t ask. It’s only because we’re releasing a general-purpose model into the wild which may be used for any number of different reasons that we must settle on a compromise score. That score might be the complete rank probability score (CRPS; see Gneiting and Raftery, Strictly Proper Scoring Rules, Prediction, and Estimation, 2007, JASA, 102, pp 359–378), which has many nice properties. This takes a probability forecast for a thing (which itself gives the probability of every possible thing that can happen) and the outcome and measures the “distance” between them (not all scores are distances in the mathematical sense). It gives a number. But how to rate it?

There has to be a comparator, which is usually a model which is known to be inferior in some aspect. Take regression (see the Classic Posts for what regression really is; don’t assume you know, because you probably don’t). A complex model might have half a dozen or more “x” variables, the things we’re using to predict the “y” variable or outcome with. A naive model is one with no x variables. This is the model which always says the “y” outcome will be this-and-such fixed probability distribution. Proper scores are calculated for the complex and naive model and their difference (or some function of their difference) is taken.

This is the skill score. The complex model has skill with respect to the simpler model if it has a better proper score. It’s usually worked so that skill scores greater than 0 indicate superiority. If a complex model does not have skill with respect to the simpler model, the complex model should not be used. (Unless, of course, your real-life decision score is completely different than the proper score underlying skill.)

We’ll wrap it up in Part III.

Update Here is an example of an improper score in real life. Tanking in sports.

Pascal’s Pensées, A Tour: V

PascalSince our walk through Summa Contra Gentiles is going so well, why not let’s do the same with Pascal’s sketchbook on what we can now call Thinking Thursdays. We’ll use the Dutton Edition, freely available at Project Gutenberg. (I’m removing that edition’s footnotes.)

Previous post.

Now that the hacking is all sorted out, we’re back to our regularly scheduled program. Thinking Thursdays!

14 When a natural discourse paints a passion or an effect, one feels within oneself the truth of what one reads, which was there before, although one did not know it. Hence one is inclined to love him who makes us feel it, for he has not shown us his own riches, but ours. And thus this benefit renders him pleasing to us, besides that such community of intellect as we have with him necessarily inclines the heart to love.

Notes This is so, but then so is this: “For the time will come when they will not endure sound doctrine; but after their own lusts shall they heap to themselves teachers, having itching ears; And they shall turn away their ears from the truth, and shall be turned unto fables” (King James). Also see this translation: “For the time will come when people will not tolerate sound doctrine but, following their own desires and insatiable curiosity, will accumulate teachers and will stop listening to the truth and will be diverted to myths.”

15 Eloquence, which persuades by sweetness, not by authority; as a tyrant, not as a king.

16 Eloquence is an art of saying things in such a way—(1) that those to whom we speak may listen to them without pain and with pleasure; (2) that they feel themselves interested, so that self-love leads them more willingly to reflection upon it.

It consists, then, in a correspondence which we seek to establish between the head and the heart of those to whom we speak on the one hand, and, on the other, between the thoughts and the expressions which we employ. This assumes that we have studied well the heart of man so as to know all its powers, and then to find the just proportions of the discourse which we wish to adapt to them. We must put ourselves in the place of those who are to hear us, and make trial on our own heart of the turn which we give to our discourse in order to see whether one is made for the other, and whether we can assure ourselves that the hearer will be, as it were, forced to surrender. We ought to restrict ourselves, so far as possible, to the simple and natural, and not to magnify that which is little, or belittle that which is great. It is not enough that a thing be beautiful; it must be suitable to the subject, and there must be in it nothing of excess or defect.

Notes Point (2) does not say something good about listeners. While it’s true the speaker has a duty to ease pain, the listener is not excused labor. If he is, we’re back to tickling ears. Anyway, it’s clear that when Pascal said, “We ought to restrict ourselves, so far as possible, to the simple and natural, and not to magnify that which is little, or belittle that which is great” he proved that he would not have been a hit on the Internet. I’m also reminded of the late philosopher David Stove’s lament “You or I might perhaps be excused if we sometimes toyed with solipsism, especially when we reflect on the utter failure of our writings to produce the smallest effect in the alleged external world.” From “Epistemology and the Ishmael Effect.”

A reminder that we’re skipping some points, like 17, which states rivers are moving roads.

18 When we do not know the truth of a thing, it is of advantage that there should exist a common error which determines the mind of man, as, for example, the moon, to which is attributed the change of seasons, the progress of diseases, etc. For the chief malady of man is restless curiosity about things which he cannot understand; and it is not so bad for him to be in error as to be curious to no purpose.

The manner in which Epictetus, Montaigne, and Salomon de Tultie wrote, is the most usual, the most suggestive, the most remembered, and the oftenest quoted; because it is entirely composed of thoughts born from the common talk of life. As when we speak of the common error which exists among men that the moon is the cause of everything, we never fail to say that Salomon de Tultie says that when we do not know the truth of a thing, it is of advantage that there should exist a common error, etc.; which is the thought above.

Notes Montaigne would have made a great blogger. Epictetus, who did not publish, would have been hired by either a faithless university or some White House administration and then, at some point, abandoned when he went on quip too far. Incidentally, Salomon de Tultie is Pascal’s nom de plume, and Salomon is the French of Solomon. But what about that curious “it is of advantage that there should exist a common error”? The analogy I see is that every ship has only one captain. It is better sailing when all are in one accord (whether openly or not) then to have many hands pointing in different directions. This doesn’t preserve all ships from foundering, but it does most. And morale is better.

What Might Pope Francis’s Upcoming Encyclical Look Like?

For the love of money is the root of all evil: which while some coveted after, they have erred from the faith, and pierced themselves through with many sorrows.

So said St Paul in his first letter to Timothy, and human history is loaded with evidence confirming this view. Latterly, I say, money has been replaced in part by Theory. Pope Francis thinks Inequality. Which, he said, is the “fruit of the law of competitiveness that means strongest survive over the weak” which is the “logic of exploitation” and “waste”.

Or so he said in Italian to a group in Milan, his words translated by Vatican Insider. There is thus the very real danger here and elsewhere of missing nuances and even of incorrect wordings. So let’s tread carefully.

It is necessary, if we really want to solve problems and not get lost in sophistry, to get to the root of all evil which is inequity. To do this there are some priority decisions to be made: renouncing the absolute autonomy of markets and financial speculation and acting first on the structural causes of inequity.

Obviously, or at least I hope obviously, you cannot push the “strongest survive over the weak” metaphor too far. Neither “inequality.” If there were absolute equality, where the weak and strong are as one, there would be no Pope and no right or wrong ideas. Neither could there be politicians in charge to renounce absolute autonomy of markets or of anything else.

Incidentally, we musn’t form a USA-centric view of the Pope’s words. Here, for instance, the markets are very much tied to government, the executives of one are the executives of the other. Market leaders assist (if I may be allowed the euphemism) the government in fashioning laws and regulations to their mutual benefit.

The Pope is interested in the kind of inequality that causes some of the world to go hungry. “[T]he number one concern must be for the actual person, how many people lack food on a daily basis and have stopped thinking about life, about family and social relationships, just fighting to survive?” And here comes the kicker:

“Despite the proliferation of different organizations and the international community on nutrition, the ‘paradox’ of John Paul II still stands.” There is food for everyone, but not everyone can eat” while “at the same time the excessive consumption and waste of food and the use of it for other means is there before our eyes.”

Despite? Is that the right word? But he’s right about waste. The amount of food we toss out would have scandalized our ancestors. My maternal grandfather was fond of saying, and of enforcing, “Take what you want, but eat what you take.”

In a different venue (also translated), Pope Francis said that humans should think of themselves as lords but not masters of creation. This strikes me as accurate. In charge but restrained by natural law. The danger to those who slaver or fume over the Pope’s environmental words lies in thinking our environmental policy must consist in jumping from wanton disregard to unthinking worship. We dearly love a false dichotomy.

A Christian who does not protect Creation, who does not let it grow, is a Christian who does not care about the work of God, that work that was born from the love of God for us. And this is the first response to the first creation: protect creation, make it grow.

And from the Milan speech (with choppy translation grammar):

The earth is entrusted to us so it may be a mother to us, capable of sustaining each one of us. Once, I heard a beautiful thing: the earth is not a legacy that we have received from our parents rather it is on loan to us from our children, so that we safeguard it, nurture it and carry it forward for them. The earth is generous will never leave those who custody it lacking. The earth, which is the mother for all, demands our respect and non-violence or worse the arrogance the masters. We have to pass it on to our children improved, guarded, because it was a loan that they have given to us.

You have to read your own (right or left) political desires into this to have any policy of consequence flow from it. No definite directives can be implied from the Pope’s words. One cannot, for instance, argue that thus a carbon tax must follow. Neither can you say (which nobody does say) you can do whatever you want.

But many think or hope they can “leverage” the Pope to further their politics. Even now “eco-ambassadors” are flowing in great numbers to Rome to have a photo-op (secular blessing) because they are sure the Pope’s upcoming encyclical can be used by them as a bludgeon. They want in on what they are sure will be a good thing. We’ll see.

« Older posts Newer posts »

© 2016 William M. Briggs

Theme by Anders NorenUp ↑