William M. Briggs

Statistician to the Stars!

The Expansion Team Syndrome: SAT Scores & Science

13991795255_103162f67e_k

What happens when a professional sports league expands? We talked about this before, but the obvious answer is that the average talent of the league decreases—for two reasons. The first is statistical, the other cultural.

Here’s an example of the statistical, where the “team” are those folks who take the SAT. Used to be only a small percentage of kids took it, those kids who were moving on to college when college was not expected of all. The percentage of kids taking the SAT has crept up over the years. And we now have this headline SAT Scores Hit Four-Decade Low:

Graduating high school students’ SAT college admission scores fell again this year—to the lowest level in four decades. Rapidly growing expenditure on education seems to be producing poor test results.

A record 1.7 million graduating seniors took the SAT test last year. With a highest possible score this year of 800 on each SAT section, according to the College Board, students scored a worst since 1999 math score of 511, worst since 1972 reading score of 495, and worst writing score since the section was added in 2005.

Government education spending as a percent of national gross domestic product on kindergarten through 12th grade jumped from about 2 percent in the 1950s to 3.9 percent in the 1970s. Spending hit 4 percent in the 2000s and popped up to 4.4 percent of GDP early in President Obama’s administration.

Imagine if only the smartest 10%~20% of kids took the test. The mean (numerical average) SAT score would be high. Now imagine if every kid in the nation took it, including those offspring of parents who broke the law to come here and who are here illegally, kids who might not have mastered English, the language of the test. The mean score would be low.

As everybody knows, the second scenario is closer to the truth than the first, and this is acknowledged in the news story. “A record 1.7 million graduating seniors took the SAT test last year.” It’s thus no surprise mean scores are lower. Not every kid is above average.

So what about the cultural reason scores are lower? That is also contained in the news story, though it’s masked with words about money. Here’s the answer: The more time the teachers have to take the less time they can teach more.

Make sense? If a high school math teacher has a class of (let us call them) gifted students, that teacher has more time to teach more material. So instead of stopping at basic calculus, the teacher would have time to move to (say) partial differential equations.

But let that same teacher be put in front of a class of average students, meaning a mixed group of kids, with most of middling intelligence and with only a few gifted and only a few dullards. PDEs will never be reached. The dullards would never understand PDEs anyway, and the gifted kids who could won’t have the chance to learn them (unless they teach them to themselves). Questions on the SAT related to higher math won’t be answered correctly as often by this group as classes with only gifted students, and the mean score would drop.

Obviously, this example works for any subject matter, including history, biology, literature, and so on, but it also works for the inclusion of subjects. For instance, how many high schools now teach, say, philosophy? Or classical languages? Thus, SAT scores must drop the more education becomes universal.

And the same thing is happening in science (and every other area of our culture, naturally). Reader Ken points us to the article “A Scientific Look at Bad Science: What recent research says about fraud, errors, and other dismaying academic problems“.

Retractions are up, questionable research practices are up, competition for research funding is up. There are more papers published than ever before, more conferences, etc. And there is a noticeable dissatisfaction among scientists that we’re not as good as we used to be. This is expressed more in the oral tradition than in print because scientists like to keep their dirty laundry concealed like any other group.

Science is suffering from the Expansion Team Syndrome. There’s more money in the system than there was, say, fifty years ago, and because of the expansion in education, there are more colleges pumping out more scientists. The statistical mean of whatever gauge of quality you like must therefore go down.

The cultural burden of the expansion isn’t as straightforward as it was in high school, but it’s there. Instead of working, how many scientists have to sit on committees about “equality” (example) and the like? How many scientists waste efforts writing grants or pushing out another marginal (or worse) paper just to keep their total up?

I know many older working scientists read this blog. How many tasks do you perform that are impeding you that didn’t used to have to do in “the old days”?

17 Comments

  1. Wow, insightful post Briggs. I’ve had thoughts along these lines before but never to these now obvious conclusions. The lower the bar…

  2. Rewarding “participation” with a diploma doesn’t help either.
    Other countries recognize that education excellence results from a learning competition, with winners and losers.

  3. “The egalitarians noticed that people with a college degree made more money so they decided that everybody should have a college degree so everybody would make more money. It’s cargo cult thinking.

  4. Off topic: Please tell me where my objection is ill founded: Dr. Soon, et al suggested: “However, correlation does not necessarily imply causation. That is, there are at least four types of correlations:” p 4 of “Re-evaluating the role of solar variability on Northern Hemisphere temperature trends since the 19th century”. I would suggest that correlation points to causation. The causation must be discovered (or demonstrated, or proven). Correlation might demonstrate that a hypothesis is better than another. Correlation is neither necessary nor sufficient to prove causation. It is convenient.
    The authors seem to be impling that correlection proves something.

  5. Of course, now the solution is to dumb down the test. Remember that when your grandchild needs a surgeon. Progressives won’t care—equality is all important and if someone dies as a result, it happens. Hard to believe we went from the space race to the Homer Simpson/Modern Family race (seeing who can create the most insulting view of families and success on television). I suppose I should be happy I got to see us get to the moon. We’ll be lucky now to achieve the level of Haiti if we continue on this path.

  6. Briggs

    September 8, 2015 at 10:47 am

    Sheri,

    See the link at bottom of the post for Related Posts; the one about why dumbing down the SAT was expected.

  7. This resounds with one of Stephen Jay Gould’s articles about why evolution has to produce more complex life forms. We started with single cell ‘creatures’ and gradually split cells and combined them as specialist functions in clusters. After development leaves the single cell some of the future development can be in the more simple direction but that has limited scope. The other more complex direction isn’t hedged in.

    The spread of many natural phenomena is not a bell curve. Distribution is often hedged in on one side by an innate brick wall and can spread as far as it’s capable on the other side.

    In the SAT case, the brick wall is on the more intelligent or educated or whatever non-PC elitist term you accept, while the lower ability side of the distribution has much more scope for expression.

  8. The other side of the story:

    “In this revolution, research has become central; it also becomes more formalized, complex, and costly. A steadily increasing share is conducted for, by, or at the direction of, the Federal government.

    “Today, the solitary inventor, tinkering in his shop, has been overshadowed by task forces of scientists in laboratories and testing fields. In the same fashion, the free university, historically the fountainhead of free ideas and scientific discovery, has experienced a revolution in the conduct of research. Partly because of the huge costs involved, a government contract becomes virtually a substitute for intellectual curiosity. For every old blackboard there are now hundreds of new electronic computers.

    “The prospect of domination of the nation’s scholars by Federal employment, project allocations, and the power of money is ever present and is gravely to be regarded.

    “Yet, in holding scientific research and discovery in respect, as we should, we must also be alert to the equal and opposite danger that public policy could itself become the captive of a scientific-technological elite.”

    From Eisenhower’s Military-Industrial Complex speech. Put two and two together, and the risk is that we’ll become captive to an increasingly ignorant and government-focused scientific-technological elite. Nah – could NEVER happen…

    Q: is it in the Government’s interest to help populate America with educated, logical people with a firm grasp of history and math? Or would such people be nothing but trouble?

  9. Like the rules of baseball, the rules for the SAT have changed over time, so there’s an inherent problem with comparing scores from different eras. See the History section at https://en.wikipedia.org/wiki/SAT for the gory details.

    A brief summary:
    Understand that test scoring is scaled. From 1926 to 1941 raw scores were adjusted to a national mean of 500 and a standard deviation of 100. But that meant each administration was unique and not absolutely comparable to the others. From 1941 to 1994 scores were adjusted to the 1941/42 scales to remedy that problem. However, during the 60s and 70s scores declined in large part to a doubling of the number of students taking the test (expansion team syndrome) and demographic changes in the test takers (let’s avoid debate here and accept it had some influence). In 1995 raw scores were “re-centered” to a mean close to 500, making them not directly comparable to previous tests. In 2005, the writing component was added which lengthened the test to almost four hours and raised questions of fatigue affecting scores. All along the way the various components have been altered. For example, antonyms were dropped from the verbal section in 1994 while calculators were permitted in the math section for the first time. Complaints that the SAT relied too much on abstract reasoning and too little on classroom experience have lead to changes making it more like the ACT test which is more popular in fly-over country than in the coastal states.

    So what’s the point? SATs are a general indicator of the ability to do college level work. They are correlated with family wealth. They receive WAY too much emphasis by students, parents, and the rankings. Here’s a secret: high school grade point average is a better predictor of success in college than the SATs. Do well in high school and it’s highly likely you will earn a college degree, even if your high school is of mediocre quality. It just may not be an ivy league school name on your diploma, but then some of the most successful people graduated from average schools or dropped out of college altogether. The place it can make a substantial difference is as a threshold for merit aid. Smart schools are becoming more sophisticated in this area, however, and using more than SAT scores.

  10. I attended a gifted high school where AP courses were mandatory and SAT tests were expected. I got the lowest SAT score in my circle of friends, but it was good enough for college entry and didn’t retake it like some of them did. I still can’t believe one guy in our class got 1590!

  11. Another consideration is that intelligent men and women tend to marry each other. That is to say, men and women now more often marry within their educational level. And as the relative success level goes up the number of kids usually goes down. Over that last hundred years or so we have inadvertently created a growing underclass that has more children than the highly intelligent couples. Does this add to the problem?

  12. This would be an interesting article IF SAT scores ever were or could be a meaningful predictor of (1) academic success in college, (2) success in life. (By the way, I did very well in standardized tests, so my criticism isn’t a case of sour grapes.) Gary hit the mark when he said high school grades were a much better indicator of success than SAT/ACT scores.
    Then of course there’s the evidence of widespread cheating by Chinese (but definitely not native USA Asian) students, which would tend to raise scores.

  13. @Bob Kurtland,
    Excuse the shorthand terminology, but SAT scores “explain” 15-20% of college first-year grade point average.

  14. Well . . . this is a ratio thing, of course. The US population has doubled over my lifetime.

    I’d estimate that over my lifetime class size has more than halved while inflation-adjusted teacher salaries have more than doubled, in spite of poorer results. Only in the public sector, with a complete absence of competition, can such an immense decrease in productivity be rewarded.

    When a society expends vastly more resources to accomplish less, the average standard of living must decline accordingly. That this hasn’t happened in the US is because the private, competitive sector has more than made up the difference. Full of themselves after this great success in Education, though, the liberals are now doing us the favor of forcing the same sorts of public sector inefficiencies on an even larger target – Health Care.

    Can the private sector save the country yet again? Maybe. But think of the incredible opportunity cost squandered in the process.

  15. The value of 13 yrs academic education for 100 pct of kids in high school then a further 3-4 yrs in an academic education in college for the majority is doubtful.
    In some Swiss Cantons 80 pct of kids leave high school at around age 16 and enter apprenticeships in 200 or so occupations. Apprenticeship in areas such as IT, accounting, hotel management, you name it.
    Switzerland has the lowest rate of youth unemployment in the OECD.
    Earlier contact of young people with the adult world is beneficial.

  16. Statistical question or comment on the conundrum.

    Because SAT/ACT scores have an upper bound (which can theoretically be achieved), wouldn’t the distribution of scores follow a Poisson distribution rather than a normal distribution?

    Looking at averages would be inappropriate and the mode should be analyzed rather than the mean.

  17. Diane Taylor (Mrs) (M.Phil)

    September 9, 2015 at 7:35 pm

    A major reason for poor education results is poor education starting at primary school level. Obsession with schoolchildren’s reproductive health has led to unhealthy focus on health. Matters that are the concern of parents have been usurped and parents have been persuaded to relinquish their own responsibilities.
    Parents must take back control and have more confidence in their abilities to assist in the education of their children – no excuses!

Leave a Reply

Your email address will not be published.

*

© 2016 William M. Briggs

Theme by Anders NorenUp ↑