Again, suppose the test were to consist of the single question, “Derive from first principles twenty-six dimensional Bosonic string theory.” Pass rates would be south of 1%.
Neither version could ascertain the differences in ability between people.
What is our hidden premise for these conclusions? The pool of examinees. If this pool were to consist of professional, working physicists pass rates for either version would be high. But if it were to consist of one-year olds, pass rates for either would be low or zero.
The SAT’s pool are near-graduation high schoolers. This pool is not static; it changes and has changed characteristics dramatically since the SAT was first introduced in 1901 (the first multiple-choice version was issued in 1926). Perhaps the most profound shift over this period is the percentage of children who attend high school and who took the test. In 1926, this percentage was low. Only the most highly educated children completed high school and took the SAT. In 2013, the percentage was high. Children of all abilities enrolled in high school and sat the exam.
If the test were static, this increase in the pool, because it contains larger and growing proportions of less able children, would drive scores lower and lower, maybe so low that the majority of takers would now fail (especially considering the SAT used to penalize for wrong answers). The static test would resemble (in a crude way) our string theory version. Discrimination would suffer.
The widening pool, along with various other cultural changes too depressing to contemplate, also caused an easing of the material taught in high school thus leaving kids even less prepared for the more difficult versions of the SAT. Perhaps the most fundamental alteration is in reading. Kids (and teachers) read much less now than a century ago. But the SAT assumes reading ability. (Similar watering downs occur with increasing frequency in college; consider, for example, “remedial” reading and math university courses using materials historically taught in middle school.)
Some even claim that intelligence on the whole has been decreasing for reasons of biology. Maybe this is so, but it is very difficult to tease out environmental versus biological influences as they change through time.
Anyway, it’s impossible for anyone but a bureaucrat or academic to say that kids are growing smarter or are better educated. Given the road our culture (and politics) are taking, there is no reversal of these downward trends in sight.
This means it was right for ETS to dumb down the SAT.
If they did not, the quondam SAT would have larger and growing clusters of scores at the low end and fewer and more strung out scores at the high end. Discriminating between students would thus become more and more difficult. (What’s ideal is a test the result of which is a spread out distribution of scores over the enter range of possibilities with the mean score somewhere near the middle.) Considering that the goal of the SAT is discrimination, no other course of action makes sense.
Dumbing down the SAT is thus like a clothing manufacturer retooling his patterns to reflect a population which is growing shorter and squatter. Further, it makes no sense to decry the manufacturer’s sensible and prudent decision.
This isn’t the first time the SAT has been dumbed down, either.
In 2005 the SAT famously removed analogy questions. Example, P-VALUES : STATISTICAL THEORY :: MARXISM : POLITICAL THEORY. Analogies are difficult and require higher levels of thinking than other types of question. For the same reasons noted above, the proportion of kids doing poorly on the analogies was growing too large, so out they went.
Interestingly, besides eliminating penalties for wrong answers and simplifying its math, in this latest purge the SAT will remove “obscure” and “esoteric” vocabulary words (the Settle Times lists “prevaricator” and “sagacious”). Both analogies and “obscure” words, of course, relate to reading. Reading ability among high schoolers must be shriveling. Well, no surprise there.
The timing is also curious. It was only a short nine years since the last major restructuring. This indicates the pace of decline is accelerating. And this is despite the increasing efforts (and funding) of teachers to “teach the test” to the exclusion of more important matters.
Or maybe “despite” is the wrong word.