z56\doc\web\2002\06\testcomp.txt
From: Patricia Hausman
Sent: Sunday, June 23, 2002 9:33 AM
Subject: [h-bd] SAT score compression
This is an old post of mine from the listserv of the Virginia
Association Scholars that gives more detail on my hypothesis about the
UC study.
_______________________________________________________
My most important point is that I believe that we
may simply be seeing a statistical artifact of the
scoring systems.
How do I explain this simply? I don't know, but
I will try.
Let's say we have a test on which the top score is 100.
For purposes of predictive validity, we'll fare better
if the mean is 60 +/- 10 than if the mean is 80 +/- 10.
With the latter, the top score would represent 2 SD
above the mean. Everyone higher than that would
look the same. The ability to predict the future for
the high performers would be compromised by score
compression at the top.
The purpose of the SAT I renorming years ago was to
raise the mean scores. Noticeable score compression
at the top came with it.
If the SAT II has less compression, it may well prove
to be a better predictor--not because its content is
more suitable, but because of the vagaries of the
scoring system.
I have found the data for California students who took
SAT II. On all of the tests--regardless of subject--the top
score is 800. I entered the data into an excel spreadsheet,
and came up with the following SD equivalents for an
800 score.
Data set: SAT II Biology Test Takers
SAT II Biology 2.12
SAT-V of this subject
test takers 1.87
SAT-M of biology
test takers 1.71
Data set: American History Test Takers
SAT II Amer Hist 2.23
SAT-V of this subject
test takers 2.17
SAT-M of this subject
test takers 2.10
Data set: Math I C Test Takers
SAT II Math I C 2.55
SAT-V of this subject
test takers 2.42
SAT-M of this subject
test takers 2.37
Data set: Writing Test Takers
SAT II writing 2.30
SAT-V of this subject
test takers 2.26
SAT-M of this subject
test takers 2.08
Data set: Literature Test Takers
SAT II Literature 2.54
SAT-V of this subject
test takers 2.30
SAT-M of this subject
test takers 2.41
There are 19 subject tests. Some deviate markedly from this pattern,
but they tend to be the language tests, where native speakers skew the
mean into the 700s.
The most important tests are those with large numbers of participants.
The subject tests with n> 10,000 are literature, Am history, math I C
and writing. (I threw in biology above because it was first on my
list--n there is 7019). And for these there is a trend for the SAT II
to have a lower mean and for 800 to represent a higher place on the
range than the SAT V or M of the same students who took the particular
subject test.
So there you have my working hypothesis. Add in Lloyd'd very
important point about controlling for courses taken and perhaps there
would not be much mystery left.
Very few schools require the SAT II. Aside from UC, most that do are
highly competitive: e.g. Harvard. I suspect there may be no
coincidence here. When the SAT I started cutting off well below 3 SD
above the mean Harvard needed something that made finer distinctions.
Patti