z56\doc\web\2002\06\testcomp.txt
From: Patricia Hausman 
Sent: Sunday, June 23, 2002 9:33 AM
Subject: [h-bd] SAT score compression




This is an old post of mine from the listserv of the Virginia
Association Scholars that gives more detail on my hypothesis about the
UC study.


_______________________________________________________




My most important point is that I believe that we 
may simply be seeing a statistical artifact of the 
scoring systems.


How do I explain this simply?  I don't know, but
I will try. 


Let's say we have a test on which the top score is 100.
For purposes of predictive validity, we'll fare better
if the mean is 60 +/- 10 than if the mean is 80 +/- 10.  
With the latter, the top score would represent 2 SD
above the mean.  Everyone higher than that would
look the same.  The ability to predict the future for
the high performers would be compromised by score 
compression at the top.  


The purpose of the SAT I renorming years ago was to 
raise the mean scores.  Noticeable score compression 
at the top came with it.


If the SAT II has less compression, it may well prove
to be a better predictor--not because its content is
more suitable, but because of the vagaries of the 
scoring system.


I have found the data for California students who took
SAT II.  On all of the tests--regardless of subject--the top 
score is 800.   I entered the data into an excel spreadsheet, 
and came up with the following SD equivalents for an 
800 score.




Data set: SAT II Biology Test Takers 


SAT II Biology                    2.12    
SAT-V of this subject
   test takers                         1.87
SAT-M of biology
   test takers                         1.71




Data set:  American History Test Takers


SAT II Amer Hist                 2.23
SAT-V of this subject
   test takers                           2.17
SAT-M of this subject         
   test takers                           2.10




Data set:  Math I C Test Takers


SAT II Math I C                    2.55
SAT-V of this subject  
   test takers                           2.42
SAT-M of this subject
     test takers                          2.37 


Data set:  Writing Test Takers


SAT II writing                         2.30
SAT-V of this subject  
   test takers                              2.26
SAT-M of this subject
     test takers                            2.08 




Data set:  Literature Test Takers


SAT II Literature                    2.54
SAT-V of this subject  
   test takers                              2.30
SAT-M of this subject
     test takers                            2.41 


There are 19 subject tests.  Some deviate markedly from this pattern,
but they tend to be the language tests, where native speakers skew the
mean into the 700s.


The most important tests are those with large numbers of participants.
The subject tests with n> 10,000 are literature, Am history, math I C
and writing.  (I threw in biology above because it was first on my
list--n there is 7019).  And for these there is a trend for the SAT II
to have a lower mean and for 800 to represent a higher place on the
range than the SAT V or M of the same students who took the particular
subject test.


So there you have my working hypothesis.  Add in Lloyd'd very
important point about controlling for courses taken and perhaps there
would not be much mystery left. 


Very few schools require the SAT II.  Aside from UC, most that do are
highly competitive: e.g. Harvard.  I suspect there may be no
coincidence here.  When the SAT I started cutting off well below 3 SD
above the mean Harvard needed something that made finer distinctions.


Patti