STEELE CLAIM OF EQUAL TEST SCORES BOGUS \doc\web\2000\01\steele.txt Date forwarded: Fri, 21 Jan 2000 19:03:36 -0500 (EST) From: "Patricia Hausman" Al quoted the LA Times on Claude Steele's research "When researchers present a test in a different way--one that lowers the stakes on the test--or if they tell students that the test is one on which all races score similarly, scores by blacks rise to levels that are comparable with their white classmates, Steele's research shows." -------- Al, I don't think Steele's research has been duplicated by anyone, and if anyone ever attempts to do so, I hope they will use a more defensible experimental design. I have never seen a study that uses some of the techniques he did. He does not present the actual scores on the test. Rather he adjusts the scores for "initial ability," defined as SAT math score (even though in his affidavit for the U Mich case affirmative action case I believe he said that he doesn't know what SATs measure). I read his paper in Am Psychologist that looked at math perfor- mance in both female and black Stanford students. From the data presented, it appears that if two students (for example, one black, one white) in his study both answered 15 of 20 questions correctly, and the black student entered Stanford with a lower SAT score, the black student's score in the experiment would be higher than the white students even though both answered the same number of questions correctly. Steele calls this adjusting for initial ability. I don't believe it's a common practice in the world of testing. Also, the problems that he gave the subjects came from the GRE. So he's adjusted perfor- mance on GRE items by SAT performance. Seems to me that his data might be described as math ability adjusted for math ability-- a weird endpoint. Oh and the really good part. To adjust for initial ability he uses *self-reported SAT scores* even though the data direct from ETS would certainly be on file at Stanford. It's not exactly a secret that self-reported scores are less accurate than those that come directly from the horse's mouth. On the sexist threat section (my primary interest as you know) he claims that when women were told that the sexes score equally well the difference in performance between males and females was minimized. (Again adjusted for "initial ability"). What he doesn't comment on is that some of the convergence was due to the MALES doing less well, rather than the females answering a lot more questions accurately. As I recall the male mean was still above the female mean--but whether the difference was statistically significant I don't recall. Steele's basic position is that stereotype threat depresses the performance of various underperforming groups. Putting the sexism threat aside, what he doesn't explain is why the under- performance of certain minorities exists from a young age, long before people can arguably be presumed to understand a concept such as stereotyping. Data from the Children of the NLSY show that minority children have quite good academic self-concepts despite the disparity in actual performance on standardized tests. Reading his review of the literature on females and math performance made me wonder if he and I live on the same planet. So, I guess you would say that I am not at all impressed with his work. My own view is that it sells because of its inherent appeal, not its scientific rigor. Patti "In his presentation before the Board of Regents, Claude M. Steele, the Stanford psychologist who has done the experiments on students succumbing to "stereotype threat," said that students don't have to believe the negative stereotype about their race for their scores to decline. They just have to be put in an uncomfortable situation.      When researchers present a test in a different way--one that lowers the stakes on the test--or if they tell students that the test is one on which all races score similarly, scores by blacks rise to levels that are comparable with their white classmates, Steele's research shows." Al Boyd here - This excerpt is from L.A.Times 1/21/00 coverage of a Ward Connerly effort to get rid of racial classification. The URL is below. "Steele's research shows", rather than "claims" or "asserts" indicates L.A.Times' approval, of course, but has this research result been duplicated (or refuted) by anyone else? h ttp://www.latimes.com/news/state/20000121/t000006720.html