\doc\web\99\02\braccrit.txt The problem is that Glaser and all the enthusiasts who took up the crusade didn't seem to realize that most educational disciplines are so vague domains that it's impossible to specify those benchmarks. Hence a lot of wasted time. Send reply to: "Gerald W. Bracey" From: "Gerald W. Bracey" To: Subject: Re: [education-consumers] Bracey -- NAEP Date sent: Mon, 8 Feb 1999 15:49:09 -0800 I thought I dealt with this. You have to keep the tests and the "proficiency levels" separate. I thought I noted that the same kids who can't read according to NAEP levels are third in the world among 26 nations in reading and third among 27 in science. Now it is possible to say that the school crisis is global, but that is not what has been done when our kids don't show well. The NAEP proficiency levels have been rejected by virtually everyone in the psychometric community. Studies from the GAO and CRESST, among others have debunked them. The kind of criterion represented by NAEP reveals the problem with virtually all criterion referenced tests in education. In 1963 Bob Glaser at the University of Pittsburgh declared that for we can imagine achievement on a scale from zero competence to conspicuous excellence. Along that continuum we can describe other behaviors and reference the behavior of who we are interested in to those descriptors. That's criterion referencing. The problem is that Glaser and all the enthusiasts who took up the crusade didn't seem to realize that most educational disciplines are so vague domains that it's impossible to specify those benchmarks. Hence a lot of wasted time. -----Original Message----- From: arthur hu To: ClearingHouse ; Donna Garner Cc: gbracey@erols.com ; wa-ed-deform@egroups.com Date: Monday, February 08, 1999 10:21 AM Subject: Re: [education-consumers] Bracey -- NAEP Look at these statments again ----- On the 1994 NAEP, 4% of the Texas fourth graders who were tested were at or above advanced, 22% were at or above proficient, 54% were at or above basic, and 46% were below basic. No wonder we high school English teachers are constantly frustrated when only 4% of our students are able to do the very things that all high school students should be able to do. ---------------------------- This is why I have such a huge problem with criterion reference tests that are so far out of line with reasonable expectations. Statistially, the "expected" value is what you are most likely to get if you select any one individual at random. That's why the 50th percentile level is "grade level", with half above, and half below. On the NAEP scale, 50th percentile is just above the portion that scored in the "knows nothing" category, yet that's also the statistical definition of grade level. I don't care how stupid the kids in your state or my state are, the top 5% is normally what gets called "gifted", and that's what you will find in your wealthiest suburbs and private schools, and they are the kids you find in the Ivy leagues and top state universities like UC Berkeley. I don't think it's reasonable to expect all children, or even the top 50% of kids to perform at such a level. I went to school in 1976 in a 50th percentile school, and I don't recall too many kids beside myself who could perform at such levels either. Does anybody have any evidence that the general HS population was ever capable of such levels of performance, except in the days when only the top 25 graduated from high school and only 10% went to college, about 100 years ago? (Bracey also writes about being careful about the "good old days") BTW, the NAEP is a performance based test with lots of subjective rubric based grading. The WA WASL math test has a similar score, their reading test were more like 50% standard or better. arthur hu kirkland WA arthurhu@halcyon.com "fairness in diversity" http://www.leconsulting.com/arthurhu