\doc\web\98\07\standset.txt


I went down to Olympia to see this, you can login to the internet at
http://www.tvw.org/\scomms\senedu.htm to listen to the work session,
I can also send out a cassette to anyone that wants it for $4 or put
you on a mail chain if more than one person is interested.


There is _no_ written standard. The EALRs do not specify how many you
need to get right. Since the OSPI and Commmision on Student Learning
now claim that they reserve the right to put questions of any grade

level on the test, the EALRs do not specify which skills you need to
pass the test since with the sample content of 40% over-grade level
that I found, you'd have to get 100% of grade level problems plus a
few over grade to "meet" standard.


 A committee sets the "pass point" This is how we can have a test
that says 80% of 4th graders are below grade level and nobody thinks
that's not crazy when 50% is exactly what we can expect the average
kid to do.




First the test questions are developed by the test developer. It's
supposed to be based on the EALRs ,but that doesn't explain why the
4th grade test has 7th and 10th grade questions on it. Now some OSPI
people told me that they deliberately put over-grade level questions
on it, but there is no documentation that says such questions are
allowed, logically, if a 4th grader got a 10th grade question wrong,
it would meet the standard, right?


Then they give the test. 


The "rubrics" are purposely "vague" because they have to round up all
200 testers and look a sample of the 50,000 responses to figure out
all of the possible answers and which ones will get a 1,2,3, or 4.
The testers are the sort of seasonal people that work for HR block,
they are not paid as well as teachers. A 10% rate of disagreement
(that's error to the rest of us) is considered acceptable. A
"consensus" process is used to determine the "training" documents
that graders will use to grade each problem (btw, the questions and
the final rubrics are top secret, nobody can 'look at them to see if
they did anything that made sense, such as the OSPI posting incorrect
solutions to two math problems)


Once the questions have all been graded from 1 to 4, then they take a
team of 40 people over 3 days, mostly 4th grade teachers, to look at
the problems sorted from easy to hard based on % that students got
"correct". BUT they are not allowed to see what % got it corrrect
(it's more important that a teacher thinks a problem can be expected
of an average 4th grader than knowing that only 2% got a problem
right). They do NOT have the opportunity to toss out problems that
are not compliant. They only set the "pass point" which is somehow
related to the number they need to get corrrect. The final point is
reached by "consensus", but this "standard" is a secret since the
questions themselves are secret.  Nowhere in the sample test is this
"standard" shown. 


This standard is set ONCE at the start of the process, and subsequent
tests are "equated" to the first by repeating some questions from the
1st test, and "equating" them with how students do on these problems.
It was not explained how this guarantees that 2 tests where the 2nd
year got 50-100% more passing the standard is just as hard as the 1st
year. We just have to "trust" them, just like they insured us that
all problems come from the 4th grade EALRs, and that all questions
are scored with computer like 90% accuracy (compared to 0.0000%)




9/18/98 - Work session Topics Include: Presentation on the process
used to set the standards for the Washington Assessment of Student
Learning (WASL) by Gordon Ensign, Commission on Student Learning.
Presentation on the scoring process used for the Washington
Assessment of Student Learning (WASL) by Nina Metzner, National
Computer Systems.


     3/2/98 - Work session Topics Include: Presentation by Berit
Kjos, author, Brave New Schools.