zip74\doc\web\2003\09\wasl2003.txt
Revised Sept 15, 2003
See Arthur Hu's Powerpoint on the 2002 WASL Math study here:
http://www.arthurhu.com/2002/10/mathstud/mathstud_files/frame.htm
To: ghall@ospi.wednet.edu
Cc: bneitzel@ospi.wednet.edu
Subject: Review of WASL and answer key as classwork
(Note I got no response to this message)
Hello Greg, Arthur Hu here. I saw this year's 4th grade WASL, near
as I can tell, it's still about 50% wrong, either obviously at the
wrong grade level, or requires significant work beyond straightforward
application of proficient 4th grade knowledge and skills.
Even though some of the problem items were removed, some items I
complained about are still in the current test.
I'd also like to ask your opinion about some classwork that I've been
shown - every item is effectively an answer key which shows you
exactly how to solve an actual WASL item, obviously after inspecting
the actual test. I've spoken to principals who believe that this is
entirely appropriate. It's about in the middle of the review. This
might be very widespread.
Since I'm still the only person in the state that can figure this
stuff out, doesn't that make me qualified to be the next assessment
director??
Kirkland Parent reviews WASL 4/5, 50% problems wrong, WASL answers
issued as classwork
May 5, 2003
Highlights
- Most parents don't know they can inspect the test
- Mr. Hu was one of the original complainers leading to 2000 study
- Mr. Hu concluded that revised 2002 test was also flawed and that
test difficulty, passing cut score, specifications, and standards
were adjusted, making comparison impossible
- The 2003 math test still has over 50% problems not compliant with
4th
grade standards or cognitive levels, includes some problems he had
complained about in 2002.
- One 4th grade problem was already declared faulty after printing,
and
not allowed to be given to students.
- Some schools are issuing class worksheets that are virtual answer
sheets to WASL problems with only a few details changed.
- Some students still being assigned old WASL problems which have
since
been determined to be not compliant with grade-level EALRs as
"problem of the week".
- The science test tests hardly any facts, and does not use standard
terminology. Most is high school or college level lab skills.
- Scores won't even be returned since nobody knows what a reasonable
"standard" is.
PARENT EXERCISES RIGHT TO INSPECT 4TH MATH AND 5TH SCIENCE TEST
I finally got a look at the 2003 4th grade WASL, and 5th grade science
WASL. It's a shame that the press have never notified the public of
their right to inspect the test if their child is taking the test, let
alone that this parent has done so and gone to OSPI with specific
complaints of problems with the WASL test.
5TH GRADE SCIENCE TEST WON'T EVEN BE GRADED
The 5th grade science test won't even be graded, so it's a real waste
of time. The least they could do is return a % correct, and how a
student did ranked compared to everybody else. If everybody gets only
20% right (which might be what comes back), that's something the
public should know. This only shows that the concept of "standard" is
competely invalid if it depends on the acceptance of a a valid test
and acceptable cut score.
(Note - Sept 2003, the superintendent says he got a look at the
science scores, but for some reason they are not being made public)
The 4th grade math test was the one that was so bad that the state
legislature forced the OSPI to look at it. The OSPI concluded it was
perfectly fine except for the 10% of problems that were competely
wrong and 25% that had other problems.
NOT JUST TEST, 2002 STANDARDS WERE ALSO MOVED AROUND
The 2002 test was the first "fixed" test reflecting not only the
revised test, but what I revealed were also revised standards and item
specifications, so when I had complained that items like mean, median
and mode were in the 7th grade, they moved them down into the 4th
grade.
2002 TEST STILL OUTRAGEOUSLY BROKEN AFTER 2001 "FIX"
What I found was the 2002 test was still outrageously broken, even
though it was less blatant about putting 10th and 7th grade items into
4th grade. About half of items were clearly not compliant with 4th
grade specifications or EALR standards, or had standards changed from
their original 1997 level without notice. Even when mean/average was
moved down to 4th grade, one problem asked students to chose the
proper method of computing an average when the specification clearly
states that students are not to be asked for a definition, and that
the definition of average was its method. Other problems, such as
picking out the winning square of a 10 x 10 tic tac toe game required
game and IQ skills clearly outside the scope of merely picking out a
row and column from a grid. Many of the spinners were not clearly
marked to allow students to easily determine which areas were larger
than others. No protractor is supplied, nor are 4th graders instructed
how to use protractors to measure the angles neccesary to solve these
problems. In contrast, the 10th grade spinners were simple to compare
without the use of protractors. Moreover, computing and comparing
probability as ratios, percentages, or areas is a 7th grade level
skill, not 4th grade. 4th grade only specifies certain vs uncertain.
Another problem asked which square would deliver a win in the next
move in the Japanese game of GO. Specifying a letter and number for
the square might be fair game, but testing on strategy is clearly
out of scope for a 4th grade proficiency test. It appears that nearly
all of the problems are designed to guarantee that if all you know are
basic 1 step 4th grade math skills, you won't be able to solve these
problems, which is also true of nearly all NCTM (National Committee of
Teachers of Mathematics ) standards based problems.
OSPI ASSESSMENT DIRECTOR ADMITS THEY GOOFED BIG TIME, BUT ONLY TO ME
Greg Hall, assessment director at OSPI at first said that none of my
complaints were valid. After meeting with him in September, he still
refused to talk about items which were not released, but fortunately,
half of the items were released. Then he admitted that most of my
complaints were valid, and they reviewed and removed many items from
the 2003 test and item bank. Furthermore I observed that the test
seemed a lot easier than the sample test, and that items had moved
around in the 2001 revision of the standards.
OSPI AGREES TO MASSIVE FIXES FOR 2003 TEST
Hall admitted that the
test was deliberately easier. They indeed had changed the cut score,
and they had even changed the standards and item specifications
without any notice. This makes any valid comparison between years
impossible
2003 TEST IS STILL MASSIVELY BROKEN
OVER HALF OF WASL PROBLEMS ARE DEFINITELY OVER GRADE LEVEL, OR
DEVLOPMENTALLY INAPPROPRIATE FOR 4TH GRADE
In the 2003 test, I still come up with about 20 of 40, or 50% of
problems that are either completely beyond the 1997, 2001 standard, or
way beyond simple 1 or 2 step application of 4th grade skills w/o
excessive construction of a complex method. In other words, a student
who knew 4th grade arithmetic, geometry, and statistics backwards and
forwards of average IQ could still get over half of the problems
wrong. I still recognize 2 or 3 problems I had complained about in the
2002 test, such as requiring students to compute an average without
being told the method for computing an average, or extrapolating a
timetable from rush hour to the middle afternoon.
PROBLEMS IN VIOLATION OF 2002 EALRS STANDARDS MANUAL
Some of state standards that are violated:
The Essential Academic Learning Manual is on the web at:
http://www.k12.wa.us/curriculuminstruct/documents/EALRs-Technical-Manual
.doc
- rate is grade 7
Grade 4: blank
Grade 7: Understand and apply the concepts of ratio and direct
proportion.
- Compute the fraction of a number in millions is grade 7
Grade 4: Add, subtract, multiply, and divide whole numbers.
Grade 7: Add, subtract, multiply, and divide nonnegative fractions
- Computing average or mode is grade 7 (2 problems)
Grade 4: Understand.... such as mean, median, and mode
Grade 7: **Calculate** and appropriately use ... measures of central
tendency
- Creating a survey method r (1 problem)
2001 study concluded that constructing a method was not allowable.
- Computing or comparing probability as a ratio other than certain or
uncertain is grade 7. Several spinner problems violate this.
Grade 4: Understand the difference between a *certain and uncertain*
event.
Grade 7: Know how to *calculate numerical measures* of chance for
simple
events.
- Very complex "slice" problem "how many slices per person" is a
fraction word problem that really require an algebraic solution
Grade 4: Evaluate simple expressions using blocks, sticks, beans,
pictures
Grade 7: Solve simple equations using blocks, sticks, beans, pictures
- one problem using stacked bar charts was so botched it was crossed
out entirely and not given to students
- Sorting triangles without a clue as to what to sort them by
Grade 4: triangle isn't even mentioned
Grade 7 standard:
describe...including..triangles, isosceles, equilateral [there is not
mention
of right triangle at all]
- Figure out without a clue that a frequency histogram (sometimes line
plot)
would be the best way to solve a problem.
The way the EALR standard seems to allow this, but this is really
beyond
the developmentally appropriate level for a 4th grader without a hint.
- Create a fully chart as fully labled as Excel or Powerpoint from a
nearly blank page.
Now the EALR standard does mention drawing charts, but it's pretty
taxing to expect every 4th grader to be able to whip out a fully
labeled chart complete with axes just like Excel. As a graphics
programmer who has created software to do this at the age of 20, this
is no mean feat. Filling in the blanks or maybe part of the data would
be reasonable for an assessment.
Grade 4:Organize and display data in numerical and graphical forms
such
as
tables, charts, pictographs, and bar graphs.
Grade 4: Draw and build simple shapes and figures using appropriate
tools, such as a straightedge, ruler, protractor, or nets. [Now this
is more reasonable]
WASL ANSWERS AS WORKSHEETS
Very interesting that someone has sent me 4th grade hand-made
worksheets that were VERY similar to many of the new WASL problems,
except that they were closer to being appropriate - they gave hints.
Example -
Organize this data on a line plot so you can answer the questions:
12 14 11 6 15 10 6 12
13 9 8 7 9 12
11 9 10 10 12 12 11
x
x x x
--------------
6 7 8
How many students could do more than 11 sit ups?
Could more than half do at least 10 sit ups?
The WASL doesn't even hint as to what sort of graph would be helpful.
Here's another worksheet problem: Match the triangle
right obtuse acute
The actual WASL wants you to match triangles, but gives no hint as to
what categories items should be sorted by.
According to the standards, such categorization is grade 7.
Standards:
Grade 4: triangle isn't even mentioned
Grade 7 standard:
describe...including..triangles, isosceles, equilateral [there is not
mention
of right triangle at all]
I have these sheets avaiable to fax to anyone who is interested.
I have no idea how widespread this is, but this gives a whole new
meaning to teaching to the test. This probably explains why a few
schools did VERY well on WASL, too well. There is nothing to keep
teachers from giving out what amount to solution methods before the
test. For that matter, there isn't anything to keep parents from
passing on hints as well.
LAKE WASHINGTON ADMINISTRATION SAYS "IMPOSSIBLE"
A letter that I got from my principal and Superintendent said that
there was absolutely no relationship between these worksheets and
unreleased test problems, even though my son brought these home
shortly before he took the WASL test. Perhaps it is not surprising
that AG Bell's 4th grade math pass rate just about doubled to 70
percent, above state average, as did a couple of other area
elementary schools which I suspect may have done the same thing.
90th PERCENTILE STUDENT FAILS WASL STANDARD
My son Peter, who scores about the 90th percentile on standardized
ITBS math, and well over 60 on verbal nevertheless failed to pass
the listening test. You only have to fail one test to fall "below
standard". You can see my comments on the listening test below, which
goes way beyond simple listening skills, but high level analysis
of plot and characters.
PROBLEMS IN VIOLATION OF COMMON SENSE
The 2001 revision of the EALRs includes Mean, Median and Mode. Mean
is the average, Median is the halfway point, Mode is the most common
data point. 3 or 4 problems out of 4 is a huge proportion, are
devoted to these concepts. However, one New York professor of
mathematics noted that Median and Mode are inappropriate even for
inclusion on the high school level Regents examination as most
mathematicians are even very familiar with the concepts. Also, neither
the college entrance SAT or ACT tests include either median or mode.
Even when they appear on the EALR standards, COMPUATATION of these
figures is pegged at grade 7, yet the 2002 or 2003 WASL tests require
computing the mean, median, and mode.
VERY LITTLE 4TH GRADE MATH
There are almost no examples of straightforward 4th grade level
arithmetic - subtracting, adding, multiplying or dividing 3 or
2 digit whole numbers. They're multiplying a number in the millions
by a fraction which has absolutely nothing to do with 4th grade level
math by the state's own standard.
READING PROBLEMS
Some of the reading was based on a book I saw used in some classrooms.
Students who had used that book would be at an advantage.
I was bothered by the sort of question - the BEST reason or the BEST
conclusion would be, when 2 or all 3 choices were in some degree true.
LISTENING
I was appalled by the number of subjective questions without even
seeing
the passages being read. Cindy was a) close b) distant c) indifferent
to
Jack.
WRITING A TOTAL JOKE
The writing prompt is a total joke. It consists of two lines, and is
just like "You walk by a hole in the ground with a ladder leading
down. Write 2 pages about what happens next".
If you don't have a very vivid imagination, you'll be sunk. It's
completely creative process. If you're an inner city deprived kid,
it's hard to imagine them putting much into this. On the other hand,
if you've been outdoors, or seen a lot of movies or read a lot of
books, you might do fine. There is no way the students are even
writing about the same thing. If they were doing a book report, or
writing about a specific historic event, it would be more fair.
The "standard" must be completely subjective. How you'd take a piece
of writing and declare it a 0, 1, 2, 3 or 4 is completely nuts since
writing ability of 4th graders is all over the place.
Science
Most of the science is interpreting experiments, or asking studnets to
write how they would set up an experiment. I never had this this stuff
in 5th grade, and 5th grade classrooms don't have science labs.
They're asking kids how many 100ml beakers and how many ml of solution
to use, and which variable are changed, or which ones to change. Stuff
I, as a kid who went to engineering school by age 18 didn't need to
worry about until high school. None of the lab scenarios required any
knowlege of science fact.
The few questions that required knowledge of facts covered topics such
as physics that weren't typically covered in K-12 in my days until
high school physics. Even then, standard terms such as "potential" and
"kinetic" energy aren't even used by WASL.
This is a completely useless test since science isn't really a K-6
topic at all. Even at the high school level, traditionally, only
college prep students even took science, and not all student took the
same number of the standard biology, physics and chemistry classes.
This goes back to the traditional reason for sticking with verbal and
math aptitude tests, because this is something that all kids do, and
all
kids
are taught at some level.
It's useless to test kids on material they haven't been taught, then
expect the schools to align their curriculum to what amounts to an
invalid test / standards.
NOT TOO LATE TO STOP GRADUATION REQUIREMENT
The statement that the WASL will be a graduation requirement isn't
really true. The State Board must find the test to be valid as a
graduation test, or the legislature could ban the State Board from
making such a requirement. Every testing manual, even the WASL
technical notes say that no single test score should be used as a
condition that can deny a student a diploma or grade-level promotion.
Every politician and educator who promotes this idea is in violation
of
this very simple rule of education.
A standard isn't a test or a passing score. A content standard can't
fairly set a passing performance standard, especially in a school
system whose goal and mission is to educate every student at every age
and ability level, up to the age of 18.
Arthur Hu
arthurhu @ comcast.net
425-891-2619
Kirkland / Lake Wash School Dist.