z63\doc\web\2003\01\weawasl.txt
1/10/2003
WEA: Studies support call for major WASL fixes WEA continues to be
watchdog for WASL validity and reliability. 

As lawmakers consider student achievement accountability issues in
the upcoming legislative session, the reliability and validity of the
Washington Assessment of Student Learning (WASL) will be at issue. 

It's a topic about which WEA members have continued to express strong
reservations. 

Unlike the more familiar Iowa Test of Basic Skills, which has
significantly more history and is nationally normed, the WASL is a
new, state-designed assessment. As any test designer will
acknowledge, it takes three to five years for new tests to be fully
developed, piloted, administered, researched and revised based on the
research. A new test will not reach sufficient levels of validity and
reliability until the "bugs" have been worked out.

In the past year, WEA has taken a series of specific positions
addressing members' concerns about the WASL. These include opposition
to the singular use of the WASL for high-stakes decisions; a demand
that the cut score be researched and, if necessary, re-set using the
error of measurement; a call to reduce and prioritize the Essential
Academic Learning Requirements (EALRs); an appeal to analyze and, if
necessary, revise test items for EALR alignment, and a demand to
eliminate cultural bias and excessive complexity from the test items.
We also called for the WASL to be subjected to standard validity and
reliability research.

Backing up concerns with research

WEA researchers John Brickell and Doris Lyon have recently completed
a thorough review of all statewide WASL studies and technical reports
regarding WASL reliability and validity. Some of their key findings
reinforce WEA's call for the 10th-grade WASL to be de-linked from
high school graduation requirements:



The standard error of measurement for WASL tests is too large to
warrant use of the scores alone when making decisions regarding a
student's academic achievement levels. 

An independent investigation on the predictive validity of the WASL
found that the test is slightly less useful than SAT and ACT scores
in predicting college freshman GPA. High school GPA was the best
predictor. 

OSPI has not studied the validity of the cut scores for the purpose
of using the WASL as a high–school graduation requirement. 

WEA has been vigilant about monitoring the validity and reliability
issues regarding the WASL. We've been called whiners. We've been
accused of being naysayers. We were told educators were afraid of
being held accountable when we brought your legitimate complaints
about WASL validity and reliability forward. 

"A lot of good work has been done but there are still areas that need
to be addressed. For example, can the WASL predict any future
academic success for students across the grades or in college, or
job-related success?" says John Brickell, one of the WEA researchers
who prepared the report examining gaps in WASL validity and
reliability. "There's not a uniform level of difficulty across the
three grade levels in the four tests.

"In terms of reliability and validity, the area of greatest concern
is the reliability of the writing test. It is simply not high enough
to make individual decisions for high school graduation or grade
promotion."

Study says math WASL needs fixing

A new state-sponsored study, released last month, confirms what WEA
has been saying: The WASL is still a work in progress.

An analysis done by SRI International, a nonprofit research
institute, confirms concerns that the math test of the seventh-grade
WASL is more difficult for seventh-graders than the 10th-grade math
test is for 10th-graders, or the fourth-grade math is for
fourth-graders. Among the findings:



Twelve percent of the math problems on the seventh-grade WASL didn't
provide enough in the question for students to know how to answer
them. Seventeen percent of the student's score is based on test
questions with excessive or inappropriate information load. 

Many of the math problems on the 10th-grade WASL required use of
specialized background knowledge or specialized vocabulary. 

On the 2001 test, 15 percent of the math problems on the
seventh-grade WASL more closely matched the 10th-grade EALRs and 20
percent of the test items on the 10th-grade WASL more closely matched
the seventh-grade EALRs. 

The SRI report makes several recommendations to improve the WASL
standard setting and test item development process.

WEA will continue to monitor and call for WASL improvement, and will
continue to update our review as more research becomes available.
Independent research of the reading, writing and listening portions
of the WASL, similar to the research just completed on the math
tests, should be commissioned.

"We will be monitoring to ensure the SRI's recommended fixes are made
to the math WASL and implemented for future WASL test development,"
says WEA's Ann Randall, who is coordinating the campaign on testing
and WASL improvement. "And we intend to play a full role in the
OSPI's recent call to evaluate and, if necessary, re-set the
performance standards on all the WASL tests.

"We are going to insist on absolute transparency for all
state-commissioned WASL research and for a broad dissemination of
that research among all the education stakeholder groups."






Yahoo! Groups Sponsor ADVERTISEMENT 

Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.