From: Don Orlich [email@example.com]
September 18, 2001 11:36 AM
14 page paper
HIGH STAKES TESTS AND
September 20, 2001, NOON.
An Invited Address on Educational
Pullman, Washington Kiwanis Club, September 20, 2001
Dr. Donald C.
Professor Emeritus, Education and Science Instruction
Pullman, Washington 99164-4237
Momentum for contemporary
American school reform can be traced to the 1983 report A Nation at Risk: The
Imperative for Educational Reform sponsored by the National Commission on
Excellence in Education. Its most famous (or infamous line, take your
pick) states, "If an unfriendly foreign power had attempted to impose on
America the mediocre instructional performance that exists today, we might well
have viewed it as an act of war." Using a war metaphor
intentionally created, as David Berliner and Bruce Biddle called it, The
Manufactured Crisis: Myths, Fraud, and the Attack on America's Public
Schools, (1995) which is the title of their book.
During the 1980's educational reform had a number
additional factors driving it. Among them were:
… Global economic competition
Economic disparities between socioeconomic groups
… Exporting of
business-sector interest in education
decline in student achievement
changes in the schools
None of these alone could provide impetus for reform,
collectively these factors caused public education to become a focal
point for social and economic changes. The current enthusiasm for
educational reforms is politically motivated, that is, the drive and energy
comes from non-educational sources. As early as 1988, Bill Chance reported
that there were more than 275 educational task forces organized in the U. S.,
generating scores of reports to "fix the
Not to be left out of the action,
the nation's governors jumped on the reform bandwagon with their endorsement of
President Bush's (that's number 41) National Goals for Education.
These evolved into a package of eight--Goals
1. All Children in
America will start school ready to learn.
high school graduation rate will increase to 90 percent.
3. American students
will leave grades 4, 8, and 12 having demonstrated competency in challenging
subject matter including English, mathematics, science, foreign language, civics
and government, economics, arts, history and geography; and every school in
America will ensure that all students learn to use their minds well, so that
they may be prepared for responsible citizenship, further learning, and
productive employment in our modern economy.
4. The nation's teaching force
will have access to programs for professional development.
5. U. S. Students will be first in the world in science and mathematics
6. Every adult American will be literate and will possess the
knowledge and skill necessary to compete in a global economy and to exercise the
rights and responsibilities of citizenship.
7. Every school in
America will be free of drugs and
violence and will offer a safe, disciplined
environment conducive to learning.
8. Every school will promote
parental involvement and
participation to promote the social, emotional,
academic growth of children.
A full plate, no less, would you not
Something Funny Happened on the way to
Somewhere along the way to fixing the
schools, somebody--maybe everybody--forgot to ask, "What needs
fixing?" Rather than examining the underlying factors of poverty,
race, Third World immigrants, non-English speaking students, dysfunctional
families, absenteeism, violence, providing enhanced learning environments, and
methods of instruction that work (as opposed to the educational fads that do
nothing except make for rich consultants): The reform movement degenerated into
a high-stakes testing phenomena!
Kay Johnston and Heidi Ross
(2001) came to the same conclusion that the "national discourse on
education has narrowed to a single-minded focus on testable standards" (p.
On August 15, 2001, ABC News
Commentator, Geraldine Sealey, reported the same focus in "School Daze,
Test Craze?" Just recently, a lawsuit was filed in Indiana requesting
an injunction that would prevent Indiana from requiring special education
students to pass that state's exit test (Wrightslaw, August 14,
In the state of Washington at least
$40,000,000 was spent on developing the Washington Assessment of Student
(WASL) and approximately $60,000,000 more was appropriated to revise
it. There are about 30 states now using test scores to rate their
schools. We call them "high stakes tests" because they either
reward students, teachers or schools or penalize them. The rewards and
punishments vary. In California, raises and bonuses are given to teachers;
while in Alabama, a school's budget can be enhanced or reduced based on test
A major testing and scoring company, NCS
Pearson, made scoring errors in Minnesota resulting in 8,000 students being told
that they failed a high school graduation test they had actually passed.
Two years ago in New York City a scoring error by CTB/McGraw-Hill caused nearly
9,000 students mistakenly to be sent to summer school. NCS Pearson is
expected to score 40 million standardized tests this year alone! (Source:
Michael A. Fletcher, Washington Post, July 9,
In Washington State the
WASL has emerged as the "Mother of all tests" (apologies to Saddam
Hussein). The WASL is touted to be a criterion-referenced assessment, with
all questions based on the Essential Academic Learning Requirements
However, my examination of the 4th grade practice test showed this
claim to be establishment propaganda. In 1999, I predicted the number of
students who would not meet the WASL standard, i.e., fail it, almost
perfectly, only missing the mark by
exposed was a fraud in the mathematics items, which were deliberately made
harder and well beyond any fourth graders ability. On the writing exam,
they were required "to write a newspaper-style article explaining their
ideas." Do fourth graders read a newspaper? (See Orlich
2000b.) But let us examine just the 1999 fourth grade math WASL
Using only SPI sources (Study of Grade
4 Math Assessment, September 2000) the following data were published
showing the number of children by ethnic group who failed. (Bergeson 2000,
Tables G-10, 11,12, page 86 and Fig. G-3, page 91.)
1. ETHNIC GROUP FAILURE RATE ON THE 1999 4TH GRADE MATHEMATICS WASL (NOT
MEETING THE ARBITRARILY SET
The Total distribution of 4th
grader math scores falls into a classic bell-shaped curve (p. 91). To get
this type of curve on criterion-referenced tests is impossible. So, I
initially concluded that the WASL was in fact an IQ test. IQ tests yield
bell-shaped or normal curves because they describe the outcome of random
traits--intelligence, weight, height, shoe sizes. But when I plugged-in
the minority student scores, my hypothesis was rendered invalid, as no minority
child would have an IQ of 100 and we know that that conclusion is totally
wrong. My conclusion was changed so that the WASL is simply a Random
Meshuggene Quotient! (If you're not up your Yiddish, that is
But, that still did not account for
the errant distribution. My newest hypothesis is that the WASL is a very
expensive predictor of poverty. To test this hypothesis, data were
collected from 27 school districts in the North Central Education Service
District. Using "Free or Reduced Lunch" as the indicator for
poverty, I found that 27 of the 27 districts had 35% or more children on free
and reduced lunch. Twenty of the 27 (74%) of the 27 had 60% or more fourth
grade pupils fail the WASL, i. e, not meet the arbitrary standard set by the
In this cohort
were five school districts that have 56% or more Hispanic students. These
districts had a median WASL flunk rate of 76.5%. Two districts that were
predominately native American (45.8% and 95.4%) had WASL flunk rates of 80.0 and
96.3 percent respectively. Conclusions!
1. The WASL is marvelous predictor
2. The WASL shows that non-English speaking students
cannot pass an English writing examination.
can be mighty proud of the WASL and of our state educational leaders who have
publicly stated that they are "concerned" about these results.
Back to World Class
Ah, but recall that act of
war metaphor. How bad are we in the world? Let us examine the top 15
countries and states in the world for science for 13 year olds.
(Maestro, a drum roll please) goes to Singapore. The next 14 medals go to:
Colorado, Connecticut, Iowa, Maine, Massachusetts, Minnesota, Montana, Nebraska,
North Dakota, Oregon, Utah, Vermont, Wisconsin and Wyoming (NCES 98-500, May
1998). Did any newspaper proclaim that interesting finding in its
front-page headlines or its editorial pages? Don't hold your breath waiting for
positive news about public schools from the media.
How are we
doing in this state? Let's examine science for 13 year- olds. In
first place is Bulgaria, followed by the Czech Republic, South Korea, Singapore
and in fifth place the Evergreen State. The kids in Washington also tied
22 other nations and beat 14
How did the state of
Washington's 13 year-olds do in Mathematics in world competition? The
First place goes to Belgium, followed by, Czech Republic, France, Hong Kong,
Hungary, Japan, South Korea, Singapore, Slovak Republic, Slovenia, Switzerland
and in 12th place is Washington State.
Our children tied 19 other nations and
beat 10 others.
You want World-Class
standards? Well, we have them!
Oh, how many boat loads of Mercer Island
or Bellevue children have been caught trying to sneak across the borders into
Bulgaria, Hungary or South Korea? Think about the flow of immigrant
traffic and why they come to the good old U. S. of A.! And please do not
forget that the USA and Canada are the only industrialized nations that do not
segregate all children between the ages of 11 and 13 into two irrevocable
educational tracks--college preparatory and trade school. Our run-of-the
mill 13 year-olds compete very nicely, even without a level-playing
And where were the reform pundits on July 11, 2001 when
the College Board announced the following? "A new study of the TIMSS
International Study Center shows that Advanced Placement students [in the United
States] who score three or higher [on a five point scale] on physics and
calculus AP Exams outperform physics and advanced math students from the United
States and other countries in mathematics and science
executive director of the College Board's Advanced Placement Program stated
"These results demonstrate that students who do well on the AP Calculus and
Physics Exams are indeed at top of the world in academic achievement. . .
Was this hidden in the want-ad pages of our
newspapers? Why did it not make front-page reading; or any page
Review of School Reform
What has been done for school reform? Quite frankly, nothing of
note. In the state of Washington, we have squandered just under one
billion dollars on WASL development (which costs $27 per test to score compared
to $2.50 for the Iowa Test of Basic Skills) and other non-empirically tested
projects. Included is the highly touted "Schools for the 21st
Century." Let us spend a moment reviewing this political
brainstorm. My major source is from Shirley Basarab's report An
Overview of Student Assessment in Washington State (2001) that is yet being
delayed publication and dissemination by The Evergreen Freedom Foundation after
they requested it.
reported that the state of Washington never received a full report on the
Schools for the 21st Century project, only a file with a "hodgepodge of 782
pages of disconnected papers and news clipping. More than half was hand
written, scribbled notes" (p. 9). No comparative test data were
provided about these schools and yet, the legislature implemented the state's
reform movement on good feelings and high level political badgering. In
1997, Basarab cited data (p. 10) compiled from WASL scores comparing the
Schools for the 21st Century with the state and private school
TABLE 2. 1997 WASL AVERAGE
RESULTS FOR "SCHOOLS FOR THE 21ST CENTURY," WASHINGTON STATE AND
for the 21st Century
The state of Washington constructed its
reform structure on a foundation of sand. That is
the nicest thing I can say. Frankly, our state's K-12 educational
system has been subject to a monumental trail and error experimental
"AP Students with a '3 or Higher' Outperform
Advanced Math and
Physics Students both in U.S. and
Abroad." (July 11, 2001).
York: The College Board, News 2000-2001. Full report:
J. Gonzalez, K. M. O'Connor, J. A. Miles. (June 2001) How well do
Advanced Placement Students Perform on the TIMSS Advanced Mathematics
and Physics Tests? Boston: The International Study Center, Lynch
School of Education, Boston College.
Shirley. (2001). An Overview of Student Assessment
State. Olympia: Unpublished Manuscript, 158 pp.
and Bruce Biddle. (1995). The
Crisis: Myths, Fraud, and the Attack on
Schools. New York: Addison-Wesley.
Bergeson, Terry, et
al. (September 2000). Study of the Grade
Mathematics Assessment, Final Report. Olympia:
Superintendent of Public Instruction, 102
Goals 2000: Reforming Education to Improve Student
(1998). U. S. Department of Education. Washington, DC:
Government Printing Office.
Fletcher, Michael A. (July 9,
2001) "As Stakes Rise, School
Groups Put Exams to the Test:
Critics Decry Heavy Reliance
On Standardized Measures."
Washington Post, p. A 01.
"High Stakes Testing: Indiana Judge
Asked for Injunction so
Graduate." (August 14, 2001)
Johnston, Kay and Heidi Ross.
(August 27, 2001). "Teaching to
Higher Standards--From Managing to
Imagining the Purposes
of Education." Teachers College
Record. ID # 10804.
A Nation at Risk: The Imperative for
(1983). U. S. Department of
Education. Washington, DC:
U. S. Government Printing
Donald C. (2000). "A Critical Analysis of the Grade
Washington Assessment of Student Learning."
Context, 27 (2) Fall/Winter, pp. 10-14. (On
March 16, 2001, this
paper was honored as the "Outstanding
Affiliate Article Award" by
the 160,000 member Association
For Supervision and Curriculum Development, at
Conference in Boston.)
Sealey, Geraldine. (August 15,
2001). "School Daze, Test
News. Website: printerfreindly.abcnews.com.
U. S. Department
of Education, National Center for
Statistics, Linking the
National Assessment of
Progress and the
Third International Mathematics and
Eighth Grade Results. Eugene G. Johnson
Adriane Siegendorf. Project
Officer, Gary W.
Washington, DC: GPO, May 1998, NCES 98-500.
Word Count: 2,434
END NOTE: ON SEPTEMBER 17, 2001, ORLICH LEARNED ABOUT THE
FOLLOWING STUDY THAT VALIDATES HIS CONCLUSION ABOUT POVERTY AND WASL
SCORES. DUE TO LATENESS, THE DATA ARE NOT INCLUDED.
L. Abbott and Jeff Joireman. The Relationships Among Achievement, Low Income,
and Ethnicity Across Six Groups of Washington State Students.
Washington Research Center, Seattle Pacific University, July 2001. Jeffrey
T. Fouts, Executive Director. Website
Donald C. Orlich
PO Box 644237
(509) 335-4844 FAX (509)