Mathematically Correct slams the new
Clinton Math Test.
To: The LOOP:;, Eduction-Consumers ,
"Conservatives for Excellence in Ed."
From: pclopton@UCSD.Edu (Paul Clopton) (by way of James Kilpatrick)
Subject: Clopton to HR Comm. on Ed. and the Workforce
Date sent: Thu, 22 Jan 1998 02:29:53 -0600
MATHEMATICALLY CORRECT
There is a mathematically correct solution ...
PO Box 22083, San Diego, CA 92192-2083
http://ourworld.compuserve.com/homepages/mathman
mathman@compuserve.com
Testimony to the United States House of Representatives
Committee on Education and the Workforce
January 21, 1998
Thank you, Chairman Goodling and Members of the Committee, for the
opportunity to comment on the planned Voluntary National Test in Mathematics.
I am a biomedical research statistician at the Department of Veterans
Affairs Medical Center in San Diego affiliated with UCSD, and I have had
extensive exposure both to mathematics and to the principles of measurement
and test construction. I served on the 1997 California Mathematics
Framework Committee. I am a member of the San Diego Grade-Level Mathematics
Standards Committee and of math textbook adoption committees. I was an
author of the Algebra exam that the Mathematics Council of Western
Pennsylvania used in their 1997 competition for middle school students.
However, I am not representing any of these organizations in my comments to
you.
I come, instead, as a cofounder and representative of Mathematically
Correct, a parents' advocacy group for mathematics education, and as the
father of two children in public school. Mathematically Correct was founded
in 1995 by parents who were frustrated by the weaknesses we saw in math
education in our public schools. We quickly found that we were not alone in
our frustration, and we are now supported by parents and mathematicians
across the United States. Indeed, through our voluntary efforts, we have
become the most widely recognized voice for parents' concerns about
mathematics education in the country.
Our Call for Voluntary Examinations in Mathematics
In April of 1996, Mathematically Correct released a position paper [1]
calling for voluntary regional or national examinations covering the
standard contents of each secondary course. It was, and still is, our
feeling that the establishment of clear expectations, and the publication of
examination results based on these expectations, could go a long way toward
improving mathematics education in this country. Two important points we
noted are:
1) The course expectations and the related examinations should be prepared
by committees of mature, well-established mathematicians; and,
2) The examinations should be externally developed and graded, independent
of the existing educational infrastructure in this country.
A Good Idea Goes Wrong
Given the similarity of our recommendations to the plan for a national test
in mathematics, it might be expected that we would be strongly supportive of
such a plan. In reality, however, we find the plan for the test so
objectionable that we are strongly against it. In August of 1997,
Mathematically Correct wrote letters to President Clinton [2] and to the
test design committee [3] outlining our criticisms of the test plan.
To put it bluntly, we feel that the planned test would be worse than no test
at all. It is difficult to clarify the detailed reasons for our objections
in a short statement, but some of the central issues can be highlighted.
A Schism in Math Education
Whether you are aware of it or not, there is great controversy about math
education in America today. The members of Mathematically Correct quickly
discovered that the weak programs our children encountered were stimulated
by what has been called a "reform" movement in math education. Although
praised by certain groups of educators, many parents see the products of
this "reform" as dumbed-down math programs. Although promoted by flowery
but empty rhetoric from some education groups, these inadequate programs are
responsible for the birth of Mathematically Correct in the first place.
One cannot fully appreciate the inferiority of these programs merely on the
say-so of irate parents. I urge you to embark on an educational experience
of your own. Look at Glencoe Interactive Mathematics: Activities and
Investigations [4]. The third book in this program, units 13 to 18, is the
8th grade text for this series. This program was highly rated in California
and is now in use in many of our public schools. Yet, the "reform" math
covered in this series is grossly inadequate. In fact, these are the worst
math books I have ever seen.
Unfortunately, the committee that designed the specifications for the
national test in mathematics was heavily laden with individuals known to be
tied to the very "reform" movement that brought such textbooks into
existence. There is far too much wrong with this "reform" to be detailed
here. The point, however, is that parents see the "reform" as a serious
threat to rigorous math education, yet the design committee was strongly
biased in favor of it. With Professor Dossey at the helm, they produced an
unbalanced test plan that is headed in the wrong direction - one that would
specifically support the movement we find so objectionable.
Test Characteristics Make a Real Difference
Without thorough study, it is difficult to understand how a mathematics test
could be slanted in one way or another to any great extent. Let me
summarize just one example that is documented on our web site [5]. Here,
traditional and "reform" introductory algebra programs were run
simultaneously in the same school. At the end of the year, students were
evaluated in two ways - with a regular, objective final exam and with
California's Golden State Exam. The Golden State Exam is used for
achievement recognition and is modestly slanted to favor the "reform" programs.
The results were that 55% of the traditional program students earned A or B
grades on the final, while only 11% of the "reform" program students earned
A or B grades. Also, 80% of the "reform" program students earned D or F
grades, while only 31% of the traditional program students did this poorly.
The traditional final exam shows a huge difference between the programs.
In contrast to these findings, the proportion of students receiving either
honors or high honors on the Golden State Exam was the same for both math
programs. Not only that, but more "reform" program students earned
recognition on the Golden State Exam than earned a C or better on the
regular final exam. Clearly, the nature of the test dramatically alters our
conclusions.
Recognizing that the tests can be designed in a way that hides dramatic
differences in student achievement, we need to look at some of the specific
problems with the National Voluntary Test plan.
Arithmetic and Algebra
There are two important areas in mathematics that are seriously slighted in
so-called "reform" math and in the planned exam. These are known as
arithmetic and algebra - subjects that parents perceive as vital for student
success. Even President Clinton, in his ten-point call to action for
American education, states that, "...every 8th grader should know basic math
and algebra." [6]
With respect to the operations of arithmetic, the design committee
intentionally avoids testing basic computational skills under the assumption
that all students will know these things. Unfortunately, this is far from
universally true among American 8th graders. Indeed the National Assessment
of Educational Progress (NAEP) indicates that 21% of 8th graders are not
proficient at the level of adding, subtracting, multiplying and dividing
whole numbers and solving one-step problems [7]. Failing to directly
address these basic skills appears to be nothing less than an attempt to
hide weakness even at the lowest levels.
The exam ought to reflect the President's prescription that all 8th graders
will learn Algebra. This is an important goal for our country. We cannot
monitor our progress toward that goal if it is not measured. In spite of
calling 25% of the test "Algebra," the content addressed is not the content
of an introductory algebra course at all - it doesn't even cover
pre-algebra. It appears that the test planners want to fool the country into
believing that these American students have learned algebra. By ignoring
the demanding content of algebra, the design will fail to measure success at
the level of our international competition. It will also fail as a vehicle
to promote greater mathematics achievement.
We have a link on the Mathematically Correct web site to samples of math
problems given to Japanese 12-year-olds [8]. The level of these problems
obviously exceeds what is planned for American 8th graders. This is clear
evidence that the planned exam is not designed to measure or promote high
levels of achievement.
Calculators, Guessing and Non-Standard Administration
Under the guise of allowing multiple solution strategies, an effort is being
made to design test items that promote guessing rather than more powerful
and general analytical methods. This works against the goal of promoting
accuracy, clear thinking processes, and careful work. This is one of the
ways "reform"- friendly tests can be constructed to disguise limited
achievement.
Similarly, the more successful countries discourage the use of calculators
in the early grades, yet the test design would effectively promote their
use. Although these "reform" educators believe that calculators promote
math learning in young students, over 80% of the public believes that their
use should be limited [9, 10]. It is distressing to read the minutes of the
design committee where we learn that they want even more calculator use than
the public might accept, so they planned to build to even more calculator
use over time. Flagrant disregard and disrespect for the opinions of
parents and the general public cannot be tolerated.
Worse yet, the policy on calculator use, and even the time permitted for the
exam, seems to allow differing examination conditions from one locale to
another. Thus, the test-administration will be inconsistent or
non-standard, greatly reducing the validity of scores for comparison purposes.
Unreliable Methods and Credit for Trying
The test plan calls for a large number of hand-scored items despite the fact
that these items are known to suffer from subjective grading and low
reliability. The fraction of total points dedicated to these items is high
as they are given more weight than objectively scored items. It is likely
that subjectively-scored questions will have a substantial, and artifactual,
impact on scores.
Unreliable, subjective items are endorsed by the so-called "reform"
educators. These items open the door to giving credit for wrong answers, as
long as the scorers feel that the student had a good approach to the
problem. In fact, this method not only leads to inconsistency in grading,
but graders may be influenced by the students' political correctness,
attitudes, and degree of social insight. These characteristics have no
place in evaluating math achievement. Yet, it is this very sort of
encroachment that contributed to the downfall of the "reform"-oriented CLAS
test in California.
Subjective items also bring a significant increase in the cost to administer
the test and delay the return of results to the concerned parties. For
these reasons, the number of subjectively-scored items and the fraction of
total points they represent should be greatly reduced.
Psychometrically Incorrect
Non-standard conditions of administration and subjective and unreliable
grading techniques are clear psychometric errors. They are inconsistent
with established, reliable assessment practices. These weak forms of
assessment should not be considered to be valid for making inferences
regarding the achievement of individual students. This places the test
design in direct conflict with the stated goals of the testing program.
Unlike the NAEP, which is intended to provide data on large, aggregate
units, the test plan is supposed to provide meaningful data on individuals.
But the range of individual achievement is necessarily much broader than for
aggregate units such as means for entire states. For such a plan to
succeed, it needs to cover an even broader range of achievement than the
NAEP - both higher and lower. For a test to be diagnostically useful, it
would also benefit from a greater number of more specifically defined
sub-scale scores. None of these features are provided.
Finally, the notion that the examination can and should be modified over
time, as suggested by the design committee, is also a serious mistake. We
cannot track progress over the years if the target keeps moving. The
intention of the committee to use changes to the examination as a scheme to
gradually pull Americans toward their own philosophy is an insult and is
inappropriate if we are to measure our progress as a nation.
What is Correct for California?
After leaping head-first into the "reform" movement and falling to the
bottom of the achievement ladder, California is now setting clear and
ambitious goals. Many have noted that our newly adopted Math Standards
[11] expect a higher level of achievement than those of any other state, and
I would have to agree with them. Our Standards and Draft Framework [12] are
geared toward completing the primary school math topics by grade 7 with
named content areas, like algebra and geometry, starting in grade 8. We are
committed and legally bound to developing tests that are designed to measure
the content of the our state Standards.
Importantly, our algebra standards are real algebra, not just algebra in
name only. While we want all of our students to learn introductory algebra
in grade 8, it is critical that they learn the real thing. Algebra in name
only for 8th grade students is nothing more than the basis of an
advertisement or public relations campaign, and is not what we are
interested in. Our standards-based exam in algebra will be an algebra test.
The planned voluntary test is not.
Baring unforseen and drastic changes in the national test plan, the 8th
grade math test will be of no value to California. If it is funded and
developed as planned, Mathematically Correct will recommend that our own
state not waste student time by participating. We would make the same
recommendation to any other state that sincerely wants to move their
students to a high level of achievement.
Alternatives to the Test Plan
The management of the national test plan has been moved to the National
Assessment Governing Board (NAGB), the semi-independent group that oversees
the NAEP. This is a move in the correct direction. It is even possible that
the NAGB will institute a wholesale revision to repair the test plan.
Whether or not the Board has the fortitude to correct the flawed design is
open to question. The same educators that are behind the "reform" movement
will pressure the Board as well. Even the NAEP faces these pressures. The
two math consultants for the NAEP Mathematics Framework itself are none
other than Professor Dossey and Cathy Seeley, both listed as "program
conceptualizers" for the text [13] that has come to be known as "Rain
Forest" Algebra [14-15]. Given this situation, we are not optimistic about
the prospects for a wholesale revision to repair the test design. The NAGB
would have to be willing to take drastic steps if there is to be any hope of
a balanced approach to a valid, reliable, and useful test plan.
There has also been talk of using existing tests as an alternative, noting
that many states already conduct their own assessments. It is clear that
these off-the-shelf tools are less than optimal. They do not address
specific course contents and do not measure up to the achievement goals our
country should be striving for. However, the Voluntary National Test, as
planned, does not improve upon this situation. In many ways, it actually
makes things worse. Mathematically Correct would rather see existing
assessment tools or no test at all than the use of the Voluntary National
Test as planned.
Our Conclusion
As representatives of concerned parents throughout the country, the members
of Mathematically Correct implore you to take every effort to address these
issues. We could benefit from a well-structured national assessment device,
but we clearly don't need a tool to further promote the "reform" agenda in
math education. Short of a balanced, major overhaul in design, the test can
be expected to do more harm than good for the mathematical education of our
nation's youth.
Respectfully Submitted
by
Paul Clopton
Member and Cofounder, Mathematically Correct
Notes:
[1] A Program for Raising the Level of Student Achievement in Secondary
School Mathematics,
Frank B. Allen, http://ourworld.compuserve.com/homepages/mathman/allen.htm
[2] Letter to President Clinton regarding the 8th Grade Mathematics Test,
Mathematically Correct
http://ourworld.compuserve.com/homepages/mathman/letter.htm
[3] Comments on the National 8th Grade Mathematics Test, Mathematically Correct
http://ourworld.compuserve.com/homepages/mathman/nvtm.htm
[4] Interactive Mathematics: Activities and Investigations, David Foster,
Sandie Gilliam, Jack Price, et. al., Glencoe Division,
Macmillan/McGraw-Hill, Westerville, Ohio, 1995.
[5] Effectiveness of CPM vs Traditional Math, Robert W. Haswell
http://ourworld.compuserve.com/homepages/mathman/study1.htm
[6] President Clinton's Call to Action for American Education in the 21st
Century
http://www.ed.gov/updates/PresEDPlan/part1.html
[7] NAEP 1996 Mathematics Report Card for the Nation and the States
http://nces.ed.gov/NAEP/96report/97488.pdf
[8] Japanese Math Challenge, Pacific Software Publishing, 1996.
http://www.japanese-online.com/math/index.htm
[9] What Do Parents and the Public Think About Our Schools?, Public Agenda.
http://www.aft.org/page34.htm
[10] 1996 State of the Union Address, Albert Shanker, President, American
Federation of Teachers.
http://www.aft.org/convention/sofunion.htm
[11] State Board of Education Ad Hoc Draft California Mathematics Standards
http://www.cde.ca.gov/board/
[12] Mathematics Framework for California Public Schools: Kindergarten
through Grade Twelve, Draft Sept. 5, 1997,
http://www.cde.ca.gov/cilbranch/eltdiv/mathfw.htm
[13] Focus on Algebra [Addison-Wesley Secondary Math: An Integrated
Approach], R.I. Charles and A. G. Thompson, et. al., Addison-Wesley, Menlo
Park, California, 1996.
[14] 'Rain Forest' Algebra Course Teaches Everything but Algebra, Marianne
Jennings
http://ourworld.compuserve.com/homepages/mathman/rainbow.htm
[15] More on Addison-Wesley Focus on Algebra, Richard Askey
http://ourworld.compuserve.com/homepages/mathman/aw.htm