Mathematically Correct slams the new Clinton Math Test. To: The LOOP:;, Eduction-Consumers , "Conservatives for Excellence in Ed." From: pclopton@UCSD.Edu (Paul Clopton) (by way of James Kilpatrick) Subject: Clopton to HR Comm. on Ed. and the Workforce Date sent: Thu, 22 Jan 1998 02:29:53 -0600 MATHEMATICALLY CORRECT There is a mathematically correct solution ... PO Box 22083, San Diego, CA 92192-2083 http://ourworld.compuserve.com/homepages/mathman mathman@compuserve.com Testimony to the United States House of Representatives Committee on Education and the Workforce January 21, 1998 Thank you, Chairman Goodling and Members of the Committee, for the opportunity to comment on the planned Voluntary National Test in Mathematics. I am a biomedical research statistician at the Department of Veterans Affairs Medical Center in San Diego affiliated with UCSD, and I have had extensive exposure both to mathematics and to the principles of measurement and test construction. I served on the 1997 California Mathematics Framework Committee. I am a member of the San Diego Grade-Level Mathematics Standards Committee and of math textbook adoption committees. I was an author of the Algebra exam that the Mathematics Council of Western Pennsylvania used in their 1997 competition for middle school students. However, I am not representing any of these organizations in my comments to you. I come, instead, as a cofounder and representative of Mathematically Correct, a parents' advocacy group for mathematics education, and as the father of two children in public school. Mathematically Correct was founded in 1995 by parents who were frustrated by the weaknesses we saw in math education in our public schools. We quickly found that we were not alone in our frustration, and we are now supported by parents and mathematicians across the United States. Indeed, through our voluntary efforts, we have become the most widely recognized voice for parents' concerns about mathematics education in the country. Our Call for Voluntary Examinations in Mathematics In April of 1996, Mathematically Correct released a position paper [1] calling for voluntary regional or national examinations covering the standard contents of each secondary course. It was, and still is, our feeling that the establishment of clear expectations, and the publication of examination results based on these expectations, could go a long way toward improving mathematics education in this country. Two important points we noted are: 1) The course expectations and the related examinations should be prepared by committees of mature, well-established mathematicians; and, 2) The examinations should be externally developed and graded, independent of the existing educational infrastructure in this country. A Good Idea Goes Wrong Given the similarity of our recommendations to the plan for a national test in mathematics, it might be expected that we would be strongly supportive of such a plan. In reality, however, we find the plan for the test so objectionable that we are strongly against it. In August of 1997, Mathematically Correct wrote letters to President Clinton [2] and to the test design committee [3] outlining our criticisms of the test plan. To put it bluntly, we feel that the planned test would be worse than no test at all. It is difficult to clarify the detailed reasons for our objections in a short statement, but some of the central issues can be highlighted. A Schism in Math Education Whether you are aware of it or not, there is great controversy about math education in America today. The members of Mathematically Correct quickly discovered that the weak programs our children encountered were stimulated by what has been called a "reform" movement in math education. Although praised by certain groups of educators, many parents see the products of this "reform" as dumbed-down math programs. Although promoted by flowery but empty rhetoric from some education groups, these inadequate programs are responsible for the birth of Mathematically Correct in the first place. One cannot fully appreciate the inferiority of these programs merely on the say-so of irate parents. I urge you to embark on an educational experience of your own. Look at Glencoe Interactive Mathematics: Activities and Investigations [4]. The third book in this program, units 13 to 18, is the 8th grade text for this series. This program was highly rated in California and is now in use in many of our public schools. Yet, the "reform" math covered in this series is grossly inadequate. In fact, these are the worst math books I have ever seen. Unfortunately, the committee that designed the specifications for the national test in mathematics was heavily laden with individuals known to be tied to the very "reform" movement that brought such textbooks into existence. There is far too much wrong with this "reform" to be detailed here. The point, however, is that parents see the "reform" as a serious threat to rigorous math education, yet the design committee was strongly biased in favor of it. With Professor Dossey at the helm, they produced an unbalanced test plan that is headed in the wrong direction - one that would specifically support the movement we find so objectionable. Test Characteristics Make a Real Difference Without thorough study, it is difficult to understand how a mathematics test could be slanted in one way or another to any great extent. Let me summarize just one example that is documented on our web site [5]. Here, traditional and "reform" introductory algebra programs were run simultaneously in the same school. At the end of the year, students were evaluated in two ways - with a regular, objective final exam and with California's Golden State Exam. The Golden State Exam is used for achievement recognition and is modestly slanted to favor the "reform" programs. The results were that 55% of the traditional program students earned A or B grades on the final, while only 11% of the "reform" program students earned A or B grades. Also, 80% of the "reform" program students earned D or F grades, while only 31% of the traditional program students did this poorly. The traditional final exam shows a huge difference between the programs. In contrast to these findings, the proportion of students receiving either honors or high honors on the Golden State Exam was the same for both math programs. Not only that, but more "reform" program students earned recognition on the Golden State Exam than earned a C or better on the regular final exam. Clearly, the nature of the test dramatically alters our conclusions. Recognizing that the tests can be designed in a way that hides dramatic differences in student achievement, we need to look at some of the specific problems with the National Voluntary Test plan. Arithmetic and Algebra There are two important areas in mathematics that are seriously slighted in so-called "reform" math and in the planned exam. These are known as arithmetic and algebra - subjects that parents perceive as vital for student success. Even President Clinton, in his ten-point call to action for American education, states that, "...every 8th grader should know basic math and algebra." [6] With respect to the operations of arithmetic, the design committee intentionally avoids testing basic computational skills under the assumption that all students will know these things. Unfortunately, this is far from universally true among American 8th graders. Indeed the National Assessment of Educational Progress (NAEP) indicates that 21% of 8th graders are not proficient at the level of adding, subtracting, multiplying and dividing whole numbers and solving one-step problems [7]. Failing to directly address these basic skills appears to be nothing less than an attempt to hide weakness even at the lowest levels. The exam ought to reflect the President's prescription that all 8th graders will learn Algebra. This is an important goal for our country. We cannot monitor our progress toward that goal if it is not measured. In spite of calling 25% of the test "Algebra," the content addressed is not the content of an introductory algebra course at all - it doesn't even cover pre-algebra. It appears that the test planners want to fool the country into believing that these American students have learned algebra. By ignoring the demanding content of algebra, the design will fail to measure success at the level of our international competition. It will also fail as a vehicle to promote greater mathematics achievement. We have a link on the Mathematically Correct web site to samples of math problems given to Japanese 12-year-olds [8]. The level of these problems obviously exceeds what is planned for American 8th graders. This is clear evidence that the planned exam is not designed to measure or promote high levels of achievement. Calculators, Guessing and Non-Standard Administration Under the guise of allowing multiple solution strategies, an effort is being made to design test items that promote guessing rather than more powerful and general analytical methods. This works against the goal of promoting accuracy, clear thinking processes, and careful work. This is one of the ways "reform"- friendly tests can be constructed to disguise limited achievement. Similarly, the more successful countries discourage the use of calculators in the early grades, yet the test design would effectively promote their use. Although these "reform" educators believe that calculators promote math learning in young students, over 80% of the public believes that their use should be limited [9, 10]. It is distressing to read the minutes of the design committee where we learn that they want even more calculator use than the public might accept, so they planned to build to even more calculator use over time. Flagrant disregard and disrespect for the opinions of parents and the general public cannot be tolerated. Worse yet, the policy on calculator use, and even the time permitted for the exam, seems to allow differing examination conditions from one locale to another. Thus, the test-administration will be inconsistent or non-standard, greatly reducing the validity of scores for comparison purposes. Unreliable Methods and Credit for Trying The test plan calls for a large number of hand-scored items despite the fact that these items are known to suffer from subjective grading and low reliability. The fraction of total points dedicated to these items is high as they are given more weight than objectively scored items. It is likely that subjectively-scored questions will have a substantial, and artifactual, impact on scores. Unreliable, subjective items are endorsed by the so-called "reform" educators. These items open the door to giving credit for wrong answers, as long as the scorers feel that the student had a good approach to the problem. In fact, this method not only leads to inconsistency in grading, but graders may be influenced by the students' political correctness, attitudes, and degree of social insight. These characteristics have no place in evaluating math achievement. Yet, it is this very sort of encroachment that contributed to the downfall of the "reform"-oriented CLAS test in California. Subjective items also bring a significant increase in the cost to administer the test and delay the return of results to the concerned parties. For these reasons, the number of subjectively-scored items and the fraction of total points they represent should be greatly reduced. Psychometrically Incorrect Non-standard conditions of administration and subjective and unreliable grading techniques are clear psychometric errors. They are inconsistent with established, reliable assessment practices. These weak forms of assessment should not be considered to be valid for making inferences regarding the achievement of individual students. This places the test design in direct conflict with the stated goals of the testing program. Unlike the NAEP, which is intended to provide data on large, aggregate units, the test plan is supposed to provide meaningful data on individuals. But the range of individual achievement is necessarily much broader than for aggregate units such as means for entire states. For such a plan to succeed, it needs to cover an even broader range of achievement than the NAEP - both higher and lower. For a test to be diagnostically useful, it would also benefit from a greater number of more specifically defined sub-scale scores. None of these features are provided. Finally, the notion that the examination can and should be modified over time, as suggested by the design committee, is also a serious mistake. We cannot track progress over the years if the target keeps moving. The intention of the committee to use changes to the examination as a scheme to gradually pull Americans toward their own philosophy is an insult and is inappropriate if we are to measure our progress as a nation. What is Correct for California? After leaping head-first into the "reform" movement and falling to the bottom of the achievement ladder, California is now setting clear and ambitious goals. Many have noted that our newly adopted Math Standards [11] expect a higher level of achievement than those of any other state, and I would have to agree with them. Our Standards and Draft Framework [12] are geared toward completing the primary school math topics by grade 7 with named content areas, like algebra and geometry, starting in grade 8. We are committed and legally bound to developing tests that are designed to measure the content of the our state Standards. Importantly, our algebra standards are real algebra, not just algebra in name only. While we want all of our students to learn introductory algebra in grade 8, it is critical that they learn the real thing. Algebra in name only for 8th grade students is nothing more than the basis of an advertisement or public relations campaign, and is not what we are interested in. Our standards-based exam in algebra will be an algebra test. The planned voluntary test is not. Baring unforseen and drastic changes in the national test plan, the 8th grade math test will be of no value to California. If it is funded and developed as planned, Mathematically Correct will recommend that our own state not waste student time by participating. We would make the same recommendation to any other state that sincerely wants to move their students to a high level of achievement. Alternatives to the Test Plan The management of the national test plan has been moved to the National Assessment Governing Board (NAGB), the semi-independent group that oversees the NAEP. This is a move in the correct direction. It is even possible that the NAGB will institute a wholesale revision to repair the test plan. Whether or not the Board has the fortitude to correct the flawed design is open to question. The same educators that are behind the "reform" movement will pressure the Board as well. Even the NAEP faces these pressures. The two math consultants for the NAEP Mathematics Framework itself are none other than Professor Dossey and Cathy Seeley, both listed as "program conceptualizers" for the text [13] that has come to be known as "Rain Forest" Algebra [14-15]. Given this situation, we are not optimistic about the prospects for a wholesale revision to repair the test design. The NAGB would have to be willing to take drastic steps if there is to be any hope of a balanced approach to a valid, reliable, and useful test plan. There has also been talk of using existing tests as an alternative, noting that many states already conduct their own assessments. It is clear that these off-the-shelf tools are less than optimal. They do not address specific course contents and do not measure up to the achievement goals our country should be striving for. However, the Voluntary National Test, as planned, does not improve upon this situation. In many ways, it actually makes things worse. Mathematically Correct would rather see existing assessment tools or no test at all than the use of the Voluntary National Test as planned. Our Conclusion As representatives of concerned parents throughout the country, the members of Mathematically Correct implore you to take every effort to address these issues. We could benefit from a well-structured national assessment device, but we clearly don't need a tool to further promote the "reform" agenda in math education. Short of a balanced, major overhaul in design, the test can be expected to do more harm than good for the mathematical education of our nation's youth. Respectfully Submitted by Paul Clopton Member and Cofounder, Mathematically Correct Notes: [1] A Program for Raising the Level of Student Achievement in Secondary School Mathematics, Frank B. Allen, http://ourworld.compuserve.com/homepages/mathman/allen.htm [2] Letter to President Clinton regarding the 8th Grade Mathematics Test, Mathematically Correct http://ourworld.compuserve.com/homepages/mathman/letter.htm [3] Comments on the National 8th Grade Mathematics Test, Mathematically Correct http://ourworld.compuserve.com/homepages/mathman/nvtm.htm [4] Interactive Mathematics: Activities and Investigations, David Foster, Sandie Gilliam, Jack Price, et. al., Glencoe Division, Macmillan/McGraw-Hill, Westerville, Ohio, 1995. [5] Effectiveness of CPM vs Traditional Math, Robert W. Haswell http://ourworld.compuserve.com/homepages/mathman/study1.htm [6] President Clinton's Call to Action for American Education in the 21st Century http://www.ed.gov/updates/PresEDPlan/part1.html [7] NAEP 1996 Mathematics Report Card for the Nation and the States http://nces.ed.gov/NAEP/96report/97488.pdf [8] Japanese Math Challenge, Pacific Software Publishing, 1996. http://www.japanese-online.com/math/index.htm [9] What Do Parents and the Public Think About Our Schools?, Public Agenda. http://www.aft.org/page34.htm [10] 1996 State of the Union Address, Albert Shanker, President, American Federation of Teachers. http://www.aft.org/convention/sofunion.htm [11] State Board of Education Ad Hoc Draft California Mathematics Standards http://www.cde.ca.gov/board/ [12] Mathematics Framework for California Public Schools: Kindergarten through Grade Twelve, Draft Sept. 5, 1997, http://www.cde.ca.gov/cilbranch/eltdiv/mathfw.htm [13] Focus on Algebra [Addison-Wesley Secondary Math: An Integrated Approach], R.I. Charles and A. G. Thompson, et. al., Addison-Wesley, Menlo Park, California, 1996. [14] 'Rain Forest' Algebra Course Teaches Everything but Algebra, Marianne Jennings http://ourworld.compuserve.com/homepages/mathman/rainbow.htm [15] More on Addison-Wesley Focus on Algebra, Richard Askey http://ourworld.compuserve.com/homepages/mathman/aw.htm