Date sent: Thu, 11 Dec 1997 23:13:47 -0900 From: family To: Alaska Ed LOOP Subject: Effectiveness of CPM vs Traditional Math Effectiveness of CPM vs Traditional Math

Effectiveness of CPM vs Traditional Math

What follows is a report produced by a Robert W. Haswell, a high school teacher, concerning the outcome of CPM compared to Traditional math in a high school in California. This is not a scientific study, but the teachers at this school made every effort to make their investigation a fair one.

The teacher has requested that references to the high school and the school district be deleted from the report.


PREFACE

This report was prepared by Robert W. Haswell, a high school math teacher for the past 27 years, during my own time and at my own expense. I wrote this analysis on behalf of many of my colleagues who are being asked to pursue a teaching approach which is unproven at best, and which may prove much less effective for many at most. I also wrote this report on behalf of a very large number of students who find the CPM approach to the teaching of mathematics ineffective.

My greatest fear is that the CPM approach to the teaching of mathematics, largely untested but widely used at this time, will produce a generation of students who feel good about their math experiences and who are are not afraid to "take on" a math problem, but, who, at the same time, are functionally illiterate about the structure of mathematics, are mathematical morons without a calculator in their hands, are unable to achieve a high level of success individually, and who are unable to actually determine the correct answer to a problem. These are many of the same fears expressed recently by Delaine Eastin, California's superintendent of public instruction.

XXXXXXX High School no longer offers a traditional approach to the teaching of first-year Algebra, and the Geometry Honors course is the single geometry course in which a traditional approach is employed. It is expected that all second-year Algebra courses will be solely CPM soon. I am currently experiencing two first-year Algebra classes for the first time.

It is my hope that this report will highlight a few of the differences in the effectiveness of each program.

INTRODUCTION

During the 1994 - 1995 school year, XXXXXXXXX High School in XXXXXXX was the single school in the district which continued to offer its first-year Algebra students a traditional algebra program, as well as one which followed the College Preparatory Mathematics (CPM) approach. This dual offering was the consequence of a philosophical difference of opinion between the more experienced teachers, who believed that the traditional approach to teaching had served their students well over many years, and the first-year and less experienced teachers who had, admittedly, not been successful in their attempts at teaching in the traditional manner.

It is the belief of the younger teachers, a belief held by many in the country, state, and district, that mathematics education is in need of radical overhaul in order to make the study of mathematics "user- friendly," and to make all levels of mathematics accessible to all students regardless of ability or previous mastery of prerequisites. To accomplish this, proponents believe that the teaching of mathematics should be student-directed as opposed to subject-matter-directed, and the content of courses should be adjusted, discarded, or completely revamped according to the comfort level of the students. It is their hope - belief - dream, that this new approach to teaching will achieve many goals: helping the student to understand better the mathematical concepts behind the operations, being able to communicate their procedures and solutions more clearly to others, and being more successful in solving problems with which they are unfamiliar. Proponents of these new approaches feel that the aformentioned objectives cannot be accomplished if we continue the 'traditional' approach with its prerequisites and consistent building upon fundamental principles to reach the higher levels of mathematics. The word 'traditional' in the teaching of math has a negative connotation to these advocates of change.

The traditional approach is certainly a teacher-directed approach which involves lectures, but also involves extensive teacher-student interaction, student boardwork, cooperative learning groups, and the traditional teachers constant "taking the temperature" of the students' comprehension and adjusting the pace and content of classwork accordingly. To blanket all teachers who use the traditional approach with the epithet of "boring lecturers" is terribly narrow and inaccurate.

In an effort to clarify the differences in the effectiveness of each approach, a comparative study of the second semester final exam results was made for the 1993 - 1994 school year. This follow-up report will concern itself with the results obtained on the final exam given to all first-year Algebra students at XXXXXXXXXX High School in June of the 1994 - 1995 school year.

THE FINAL EXAM

The intial task of the department was to formulate a final exam which would be comprehensive, but which would not skew the problems unfavorably toward the traditional or the CPM approach. It was agreed that two teachers of the CPM persuasion, and one teacher of the traditional persuasion, would spend a day during the middle of May revising and reconfiguring the final exam that had been given at the end of the previous year.

It was during this work session that the following decisions were agreed upon. Initially, the original multiple-choice test was reduced from a seventy-item test to a sixty-item test. This was done to eliminate the test items which tested material taught in a traditional course, a complete first-year Algebra course, but not taught in a CPM course, a partial first-year Algebra course. A full course in CPM algebra is completed only if the student completes CPM geometry.

It was then agreed upon that the three "story problem" type problems, which had been administered along with the multiple-choice questions during the two-hour final exam period the previous year, would be given to the students during a normal class period prior to the final exam. It was felt the time restrictions of a two-hour session prevented the students from doing their best on these problems. At this time, a rubric for grading each of the three problems was also agreed upon, with the understanding that every involved teacher in the department would meet in the math office following the administration of the tests for grading. In fact, this method of grading worked wonderfully. With all of the papers piled in the middle of the table, each teacher took the top paper in order and graded it. If any questions regarding the rubric surfaced, the score for the paper was determined by the group as a whole.

At the behest of the traditional teachers, it was then agreed upon that fifteen free-response questions would be added to the final exam from the previous year. These questions were to be administered after the multiple-choice questions had been completed. No partial credit for answers was given for the purposes of this analysis, although several CPM teachers gave their students partial credit on a personal basis.

THE STUDY POPULATION

A pre requisite to the data analysis in any study is the determination of the study population. In this case, the only effective approach required that the first semester of first-year Algebra final rosters be correlated student-by-student with the second semester first- year Algebra final rosters.

During this process, all students who completed the first semester under one approach, but completed the second semester under the alternative approach, were eliminated from the study. Any student who did not complete the second semester, for whatever reason, remained part of the study in the character of a casualty of the respective program. It was also the decision of the author to eliminate all seniors (3) from the study for fear that their backgrounds in math education include too many unknowns. Also eliminated from the study population were students who entered either program during the second semester ftom other schools in or out of the district.

For the students taught solely under the traditional approach, it was revealed that 56 out of 94 first semester students completed the second semester of first-year Algebra. This number represents 59.6% of the original traditional enrollment.

For students taught solely under the CPM approach, it was revealed that 113 out of 151 first semester students completed the second semester of first-year Algebra. This number represents 74.8% of the original CPM enrollment.

These figures represent critical information because of the potential distortions in the analytical data they could cause were they to remain unknown. It is clear that an adjustment must be made to the study population to guarantee that the traditional student population and the CPM student population are as comparable as possible. Since 40.4% of the traditionally-taught students did not complete the second semester, and 25.2% of the CPM students did not complete the second semester, itis easy to see that 15.2% more of the traditionally-taught students did not complete the second semester. and thus were not included in the final exam results.

Most of the traditional students who did not complete the second semester failed to do so because they failed the first semester. Indeed, those involved with the CPM program constantly point out the fact more of their students complete first-year Algebra. Many question whether the difference is due to the difference in programs, or the difference in expectations and standards. Whatever the case, it is necessary to balance the study population from each approach.

It was decided that the most equitable way to make the study populations as uniform as possible was to eliminate 15.20% of the original CPM population, or 23 students. from their final exam results. In reality, the elimination of these 23 students resulted in the elimination of ALL of those students who received D grades at the end of the first semester from all of the teachers of CPM, plus several C- students. Therefore, the final study population included traditional students who received grades of A through D, and CPM students who received grades of A through C.

With this adjustment, we can reasonably examine Schedule A, a diagram of the students' scores on the multiple-choice portion of the final exam. The divisions separating the grades were based upon an agreed upon curve which made 51 out of 60 items a 100% paper. Schedule A shows that:

SECOND SEMESTER ALGEBRA FINAL EXAM SCORES


GRADES   TRADITIONAL   CPM


  A         29.1%     4.5%
  B         25.5%     6.7%
  C         14.5%     9.0%
  D         18.2%    16.9%
  F         12.7%    62.9%


Median      42        27

These figures show that 54.6% of the traditional students earned A or B grades on the multiple-choice portion of the final exam, while 11.2% of the CPM students earned A or B grades. This is almost FIVE TIMES as many A's and B's earned by the traditional students as the CPM students. It can also be seen that 79.8% of the CPM students earned D or F grades, while 30.9% of the traditional students earned D or F grades.

The mean (average) score for the traditional students was 40.5, while the mean score for the CPM students was 28.1. Certainly, the difference between the two scores is significant.

FRESHMEN

Traditionally, first-year Algebra has been considered a college-prep course for ninth graders. It is not unreasonable, therefore, to examine the performance of the freshmen on the multiple-choice portion of the final exam. It is understood that the freshmen students entering XXXXXXX come from many different feeder schools with varying approaches to teaching eighth-grade mathematics, but we are certain that they have not received traditional instruction within a secondary school setting.

For this analysis, we examine Schedule B:

FRESHMEN


SECOND SEMESTER ALGEBRA FINAL EXAM SCORES


GRADES  TRADITIONAL        CPM
  A        34.2%           6.1%
  B        22.0%          10.2%
  C        19.5%          12.2%
  D        19.5%          20.4%
  F         4.9%          51.0%
Median      43             30

These figures show that 56.2% of the traditional freshmen students earned grades of A or B on the final exam, while 16.3% of the CPM students earned grades of A or B. The traditional students earned more than three times as many high grades. Similarly, 71.4% of the CPM students earned grades of D or F, while a comparatively small 24.4% of the traditional students earned D or F grades. The difference in median scores, 43 (traditional) verses 30 (CPM), and mean (average) scores, 42 (traditional) and 31 (CPM), is certainly extremely significant.

If, as the figures in Schedule A and Schedule B indicate, the students enrolled in the traditional program have a significantly higher level of success on the first-year Algebra final exam (multiple-choice), is it because all of the lower ability students have dropped out at the end of the first semester? Not likely. Remember, the 23 weakest CPM students at the end of the first semester are not included in the figures for Schedules A and B.

ANALYTICAL PROBLEMS

Supporters of the CPM approach to teaching place a strong emphasis on students' ability to successfully analyze, organize, and solve "story problems." The CPM first-year Algebra program also places a great deal of importance on the ability of students to successfully answer graphing problems. Three of these types of problems were administered to the algebra students during a normal period the week prior to the final exam. This was done to ensure that all of the students had ample time to do as well as they were able.

In order to maintain uniformity in the grading of the three problems, and because the questions were being graded within the CPM format, a rubric for each of the questions was agreed upon by the traditional and CPM representatives, prior to the administration of the questions. The papers were then graded simultaneously and randomly by six members of the math department. The directions for each problem were the same:

In order to receive full credit for this problem, you must show
all your work clearly enough that other students could
understand it. Your answer must be clearly identified. This may
mean using equations, organized charts, pictures, andlor
written explanations. When your answers are not integers, round
to two decimal places.

Problem number 1 was a graphing problem which required the students to draw a line through the points (-2,4) and (6,14). They then had to answer three questions:

(a) Determine where the line crosses the x-axis and where
it crosses the y-axis.
(b) If the point (2,a) lies on this line, then what is the
value of a? A correct, or incorrect, answer will receive
no credit unless you show work, or explain how you got your
answer.
(c) Write an equation of a second line that would never
intersect the line formed by the two points above.

The rubric for this problem allowed for scores to range from 0 to 4, with a 4 being equivalent to an A grade. An analysis of the results indicate the following:

ALGEBRA 2 FINAL EXAM SCORES 1995


STORY PROBLEM #1


TRADITIONAL vs CPM


        # TRADITIONAL         # CPM
SCORE      STUDENTS          STUDENTS


  4           24                15
  3           12                 9
  2           13                 8
  1            7                47
  0            1                10
  
Average      2.89              1.66

With a scale which ranges only from 0 to 4, the difference in median scores, 3 for traditional and 1 for CPM, and the difference in mean scores, 2.89 for traditional and 1.66 for CPM, may not seem significant. However, it should be noted that the traditional mean score is 74.1% higher than the CPM mean score. and that is significant.

The major cause for the difference in the scores seemed to focus on the fact that the x- and y-intercepts were not integral solutions. The CPM students were only able to estimate the solutions to be (-5,0) and (0,6) from an examination of their graph, but most were unable to formulate a linear equation in y = mx + b form for the given line. Thus they were unable to determine the correct answers of (-26/5,0) and (0,13/2). A significantly large number of the traditional students were able to determine the proper linear equation for the line through the points given, and this equation helped the traditional students to answer more successfully parts (b) and (c) of question 1.

Problem #3 was a standard story problem:

Five tapes cost $3.00 more than two CDs at the TAPE AND
CD EXCHANGE. Carla bought four tapes and three CDs for
$88.50. What was the cost of each tape?

The rubric for this problem allowed for scores to range from 0 to 5, with 5 being equivalent to an A grade. An examination of the test papers showed the following:

ALGEBRA 2 FINAL EXAM SCORES 1995


STORY PROBLEM #3


TRADITIONAL VS CPM


        # TRADITIONAL         # CPM
SCORE      STUDENTS          STUDENTS


  5           10                 7
  4            5                 7
  3           12                 7
  2           12                22
  1            8                32
  0            9                13
  
Average      2.47              1.82

The results show that the traditional students completed this problem more successfully than the CPM students. The traditionally taught students' median score was 3 compared to the CPM students' median of 1, and the traditional mean score was 2.47 compared to 1.82 for the CPM students. Thus the traditional mean score was 35.7% higher than the CPM mean score.

This third problem is recognized within a traditional framework as being a problem which requires the formulation and solution of two simultaneous equations. The students who had been taught traditionally were able to do this with a reasonable measure of success. Students within the CPM program, however, are introduced to the solution of equations through the use of guess-and-check tables.

It is the belief of supporters of the CPM approach that once the students are able to solve equations using the guess and check method, they will be able to redefine this skill to the point that the CPM students will be able to formulate and solve linear equations. CPM supporters also believe, however, that if a student never learns to formulate the proper equation(s), but can successfully employ the guess-and-check method throughout the year's work, then their work is totally acceptable, and worthy of full credit.

The differences in the approaches outlined above delineate the different success levels between the two approaches. The traditionally taught students tried to formulate two linear equations and to solve them simultaneously. They were able to do so with a reasonable level of success. Although a number of the CPM students attempted to formulate linear equations, as you would expect them to be able to do at the end of a year of algebra, most of the CPM students unsuccessfully tried to use the guess-and-check method. The guess-and-check method failed them because the correct answer, $8.09, was not a "neat" solution.

Problem #2 was a graphing problem:

Find as accurately as you can the point or points of
intersection of the two equations given below. Make
sure that you state both the x- and y- coordinates of
the point(s).


        y = x2 - 5          y = x + 1

The rubric for this problem was from 5 to 0, with 5 being equivalent to an A grade. An examination of the item scores indicates the following:

ALGEBRA 2 FINAL EXAM SCORES 1995


STORY PROBLEM #2


TRADITIONAL VS CPM


        # TRADITIONAL         # CPM
SCORE      STUDENTS          STUDENTS


  5           15                40
  4            2                 6
  3            4                11
  2           15                20
  1            3                 6
  0           18                 4
  
Average      2.25              3.50

On this problem, the CPM students performed better than the traditional students. The CPM students had a higher median score, 4 for CPM versus 2 for traditional, and they also had a higher mean score, 3.5 for CPM versus 2.25 for traditional. This CPM mean score is 55.6% higher than the traditional score.

It was mentioned previously that the CPM program strongly emphasizes graphing, and this was the manner used by almost all CPM students to solve this problem. The higher level of success for the CPM students relative to problem #1 is probably due to the fact that this problem has integral solutions which can be taken directly from the graph. Most traditional students, on the other hand, attempted to solve the problem through the use of simultaneous equations, with a limited measure of success.

FREE RESPONSE

At the request of the traditionalists, the CPM teachers agreed to administer 15 free-response questions to the first-year Algebra students. While the use of multiple-choice questions is a standard means to test many important algebraic concepts in a short period of time without overburdening the teacher with the scoring of the tests, a student faced with free-response questions necessarily must rely totally on his individual knowledge and understanding of many mathematical processes. Although one or two individual CPM teachers, who were disappointed with their students' performance, gave partial credit on this portion of the final exam for purposes of grading their own students, it was agreed that the results shown below do not reflect the use of partial credit. Answers were scored right or wrong.

It can be seen that, although this portion of the test consisted of 15 items, the scores should be based on a high score of 10. With this assumption accepted, the divisions indicated on Schedule C were used to differentiate varying levels of success on this portion of the test. An examination of Schedule C shows us that:

SECOND SEMESTER ALGEBRA FINAL EXAM SCORES


FREE RESPONSE


GRADES  TRADITIONAL       CPM
  A        14.3%          6.7%
  B        14.3%          9.0%
  C        32.1%         19.1%
  D        23.2%         34.8%
  F        16.1%         30.3%

The advocates of neither approach can be pleased with these results, but it is also very clear that the success of those taught traditionally continues to significantly outshine that of the CPM students. While 28.6% of the traditional students earned grades of A or B, only 15.7 % of the CPM students did so. Even though a disappointing 39.3% of the traditional students earned grades of D or F, 65.1% of the CPM students earned grades of D or F. It is possible that fatigue was a factor in all of the students earning scores lower than everyone would have liked, but the fatigue factor was certainly the same for all students.

The mean scores of 4.3 for the traditional students, and 3 for the CPM students appear to be very close because of the relatively few number of items on this portion of the test. The traditional students' mean is, however, 43.3% higher than that of the CPM students'.


Mathematically Correct feels that these findings, though preliminary, give sufficient cause to seriously question the efficacy of CPM.

Mathematically Correct argues that ...
Such radical program changes should not be introduced without clear, well documented, overwhelmingly compelling, quantitative data to support them.


Golden State Exam Results

Since many advocates of CPM have touted the success of their program in preparing students on the Golden State Exam, we thought that it was important to normalize Golden State results to more traditional tests of Algebra knowledge. This is particularly important as the Golden State Exam was modified in 1993 in a way that was likely to make it more CPM friendly. As a step toward placing Golden State Exam results in a context of more traditional exams, we have obtained the Golden State Exam scores from this same school and group of students. With the above question in mind, it is extremely interesting to note that a higher fraction of CPM students were awarded honors on the Golden State Exam that were able to earn a C or better grade on the more traditional test of Algebra knowledge. Clearly, for this group of students, the Golden State Exam seems to test some aspect of Algebra that is favored by the CPM approach and that is different from what even the CPM teachers themselves thought was important in a test.

Grade           Traditional           CPM


High Honors          3%                6%
Honors              13%               10%
School Recognition  13%                7%
                    ---               ---
TOTAL               29%               23%