EDUCATIONAL REFORM AND LIMITS TO STUDENT ACHIEVEMENT

 

Donald C. Orlich

Science Mathematics Engineering Education Center

Washington State University

Pullman, Washington 99164-4237

 

 

“We begin with the hypothesis that any subject can be taught effectively in some intellectually honest form to any child at any state of development” (p. 33) wrote Jerome S. Bruner in his seminal work The Process of Education (1960).  Thus, the stage was set for massive curriculum design efforts with the unanticipated consequences being an inappropriate shifting of university and high school topics into lower high, middle and elementary school grades.

But reread Bruner’s classic statement,  “We begin with the hypothesis that. . . .”  This introductory clause has been either ignored or simply deleted by writers who neglect to realize that Bruner was establishing a testable hypothesis and in 1960 had not been validated.  There has been a plethora of writers who inappropriately quote Bruner out of context.  And even as late as 1998, the fourth edition of Becoming a Teacher (Parkay and Stanford) perpetuates a logical error by treating Bruner’s assumption as fact (see p. 362).

Bruner (p. 52) created the needed stimulus for curricula to be developed “around the great issues, principles and values that a society deems worthy of the continual con­cern of its members.”  (Who said that John Dewey is ancient history?)  Of course, Bruner cautioned that expanding the curriculum would be contingent on his basic assumption being valid.  It is to that end that this paper will provide if not empirical data, at least inferential data that the assumption is not tenable.


The Background of Educational Reform

 

The educational reforms from the early 1980’s to the present can be traced almost as cause-and-effect to A Nation at Risk: The Imperative for Educational Reform (1983).  This report sponsored by the U. S. Department of Education presented a rhetorical call to reform with its infamous assertion that: “If an unfriendly foreign power had attempted to impose on America the mediocre instructional performance that exists today, we might have viewed it as an act of war.”

A Nation at Risk recommended (1) a tougher set of academic basics for high school graduation, (2) higher standards for universities, (3) a longer school year or school day, (4) merit pay for top teachers, and (5) more citizen participation.

C. H. Edson (1983) concluded that A Nation at Risk very closely resembled the famous report issued by the Committee of Ten in 1893.  Non-public school personnel dominated both groups.  Both reports had recommendations that were intuitively based, rather than based on empirical or evaluative data.  Both groups recommended longer school terms.  Both reports endorsed a philosophy of social Darwinism—survival of the academic fittest.  One difference between the reports is that the Committee of Ten estab­lished the concept of academic, general and vocational duration for the high school, whereas A Nation at Risk implied that the high school should be an academically elite institution.  David C. Berliner and Bruce J. Biddle (1995) were less kind in their critique of The Nation . . . , viewing it as a cheap political attempt to influence public policy.

Few reformers in the 1980’s or 1990’s heeded James B. Conant’s advice from 1959 that schools in America can be improved—but only “school by school” (p. 96).

The national reform agenda for the 1990’s, however, was set by President George W. Bush in October 1989 with the announcement of America 2000: An Education Strat­egy, wherein he and the nation’s governors endorsed six major national goals.  In 1994, the U.S. Congress would expand the list to eight under the Goals 2000: Educate America Act.  These acts are two of the more prominent reform pieces.  Bill Chance (1986) reported that there were more than 275 educational task forces organized in the United States, with scores of reports being produced to fix the schools.  Chance suggested that most school reform efforts were political and ephemeral.

Traits of the Current Reform Efforts

 

While school reform might be operationally defined “as anything you can get away with,” the bulk of reforms in the United States of America seem to illustrate eight general traits.

1.      The reforms are politically inspired and coerced by state governments (see Kelly 1999).

2.      The trend to stress higher student achievement is couched in standards-based reports prepared by professional associations, not local school boards (see Fuhrman 1999; National Research Council 1996).

3.      Content standards tend to be collections of outcomes or student behaviors, assembled in a nonsystematic manner without content hierarchies clearly shown (see Masell, Kirst and  Hoppe 1997; Ohanian 1999).

4.      Cost-benefit analyses are lacking from the reports on state school reforms (House 1996).

5.      Control of education has shifted to national and state frames of reference away from local curriculum designs (Schmoker and Marzano 1999).

6.      The reform agendas are fragmentary, but broad in scale encompassing most of the 50 states (Hess 1999).

7.      Politically inspired as the educational reform movement has evolved, it must be clas­sified as being anti-theoretical, that is, its basic premises are not grounded in empiri­cally sound studies, primarily political enthusiasms and intuitions (see Orlich 1979; Sarason 1998; Shiland 1998).

8.      Implied within these traits is the conclusion that as a consequence of standards and high stakes state testing and assessing programs, there should be a dramatic increase in student achievement (see O’Day, Goertz and Floden 1995).

It is now time to challenge the premise that as a consequence of massive funding, written standards and reform resolve that students a priori will achieve more.

Establishing the Limits to Student Achievement

 

To initiate my premise that there is a limit to the quantity and quality of student achievement, two psychological perspectives need to be introduced: (1) the cognitive notion of Development and (2) the behavioral principles of Instructional Design.

The Developmental Perspective.  This approach is closely associated with Piaget’s model (1969).  The model assumes that humans evolve intellectually in various overlapping stages.  Piaget describes four stages or periods of development—the sensorimotor stage, from birth to two years; the preoperational stage, from two to eight years; the concrete operational stage, from eight to eleven years; and the formal stage, from eleven to fifteen years and up.

The last stage is what schools attempt to reach in what we generally call thinking and analyzing.  However, the majority of students in middle and high school are still in the concrete developmental stages.

Table 1 provides the relative percentages of students at Piaget’s stages of devel­opment.

 

Insert Table 1 About Here

 

 

The Behavioral Perspective.  According to this theory, learning can be defined as an observable change in behavior.  B. F. Skinner (1938), who coined the term operant conditioning, adapted from the stimulus-response concept long known in psychology, initiated the modern behavioral movement.  Applying behaviorism in classrooms requires the use of some form of behavior modification and a reward system to reinforce students for displaying appropriate behaviors.  Obviously, there is much more to applying behaviorism.

As one exemplar, the work of Gagne and his associates (1992) will suffice.  Through task analysis, learning is carefully sequenced in hierarchical increments so that complex concepts or tasks are subdivided into prerequisite elements.  Feedback is an essential aspect of this model, which was the foundation for Science: A Process Approach (1967), an empirically verified K-6 science program based on behavioral prin­ciples.

The importance of these two learning perspectives is that they form the basis of my interpretation of Tables 2-5, which present the most current data from the National Assessment of Educational Progress (NAEP) for grades four, eight and eleven for science, mathematics, reading and writing (see Campbell, Voelkl, and Donahue 1998).

 

Insert Tables 2, 3, 4 and 5 About Here

 

 

Examining the NAEP Data Through Developmental Screens

 

Table 1 illustrates the relative percentages of school-aged children and their cog­nitive levels.  Note that until grade 4 (ages 9 or 10) that 100 to 99 percent of children, respectively, are yet in the concrete or intuitive levels of cognition.  Examine Tables 2, 3, 4 and 5.  Observe how the data in Table 1 predicted that zero percent of the nine-year-olds would be able to answer questions on the NAEP 350 Level!  This is evidence that can only be interpreted that over a 20-year period of time, No nine year olds have been able to answer the higher level thinking items on the NAEP tests.  One can equate the NAEP 350 Level with “Bloom’s Taxonomy” Levels of synthesis and evaluation or the so-called “higher order” domains (Bloom 1956).

Conversely, observe the gradual decrease in fourth grader percentages correct by moving from Levels 150 to 250.  At NAEP Level 150, the percentages range from 91 to 99 percent.  No question, these are concrete cognition questions, along with NAEP Level 250.  One would predict the downward scores from the Table 1 descriptions of the cogni­tive levels.  It appears that the critical level for fourth graders is NAEP Level 250 or the equivalent of Bloom’s Application Level.

Observe parallel patterns for 13 and 17-year olds.  American youth do brilliantly at NAEP Level 150.  Teachers and textbooks have long focused on this “knowledge” or recall level.  (I call this the “quiz show mentality of American education.”)

As students are tested at the top three NAEP Levels, there are fewer and fewer correct responses.  And again these data show the patterns already identified for fourth graders.  Figure 1 attempts to display these correlational phenomena in cumulative or conceptual display.

 

Insert Figure 1 About Here

 

 

Discussion and Conclusion

 

One could argue, as do the naïve reformers, that American kids just don’t work hard enough.  There is some validity in that argument, but only up to a point (see Bishop 1992).  It will do little good to make nine or ten-year-olds work harder if their cognitive development has not provided them with the needed cerebral connections and the schools with appropriate experiences.  Or, if one were in the Vogotsky camp of constructivism (1962), then the conclusion would be that these children have not yet approached their zone of proximal development.

For children aged 9 or 10 years old, there is a cognitive limit, albeit not fixed, but definitely testable using the NAEP achievement levels.  Michael Shayer and Philip Adey (1982) made a similar argument stating: “In two studies it was found that no evidence of formal thinking capacity could be found in children under the age of 10, no matter how clever they were” (pp. 135-136).  Clever in this regard meant extrapolating exceptionally high intelligence quotients or IQ’s in the 160 range.  (Of course IQ as a concept has been long banned in the USA due to political correctness).

But what of the 13 and 17-year olds?  Cannot raise the bar reformers expect more from them?  The answer from the NAEP data appears to be a tentative Yes.  Tentative in that 13-year olds could do better at the NAEP 250 Level, however, there is little room for improvement among the 17-year olds at the 250 Level since they are already approaching the maximum score.  There might be developmental limits for 13-and-17-year olds when seeking improvement at the NAEP 300 and 350 levels.  (Again, refer to Table 1.)

One might predict improved performances by adapting many of the instructional techniques shown to have positive effect sizes on achievement (see Bloom 1984 Walberg 1991).  Thus being more efficient with the use of time and utilizing more inquiry-oriented and active-teaching strategies, student achievement could improve at levels 300 and 350.  Yet, cost-benefit analyses (see Hanushek 1997) would be needed to justify the predicted modest gains in student achievement as a consequence of teaching more in-depth and less in breadth, i.e., scope and sequence (see Eylon and Linn 1988).

No reform group in this nation used any empirically based, theoretical model to design its program—save the Tennessee STAR project for small class size (see Moesteller 1995; Nye, Hedges and Konstantopoulos 1999; Ritter and Boruch 1999).  Thus, virtually all state-sponsored reform movements have simply become the planet’s most expensive and resource wasting intuitively designed trial and error experiments.

Not to be polemic, but the question to be raised is “What should the achievement levels be on the NAEP to reflect optimal learning and teaching?”

A second question must also follow: “What fiscal commitments are we willing to provide the educational conditions for the potential gains in achievement?”

As Figure 1 conceptually shows, the general lines of achievement on the NAEP test closely follow the predicted lines from data on cognitive development (Table 1).  Through use of developmental psychology, we can reasonable predict what standard of achievement children in the schools might  reach.  I fear that state politicians will simply use the NAEP data to seek bigger budgets to feed the monster of school reform.

The children of this nation deserve better.


TABLE 1.  PERCENTAGE OF STUDENTS AT PIAGET’S COGNITIVE LEVELS

 

 

Age

 

Grade

 

Intuition

Entry Concrete (a)

Advanced Concrete (b)

Entry Formal (a)

Middle Formal (b)

 

Ref.

5.5

P

78

22

 

 

 

J

6

K

68

27

5

 

 

A

7

1

35

55

10

 

 

A,W

8

2

25

55

20

 

 

A

 

 

 

 

 

 

 

 

9

3

15

55

30

 

 

A

10

4

12

52

35

1

 

S

11

5

6

49

40

5

 

S

12

6-7

5

32

51

12

 

S

 

 

 

 

 

 

 

 

13

7-8

2

34

44

14

6

S

14

8-9

1

32

43

15

9

S

15

9-10

1

15

53

18

13

S

16

10-11

1

13

50

17

19

S

 

 

 

 

 

 

 

 

16-17

11-12

3

19

47

19

12

R

17-18

12

1

15

50

15

19

R

Adult

---

20

22

26

17

15

R

 

Table Notes and References

 

1.      Level (a) in each category is composed of children who have just begun to manifest one or two of that level’s reasoning schemes, while level (b) refers to children mani­festing a half dozen or more reasoning schemes.

2.      Table derived by Herman T. Epstein, personal communication, June 8, 1999.

J            Smedslund, J. (1964).  Concrete Reasoning: A Study of Intellectual Development.  Lafayette, IN: Child Development Publications of the Society for Research in Child Development.

A         Arlin, P.  Personal Communication with H. T. Epstein.

W        Wei, T. D., et al. (1971).  “Piaget’s Concept of Classification: A Comparative Study of Socially Disadvantaged and Middle-Class Young Children.”  Child Development (42): 919-927.

R          Renner, J. W., Stafford, D. G., Lawson, A. E., McKinnon, J. W., Friot, F. E. and Kellogg, D. H.  (1976).  Research, Teaching and Learning With the Piaget Model.  Norman: University of Oklahoma Press.

S          Shayer, M. and Adey, P.  (1981).  Towards a Science of Science Teaching.  London: Heinemann.

 

TABLE 2.  PERCENTAGES OF STUDENTS PERFORMING AT OR ABOVE SCIENCE PERFORMANCE LEVELS, AGES 9, 13 AND 17, 1977 AND 1996.

 

 

 

AGE 9

AGE 13

AGE 17

 

Level

 

Percent in 1977

Percent in 1996

Percent in 1977

Percent in 1996

Percent in 1977

Percent in 1996

350

Can infer relationships and draw conclusions using de­tailed scientific knowledge.

 

0

 

0

 

1

 

0

 

9

 

11

300

Has some detailed scientific knowledge and can evaluate the appropriateness of scien­tific procedures.

 

3

 

4

 

11

 

12

 

42

 

48*

250

Understands and applies gen­eral information from the life and physical sciences.

 

26

 

32*

 

49

 

58*

 

82

 

84

200

Understands some simple prin­ciples and has some knowl­edge, for example, about plants and animals.

 

68

 

76*

 

86

 

92*

 

97

 

98

150

Knows everyday science facts

94

97*

99

100*

100

100

 

*    Indicates that the percentage in 1996 is significantly different than that in 1977.

 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP).  Report in Brief, NAEP 1996 Trends in Academic Prog­ress.  Revised 1998.  NCES 98-530, Table 1, p. 9.

 

 


TABLE 3.  PERCENTAGES OF STUDENTS PERFORMING AT OR ABOVE MATHEMATICS PERFORMANCE LEVELS, AGES 9, 13 AND 17, 1978 AND 1996.

 

 

 

AGE 9

AGE 13

AGE 17

 

Level

 

Percent in 1978

Percent in 1996

Percent in 1978

Percent in 1996

Percent in 1978

Percent in 1996

350

Can solve multistep problems and use beginning algebra.

0

0

1

1

7

7

300

Can compute with decimals, fractions and percents; recog­nize geometric figures; solve simple equations; and use moderately complex reasoning.

 

1

 

2*

 

18

 

21

 

52

 

60*

250

Can add, subtract, multiply and divide using whole numbers and solve one-step problems.

 

22

 

30*

 

65

 

79*

 

92

 

97*

200

Can add and subtract two-digit numbers and recognize rela­tionships among coins.

 

70

 

82*

 

95

 

99*

 

100

 

100

150

Knows some addition and sub­traction facts.

97

99*

100

100

100

100

 

*    Indicates that the percentage in 1996 is significantly different than that in 1978.

 

SOURCE:  National Center for Education Statistics, National Assessment of Educational Progress (NAEP).  Report in Brief, NAEP 1996 Trends in Academic Prog­ress.  Revised 1998.  NCES 98-530, Table 2, p. 10.

 


TABLE 4.  PERCENTAGES OF STUDENTS PERFORMING AT OR ABOVE READING PERFORMANCE LEVELS, AGES 9, 13 AND 17, 1971 AND 1996.

 

 

 

AGE 9

AGE 13

AGE 17

 

Level

 

Percent in 1971

Percent in 1996

Percent in 1971

Percent in 1996

Percent in 1971

Percent in 1996

350

Can synthesize and learn from specialized reading materials.

0

0

0

1*

7

6

300

Can find, understand, summa­rize and explain relatively complicated information.

 

1

 

1

 

10

 

14*

 

39

 

39

250

Can search for specific infor­mation, interrelate ideas and make generalizations.

 

16

 

18*

 

58

 

61*

 

79

 

81*

200

Can comprehend specific or sequentially related informa­tion.

 

59

 

64*

 

93

 

93

 

96

 

97*

150

Can carry out simple, discrete reading tasks.

91

93*

100

100

100

100

 

 

 

 

 

 

 

 

 

*    Indicates that the percentage in 1996 is significantly different than that in 1971.

 

SOURCE:  National Center for Education Statistics, National Assessment of Educational Progress (NAEP).  Report in Brief, NAEP 1996 Trends in Academic Prog­ress.  Revised 1998.  NCES 98-530, Table 3, p. 11.

 


TABLE 5.  PERCENTAGES OF STUDENTS PERFORMING AT OR ABOVE WRITING PERFORMANCE LEVELS, GRADES 4, 8 AND 11, 1984 AND 1996.

 

 

 

GRADE 4

GRADE 8

GRADE 11

 

Level

 

Percent in 1984

Percent in 1996

Percent in 1984

Percent in 1996

Percent in 1984

Percent in 1996

350

Can write effective responses containing details and discus­sion.

 

0

 

0

 

0

 

1

 

2

 

2

300

Can write complete responses containing sufficient informa­tion.

 

1

 

1

 

13

 

16

 

39

 

31*

250

Can begin to write focused and clear responses to tasks.

 

10

 

13

 

72

 

66*

 

89

 

83*

200

Can write partial or vague responses to tasks.

 

54

 

59

 

98

 

96*

 

   100

 

    99

150

Can respond to tasks in abbre­viated, disjointed or unclear ways.

 

     93

 

     93

 

   100

 

  100

 

   100

 

  100

 

*    Indicates that the percentage in 1996 is significantly different than that in 1984.

 

SOURCE:  National Center for Education Statistics, National Assessment of Educational Progress (NAEP).  Report in Brief, NAEP 1996 Trends in Academic Prog­ress.  Revised 1998.  NCES 98-530, Table 4, p. 12.

 

 

 

 


BLOOM’S TAXONOMY

 

       Knowledge                               Application                                 Synthesis/    Approximate

                        Comprehension                                    Analysis         Evaluation           Piagetian Cognitive

                                                                                                                                      Levels

 

100

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Early Concrete

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

75

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Percentage

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

of Items

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Correct

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

on NAEP Tests

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Advanced Concrete

 

50

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Early Formal

 

25

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Advanced Formal

 

0

 

 

 

150

 

 

200

 

 

250

 

 

300

 

 

350

 

 

 

 

 

NAEP Levels

 

FIGURE 1.  COMPOSITE OF NAEP SCORES AND STUDENT COGNITIVE LEVELS.

 

 

4th Grade

 

8th Grade

 

11th Grade