EDUCATIONAL REFORM AND LIMITS TO STUDENT ACHIEVEMENT
Donald C. Orlich
Science Mathematics Engineering Education Center
Washington State University
Pullman, Washington 99164-4237
“We begin with the hypothesis that any subject can be taught effectively in some intellectually honest form to any child at any state of development” (p. 33) wrote Jerome S. Bruner in his seminal work The Process of Education (1960). Thus, the stage was set for massive curriculum design efforts with the unanticipated consequences being an inappropriate shifting of university and high school topics into lower high, middle and elementary school grades.
But reread Bruner’s classic statement, “We begin with the hypothesis that. . . .” This introductory clause has been either ignored or simply deleted by writers who neglect to realize that Bruner was establishing a testable hypothesis and in 1960 had not been validated. There has been a plethora of writers who inappropriately quote Bruner out of context. And even as late as 1998, the fourth edition of Becoming a Teacher (Parkay and Stanford) perpetuates a logical error by treating Bruner’s assumption as fact (see p. 362).
Bruner (p. 52) created the needed stimulus for curricula to be developed “around the great issues, principles and values that a society deems worthy of the continual concern of its members.” (Who said that John Dewey is ancient history?) Of course, Bruner cautioned that expanding the curriculum would be contingent on his basic assumption being valid. It is to that end that this paper will provide if not empirical data, at least inferential data that the assumption is not tenable.
The educational reforms from the early 1980’s to the present can be traced almost as cause-and-effect to A Nation at Risk: The Imperative for Educational Reform (1983). This report sponsored by the U. S. Department of Education presented a rhetorical call to reform with its infamous assertion that: “If an unfriendly foreign power had attempted to impose on America the mediocre instructional performance that exists today, we might have viewed it as an act of war.”
A Nation at Risk recommended (1) a tougher set of academic basics for high school graduation, (2) higher standards for universities, (3) a longer school year or school day, (4) merit pay for top teachers, and (5) more citizen participation.
C. H. Edson (1983) concluded that A Nation at Risk very closely resembled the famous report issued by the Committee of Ten in 1893. Non-public school personnel dominated both groups. Both reports had recommendations that were intuitively based, rather than based on empirical or evaluative data. Both groups recommended longer school terms. Both reports endorsed a philosophy of social Darwinism—survival of the academic fittest. One difference between the reports is that the Committee of Ten established the concept of academic, general and vocational duration for the high school, whereas A Nation at Risk implied that the high school should be an academically elite institution. David C. Berliner and Bruce J. Biddle (1995) were less kind in their critique of The Nation . . . , viewing it as a cheap political attempt to influence public policy.
Few reformers in the 1980’s or 1990’s heeded James B. Conant’s advice from 1959 that schools in America can be improved—but only “school by school” (p. 96).
The national reform agenda for the 1990’s, however, was set by President George W. Bush in October 1989 with the announcement of America 2000: An Education Strategy, wherein he and the nation’s governors endorsed six major national goals. In 1994, the U.S. Congress would expand the list to eight under the Goals 2000: Educate America Act. These acts are two of the more prominent reform pieces. Bill Chance (1986) reported that there were more than 275 educational task forces organized in the United States, with scores of reports being produced to fix the schools. Chance suggested that most school reform efforts were political and ephemeral.
While school reform might be operationally defined “as anything you can get away with,” the bulk of reforms in the United States of America seem to illustrate eight general traits.
1. The reforms are politically inspired and coerced by state governments (see Kelly 1999).
2. The trend to stress higher student achievement is couched in standards-based reports prepared by professional associations, not local school boards (see Fuhrman 1999; National Research Council 1996).
3. Content standards tend to be collections of outcomes or student behaviors, assembled in a nonsystematic manner without content hierarchies clearly shown (see Masell, Kirst and Hoppe 1997; Ohanian 1999).
4. Cost-benefit analyses are lacking from the reports on state school reforms (House 1996).
5. Control of education has shifted to national and state frames of reference away from local curriculum designs (Schmoker and Marzano 1999).
6. The reform agendas are fragmentary, but broad in scale encompassing most of the 50 states (Hess 1999).
7. Politically inspired as the educational reform movement has evolved, it must be classified as being anti-theoretical, that is, its basic premises are not grounded in empirically sound studies, primarily political enthusiasms and intuitions (see Orlich 1979; Sarason 1998; Shiland 1998).
8. Implied within these traits is the conclusion that as a consequence of standards and high stakes state testing and assessing programs, there should be a dramatic increase in student achievement (see O’Day, Goertz and Floden 1995).
It is now time to challenge the premise that as a consequence of massive funding, written standards and reform resolve that students a priori will achieve more.
To initiate my premise that there is a limit to the quantity and quality of student achievement, two psychological perspectives need to be introduced: (1) the cognitive notion of Development and (2) the behavioral principles of Instructional Design.
The Developmental Perspective. This approach is closely associated with Piaget’s model (1969). The model assumes that humans evolve intellectually in various overlapping stages. Piaget describes four stages or periods of development—the sensorimotor stage, from birth to two years; the preoperational stage, from two to eight years; the concrete operational stage, from eight to eleven years; and the formal stage, from eleven to fifteen years and up.
The last stage is what schools attempt to reach in what we generally call thinking and analyzing. However, the majority of students in middle and high school are still in the concrete developmental stages.
Table 1 provides the relative percentages of students at Piaget’s stages of development.
The Behavioral Perspective. According to this theory, learning can be defined as an observable change in behavior. B. F. Skinner (1938), who coined the term operant conditioning, adapted from the stimulus-response concept long known in psychology, initiated the modern behavioral movement. Applying behaviorism in classrooms requires the use of some form of behavior modification and a reward system to reinforce students for displaying appropriate behaviors. Obviously, there is much more to applying behaviorism.
As one exemplar, the work of Gagne and his associates (1992) will suffice. Through task analysis, learning is carefully sequenced in hierarchical increments so that complex concepts or tasks are subdivided into prerequisite elements. Feedback is an essential aspect of this model, which was the foundation for Science: A Process Approach (1967), an empirically verified K-6 science program based on behavioral principles.
The importance of these two learning perspectives is that they form the basis of my interpretation of Tables 2-5, which present the most current data from the National Assessment of Educational Progress (NAEP) for grades four, eight and eleven for science, mathematics, reading and writing (see Campbell, Voelkl, and Donahue 1998).
Table 1 illustrates the relative percentages of school-aged children and their cognitive levels. Note that until grade 4 (ages 9 or 10) that 100 to 99 percent of children, respectively, are yet in the concrete or intuitive levels of cognition. Examine Tables 2, 3, 4 and 5. Observe how the data in Table 1 predicted that zero percent of the nine-year-olds would be able to answer questions on the NAEP 350 Level! This is evidence that can only be interpreted that over a 20-year period of time, No nine year olds have been able to answer the higher level thinking items on the NAEP tests. One can equate the NAEP 350 Level with “Bloom’s Taxonomy” Levels of synthesis and evaluation or the so-called “higher order” domains (Bloom 1956).
Conversely, observe the gradual decrease in fourth grader percentages correct by moving from Levels 150 to 250. At NAEP Level 150, the percentages range from 91 to 99 percent. No question, these are concrete cognition questions, along with NAEP Level 250. One would predict the downward scores from the Table 1 descriptions of the cognitive levels. It appears that the critical level for fourth graders is NAEP Level 250 or the equivalent of Bloom’s Application Level.
Observe parallel patterns for 13 and 17-year olds. American youth do brilliantly at NAEP Level 150. Teachers and textbooks have long focused on this “knowledge” or recall level. (I call this the “quiz show mentality of American education.”)
As students are tested at the top three NAEP Levels, there are fewer and fewer correct responses. And again these data show the patterns already identified for fourth graders. Figure 1 attempts to display these correlational phenomena in cumulative or conceptual display.
One could argue, as do the naïve reformers, that American kids just don’t work hard enough. There is some validity in that argument, but only up to a point (see Bishop 1992). It will do little good to make nine or ten-year-olds work harder if their cognitive development has not provided them with the needed cerebral connections and the schools with appropriate experiences. Or, if one were in the Vogotsky camp of constructivism (1962), then the conclusion would be that these children have not yet approached their zone of proximal development.
For children aged 9 or 10 years old, there is a cognitive limit, albeit not fixed, but definitely testable using the NAEP achievement levels. Michael Shayer and Philip Adey (1982) made a similar argument stating: “In two studies it was found that no evidence of formal thinking capacity could be found in children under the age of 10, no matter how clever they were” (pp. 135-136). Clever in this regard meant extrapolating exceptionally high intelligence quotients or IQ’s in the 160 range. (Of course IQ as a concept has been long banned in the USA due to political correctness).
But what of the 13 and 17-year olds? Cannot raise the bar reformers expect more from them? The answer from the NAEP data appears to be a tentative Yes. Tentative in that 13-year olds could do better at the NAEP 250 Level, however, there is little room for improvement among the 17-year olds at the 250 Level since they are already approaching the maximum score. There might be developmental limits for 13-and-17-year olds when seeking improvement at the NAEP 300 and 350 levels. (Again, refer to Table 1.)
One might predict improved performances by adapting many of the instructional techniques shown to have positive effect sizes on achievement (see Bloom 1984 Walberg 1991). Thus being more efficient with the use of time and utilizing more inquiry-oriented and active-teaching strategies, student achievement could improve at levels 300 and 350. Yet, cost-benefit analyses (see Hanushek 1997) would be needed to justify the predicted modest gains in student achievement as a consequence of teaching more in-depth and less in breadth, i.e., scope and sequence (see Eylon and Linn 1988).
No reform group in this nation used any empirically based, theoretical model to design its program—save the Tennessee STAR project for small class size (see Moesteller 1995; Nye, Hedges and Konstantopoulos 1999; Ritter and Boruch 1999). Thus, virtually all state-sponsored reform movements have simply become the planet’s most expensive and resource wasting intuitively designed trial and error experiments.
Not to be polemic, but the question to be raised is “What should the achievement levels be on the NAEP to reflect optimal learning and teaching?”
A second question must also follow: “What fiscal commitments are we willing to provide the educational conditions for the potential gains in achievement?”
As Figure 1 conceptually shows, the general lines of achievement on the NAEP test closely follow the predicted lines from data on cognitive development (Table 1). Through use of developmental psychology, we can reasonable predict what standard of achievement children in the schools might reach. I fear that state politicians will simply use the NAEP data to seek bigger budgets to feed the monster of school reform.
The children of this nation deserve better.
Age
|
Grade
|
Intuition |
Entry
Concrete (a) |
Advanced
Concrete (b) |
Entry
Formal (a) |
Middle
Formal (b) |
Ref. |
|
5.5 |
P |
78 |
22 |
|
|
|
J |
|
6 |
K |
68 |
27 |
5 |
|
|
A |
|
7 |
1 |
35 |
55 |
10 |
|
|
A,W |
|
8 |
2 |
25 |
55 |
20 |
|
|
A |
|
|
|
|
|
|
|
|
|
|
9 |
3 |
15 |
55 |
30 |
|
|
A |
|
10 |
4 |
12 |
52 |
35 |
1 |
|
S |
|
11 |
5 |
6 |
49 |
40 |
5 |
|
S |
|
12 |
6-7 |
5 |
32 |
51 |
12 |
|
S |
|
|
|
|
|
|
|
|
|
|
13 |
7-8 |
2 |
34 |
44 |
14 |
6 |
S |
|
14 |
8-9 |
1 |
32 |
43 |
15 |
9 |
S |
|
15 |
9-10 |
1 |
15 |
53 |
18 |
13 |
S |
|
16 |
10-11 |
1 |
13 |
50 |
17 |
19 |
S |
|
|
|
|
|
|
|
|
|
|
16-17 |
11-12 |
3 |
19 |
47 |
19 |
12 |
R |
|
17-18 |
12 |
1 |
15 |
50 |
15 |
19 |
R |
|
Adult |
--- |
20 |
22 |
26 |
17 |
15 |
R |
1. Level (a) in each category is composed of children who have just begun to manifest one or two of that level’s reasoning schemes, while level (b) refers to children manifesting a half dozen or more reasoning schemes.
2. Table derived by Herman T. Epstein, personal communication, June 8, 1999.
J Smedslund, J. (1964). Concrete Reasoning: A Study of Intellectual Development. Lafayette, IN: Child Development Publications of the Society for Research in Child Development.
A Arlin, P. Personal Communication with H. T. Epstein.
W Wei, T. D., et al. (1971). “Piaget’s Concept of Classification: A Comparative Study of Socially Disadvantaged and Middle-Class Young Children.” Child Development (42): 919-927.
R Renner, J. W., Stafford, D. G., Lawson, A. E., McKinnon, J. W., Friot, F. E. and Kellogg, D. H. (1976). Research, Teaching and Learning With the Piaget Model. Norman: University of Oklahoma Press.
S Shayer, M. and Adey, P. (1981). Towards a Science of Science Teaching. London: Heinemann.
TABLE 2. PERCENTAGES OF STUDENTS PERFORMING AT OR ABOVE SCIENCE PERFORMANCE LEVELS, AGES 9, 13 AND 17, 1977 AND 1996.
|
|
|
AGE 9
|
AGE 13
|
AGE 17
|
|||
|
Level
|
|
Percent in
1977 |
Percent in
1996 |
Percent in
1977 |
Percent in
1996 |
Percent in
1977 |
Percent in
1996 |
|
350 |
Can infer relationships and draw conclusions using detailed
scientific knowledge. |
0 |
0 |
1 |
0 |
9 |
11 |
|
300 |
Has some detailed scientific knowledge and can evaluate the
appropriateness of scientific procedures. |
3 |
4 |
11 |
12 |
42 |
48* |
|
250 |
Understands and applies general information from the life and
physical sciences. |
26 |
32* |
49 |
58* |
82 |
84 |
|
200 |
Understands some simple principles and has some knowledge, for
example, about plants and animals. |
68 |
76* |
86 |
92* |
97 |
98 |
|
150 |
Knows everyday science facts |
94 |
97* |
99 |
100* |
100 |
100 |
* Indicates that the percentage in 1996 is significantly different than that in 1977.
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP). Report in Brief, NAEP 1996 Trends in Academic Progress. Revised 1998. NCES 98-530, Table 1, p. 9.
TABLE 3. PERCENTAGES OF STUDENTS PERFORMING AT OR ABOVE MATHEMATICS PERFORMANCE LEVELS, AGES 9, 13 AND 17, 1978 AND 1996.
|
|
|
AGE 9
|
AGE 13
|
AGE 17
|
|||
|
Level
|
|
Percent in
1978 |
Percent in
1996 |
Percent in
1978 |
Percent in
1996 |
Percent in
1978 |
Percent in
1996 |
|
350 |
Can solve multistep problems and use beginning algebra. |
0 |
0 |
1 |
1 |
7 |
7 |
|
300 |
Can compute with decimals, fractions and percents; recognize
geometric figures; solve simple equations; and use moderately complex
reasoning. |
1 |
2* |
18 |
21 |
52 |
60* |
|
250 |
Can add, subtract, multiply and divide using whole numbers and solve
one-step problems. |
22 |
30* |
65 |
79* |
92 |
97* |
|
200 |
Can add and subtract two-digit numbers and recognize relationships
among coins. |
70 |
82* |
95 |
99* |
100 |
100 |
|
150 |
Knows some addition and subtraction facts. |
97 |
99* |
100 |
100 |
100 |
100 |
* Indicates that the percentage in 1996 is significantly different than that in 1978.
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP). Report in Brief, NAEP 1996 Trends in Academic Progress. Revised 1998. NCES 98-530, Table 2, p. 10.
TABLE 4. PERCENTAGES OF STUDENTS PERFORMING AT OR ABOVE READING PERFORMANCE LEVELS, AGES 9, 13 AND 17, 1971 AND 1996.
|
|
|
AGE 9
|
AGE 13
|
AGE 17
|
|||
|
Level
|
|
Percent in
1971 |
Percent in
1996 |
Percent in
1971 |
Percent in
1996 |
Percent in
1971 |
Percent in
1996 |
|
350 |
Can synthesize and learn from specialized reading materials. |
0 |
0 |
0 |
1* |
7 |
6 |
|
300 |
Can find, understand, summarize and explain relatively complicated
information. |
1 |
1 |
10 |
14* |
39 |
39 |
|
250 |
Can search for specific information, interrelate ideas and make
generalizations. |
16 |
18* |
58 |
61* |
79 |
81* |
|
200 |
Can comprehend specific or sequentially related information. |
59 |
64* |
93 |
93 |
96 |
97* |
|
150 |
Can carry out simple, discrete reading tasks. |
91 |
93* |
100 |
100 |
100 |
100 |
|
|
|
|
|
|
|
|
|
* Indicates that the percentage in 1996 is significantly different than that in 1971.
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP). Report in Brief, NAEP 1996 Trends in Academic Progress. Revised 1998. NCES 98-530, Table 3, p. 11.
TABLE 5. PERCENTAGES OF STUDENTS PERFORMING AT OR ABOVE WRITING PERFORMANCE LEVELS, GRADES 4, 8 AND 11, 1984 AND 1996.
|
|
|
GRADE 4
|
GRADE 8
|
GRADE 11
|
|||
|
Level
|
|
Percent in
1984 |
Percent in
1996 |
Percent in
1984 |
Percent in
1996 |
Percent in
1984 |
Percent in
1996 |
|
350 |
Can write effective responses containing details and discussion. |
0 |
0 |
0 |
1 |
2 |
2 |
|
300 |
Can write complete responses containing sufficient information. |
1 |
1 |
13 |
16 |
39 |
31* |
|
250 |
Can begin to write focused and clear responses to tasks. |
10 |
13 |
72 |
66* |
89 |
83* |
|
200 |
Can write partial or vague responses to tasks. |
54 |
59 |
98 |
96* |
100 |
99 |
|
150 |
Can respond to tasks in abbreviated, disjointed or unclear ways. |
93 |
93 |
100 |
100 |
100 |
100 |
* Indicates that the percentage in 1996 is significantly different than that in 1984.
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP). Report in Brief, NAEP 1996 Trends in Academic Progress. Revised 1998. NCES 98-530, Table 4, p. 12.
Knowledge Application Synthesis/ Approximate
Comprehension Analysis Evaluation Piagetian Cognitive
Levels
|
|
100 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Early Concrete |
|
|
||||||||||||||||||||||||||||||||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
75 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Percentage |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
of Items |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Correct |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
on NAEP Tests |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Advanced Concrete |
|
|
50 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Early Formal |
|
|
25 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Advanced Formal |
|
|
0 |
|
|
|
150 |
|
|
200 |
|
|
250 |
|
|
300 |
|
|
350 |
|
|
|
|
|
||||||||||
NAEP Levels
FIGURE 1. COMPOSITE OF NAEP SCORES AND STUDENT COGNITIVE LEVELS.
|
|
4th Grade |
|
|
8th Grade |
|
|
11th Grade |