CUNNINGHAM ON NORM-BASED TESTING \doc\web\98\06\norm.txt Date sent: Tue, 15 Sep 1998 13:41:19 -0400 To: "ClearingHouse" From: "George K. Cunningham" Subject: [education-consumers] Norm-referenced testing Send reply to: "ClearingHouse" ===================================================================== First of all, I would like to congratulate John on the new/old format. I look forward to a renewal of the interesting dialogues that used to take place on this media. There have been several comments on the testing programs in different states that hinged on the distinction between norm- and criterion-referenced assessment. I would like to clear up some misunderstandings and also present my case for the value of norm-referenced tests. What is being called criterion-referenced testing on these pages is really standards-based assessment. There is very little criterion-referenced testing being used anywhere now days. During the late sixties and early seventies its use, together with mastery learning was an early form of educational reform. The basic principles of criterion referenced testing are antithetical to most modern educational reform because it is the opposite of holistic learning. These three approaches: standards-based, criterion- and norm referenced all refer to the way that scores are interpreted, rather than the form of the test itself. You could use any of the three methods of interpretation with any test. When scores are interpreted with norm-referenced methods they are compared to typical student performance. They permit us to interpret scores as being above or below average on scales that tell us how much the score differ from the average. Criterion-referenced assessment like the use of standards based approaches requires the setting of absolute standards. In the case of criterion-referenced testing, the items used on a test must beassumed to have come from a well defined domain. For example, the domain of single digit multiplication facts is all 100 such items. A test over this standard would sample from this domain. Defining domains in more abstract topics is almost impossible. What is the domain of all multiple-choice items that assess knowledge of the civil war. The impossibility of defining domains plus criterion-referenced dependency on behaviorism and reductionism rendered this approach inoperable. One reason why some people are confused about criterion-referenced testing is that William Spady appropriated some of its terminology in his originalarticles describing what he called outcomes-based education. This was a perversion of criterion-referenced testing and could not have been more different from true criterion-referenced testing. Educational reformers hate norm-referenced testing because it is based on the assumption that there is variability among student performance. They choose to believe that all students can perform at the same high level. When norm-referenced assessment is used there will always be a range of student performance. Because they reject the idea of differences in student aptitude, educational reformers despise norm-referenced assessment. Their only other choice is standards-based assessment, which means they define ahead of time how well students should perform. This approach permits the evaluation of schools in terms of the percentage ofstudents who are proficient (or some similar term like basic). This is how NAEP scores are reported. The use of thee techniques are useful when the goal is to manipulate the results for public relations purposes, but it makes for lousy measurement. If you are interested in making comparisons among schools or students and rendering decisions about which are performing best can be achieved using norm-referenced assessment. This is a powerful and time tested technique. It has fallen out of favor because it forces decisions about quality andit is viewed as the old way of doing things. The problem is that no one has come up with a better way of assessing student performance. It is simply not possible to define appropriate levels of student performance ahead of time without reference to typical student performance. Attempts to do this have caused many of the assessment problems we are seeing. In particular, it leads to the kind of problems faced by Kentucky and Washington where tests have been put in place which place totally unrealistic demands on students and which have incredibly high failure rates. ===================================================================== EDUCATION CONSUMERS CLEARINGHOUSE networking and information for parents and taxpayers on the internet Subscriptions & Archives: http://education-consumers.com or You are currently subscribed to education-consumers as: arthurhu@nwnexus.com TO UNSUBSCRIBE: Send a blank email to leave-education-consumers-989528N@lists.dundee.net ===================================================================== For less mail, click on the following link and choose 1) a daily digest, 2) a daily list of subjects, or 3) no mail (read postings on Web) http://lists.dundee.net/scripts/lyris.pl?enter=education-consumers&text_mo de=0 For more help & info: http://www.lyris.com/help or