CUNNINGHAM - NO DATA THAT SHOWS OPEN RESPONSE TESTS ARE BETTER \doc\web\99\01\perftest.txt Date: Fri, 29 Jan 1999 00:03:32 -0500 From: "George K. Cunningham" "We use MC items because they are more inexpensive, more reliable and more valid. We select CR items because some people think they may be better for measuring problem solving. We know the advantages of MC items but the best we can say for CR is that someone likes them." At 10:41 PM 1/27/99 -0600, Donna Garner wrote: >Intelligent people should be saying: "Do performance assessments actually >raise academic achievement? Show me the peer-reviewed, replicated, >empirical data. If you can't produce the evidence, then quit trying to >shove it down our throats." > First of all, it is important to distinguish between performance assessments and open-response tests. As far as I know, there are no states using performance assessments. Kentucky tried it for a few years and finally tossed in the towel. The question is important if you are asking whether there is any evidence for the superiority of open response questions when compared to multiple choice questions. You would think that there was tons of evidence since almost every state is rushing to adopt this item format. The truth is that all of the evidence is to the contrary. It is interesting to look at the few studies that make these comparisons. In a highly technical article that appeared in the Journal of Educational Measurement dealing with scaling the two formats, the reasons for their use is described as follows. "MC items are efficeint in terms of testing time and the information they provide. Therefore, they can be included in a test to enhance the reliability of the test as well as the content validity of the test by providing the opportunity to assess a wide range of contern. CR items, on the other hand, are often thought to provide more appropriate formats for certain skills, such as problem solving." Examine the evidence in this argument. We use MC items because they are more inexpensive, more reliable and more valid. We select CR items because some people think they may be better for measuring problem solving. We know the advantages of MC items but the best we can say for CR is that someone likes them. By the way there are many other sources that treat the comparison in this way. Here is what Lori Shepherd, one of the nation's leading experts on testing and an advocate of performance assessment (open-response tests) says about the evidence: "However, to date little research has been done to evalute the actual effects of performance assessments on instructional practices or on student learning." She also states: "These anticipated benefits of performance assessments have been inferred by analogy from research documenting negative effects of traditional, standardized testing." There are problems with MC tests and therefore CR tests must be good. Pretty flimsy evidence for CR tests. Sort of like establishing the effectiveness of screwdrivers for pounding in nails by showing that pliers don't do that job well. ===================================================================== EDUCATION CONSUMERS CLEARINGHOUSE networking and information for parents and taxpayers on the internet Subscriptions & Archives: http://education-consumers.com or You are currently subscribed to education-consumers as: arthurhu@halcyon.com TO UNSUBSCRIBE: Send a blank email to leave-education-consumers-989462S@lists.dundee.net ===================================================================== For less mail, click on the following link and choose 1) a daily digest, 2) a daily list of subjects, or 3) no mail (read postings on Web) http://lists.dundee.net/scripts/lyris.pl?enter=education-consumers For more help & info: http://www.lyris.com/help or . The WA legislature really needs to hear George Cunningham of Kentucky testify about the truth of "performance based" or open response testing. The fact is that these tests have _never_ been openly debated anywhere. If they were, they would be demolished in an instant, but instead, they have been pushed by the one-party education system that dominates every state in the union. The consensus Delphi system insures that all opposing community "input" gets tossed into the recycle bin, and the desired predestined outcome is insured passage. If you compare the arguments presented for "performance" testing with established cognitive testing works like the Bell Curve, these new tests obviously reverse every feature of proven tests (multiple choice->open response, basic skills->problem solving, recall-> figure out content that is not taught, timed->untimed, culturally neutral->multicultural) on the unfounded assumption that IQ and SAT style tests are wrong. Even the assumption that the new tests are unbiased is wrong, when the outcomes flunk 95% of minorities compared to 80% of the majority. The entire problem with the critique of IQ and SAT test is not that the alternative is better, or any proof that people do have identical measurable intelligence, it's based solely on denying the evidence, and 100% of it favors traditional IQ style tests. So it's the old political trick that if you don't have any evidence to support your side, you just demolish all of the evidence on the other side, and what you're left with is your more politically popular contention that all intelligence is equal among all people, or that alternative testing is more appropriate, accurate and reliable than standardized testing. If people were honest enough to accept the scientic fact of the variability of human cognitive ability, they would run away from, not flock to the promise that "all will suceed" at the highest levels. People would demand proof that indeed, ALL children can be taught to master such tests when in fact no group of children, not even small test groups have demonstrated what should be an obvious impossibility. Conservatives say that the trick to promising that all will meet the same standard will be to lower the standard. The truth is that the standard is raised to such a high level that no one will ever meet the standard. But by the time peope figure this out and have thrown the rascals out, the state will have already doubled their education budget for 10 years, by which time the rascals will have figured out another scam, and blame the failure on "not enough reform". These guys are the new American Bolsheviks, and we need to call them that. How dare they call for "root and branch" and total "restructuring", and then call us, the people who want careful gradual change, "extremists"???? A small cadre of elite mis-thinkers crusading in the name of the people but who are really grabbing power away from the people.