CUNNINGHAM - NO DATA THAT SHOWS OPEN RESPONSE TESTS ARE BETTER
Date: Fri, 29 Jan 1999 00:03:32 -0500
From: "George K. Cunningham"
"We use MC items because they are more inexpensive, more reliable and
more valid. We select CR items because some people think they may be
better for measuring problem solving. We know the advantages of MC
items but the best we can say for CR is that someone likes them."
At 10:41 PM 1/27/99 -0600, Donna Garner wrote:
>Intelligent people should be saying: "Do performance assessments actually
>raise academic achievement? Show me the peer-reviewed, replicated,
>empirical data. If you can't produce the evidence, then quit trying to
>shove it down our throats."
First of all, it is important to distinguish between performance
assessments and open-response tests. As far as I know, there are no states
using performance assessments. Kentucky tried it for a few years and
finally tossed in the towel.
The question is important if you are asking whether there is any evidence
for the superiority of open response questions when compared to multiple
choice questions. You would think that there was tons of evidence since
almost every state is rushing to adopt this item format. The truth is that
all of the evidence is to the contrary. It is interesting to look at the
few studies that make these comparisons.
In a highly technical article that appeared in the Journal of Educational
Measurement dealing with scaling the two formats, the reasons for their use
is described as follows.
"MC items are efficeint in terms of testing time and the information they
provide. Therefore, they can be included in a test to enhance the
reliability of the test as well as the content validity of the test by
providing the opportunity to assess a wide range of contern. CR items, on
the other hand, are often thought to provide more appropriate formats for
certain skills, such as problem solving."
Examine the evidence in this argument. We use MC items because they are
more inexpensive, more reliable and more valid. We select CR items because
some people think they may be better for measuring problem solving. We
know the advantages of MC items but the best we can say for CR is that
someone likes them. By the way there are many other sources that treat the
comparison in this way.
Here is what Lori Shepherd, one of the nation's leading experts on testing
and an advocate of performance assessment (open-response tests) says about
"However, to date little research has been done to evalute the actual
effects of performance assessments on instructional practices or on student
She also states:
"These anticipated benefits of performance assessments have been inferred
by analogy from research documenting negative effects of traditional,
There are problems with MC tests and therefore CR tests must be good.
Pretty flimsy evidence for CR tests. Sort of like establishing the
effectiveness of screwdrivers for pounding in nails by showing that pliers
don't do that job well.
EDUCATION CONSUMERS CLEARINGHOUSE
networking and information for parents and taxpayers on the internet
Subscriptions & Archives: http://education-consumers.com or
You are currently subscribed to education-consumers as: email@example.com
TO UNSUBSCRIBE: Send a blank email to leave-education-consumers-989462S@lists.dundee.net
For less mail, click on the following link and choose
1) a daily digest,
2) a daily list of subjects, or
3) no mail (read postings on Web)
For more help & info: http://www.lyris.com/help or
The WA legislature really needs to hear George Cunningham of
Kentucky testify about the truth of "performance based" or open
The fact is that these tests have _never_ been
openly debated anywhere. If they were, they would be demolished
in an instant, but instead, they have been pushed by the one-party
education system that dominates every state in the union. The
consensus Delphi system insures that all opposing community
"input" gets tossed into the recycle bin, and the desired predestined
outcome is insured passage.
If you compare the arguments presented for "performance" testing with
established cognitive testing works like the Bell Curve, these new
tests obviously reverse every feature of proven tests (multiple
choice->open response, basic skills->problem solving, recall-> figure
out content that is not taught, timed->untimed, culturally
neutral->multicultural) on the unfounded assumption that IQ and SAT
style tests are wrong. Even the assumption that the new tests are
unbiased is wrong, when the outcomes flunk 95% of minorities compared
to 80% of the majority.
The entire problem with the critique of IQ and SAT test is not
that the alternative is better, or any proof that people do have
identical measurable intelligence, it's based solely on denying
the evidence, and 100% of it favors traditional IQ style tests.
So it's the old political trick that if you don't have any evidence
to support your side, you just demolish all of the evidence
on the other side, and what you're left with is your more
politically popular contention that all intelligence is equal among
all people, or that alternative testing is more appropriate,
accurate and reliable than standardized testing.
If people were honest enough to accept the scientic fact of the
variability of human cognitive ability, they would run away from, not
flock to the promise that "all will suceed" at the highest levels.
People would demand proof that indeed, ALL children can be
taught to master such tests when in fact no group of children, not
even small test groups have demonstrated what should be
an obvious impossibility.
Conservatives say that the trick to promising that all will meet the
same standard will be to lower the standard. The truth is that the
standard is raised to such a high level that no one will ever meet
But by the time peope figure this out and have thrown
the rascals out, the state will have already doubled their education
budget for 10 years, by which time the rascals will have figured out
another scam, and blame the failure on "not enough reform".
These guys are the new American Bolsheviks, and we need to call them
that. How dare they call for "root and branch" and total
"restructuring", and then call us, the people who want careful
gradual change, "extremists"???? A small cadre of elite mis-thinkers
crusading in the name of the people but who are really grabbing power
away from the people.