+OK 12397 octets Received: from smtp00.nwnexus.com (smtp00.nwnexus.com [192.135.191.25]) by mail3.halcyon.com (8.8.8/8.8.8) with ESMTP id KAA16374 for ; Fri, 8 Oct 1999 10:38:19 -0700 (PDT) Received: from mv.egroups.com (mv.egroups.com [207.138.41.150]) by smtp00.nwnexus.com (8.8.8/8.8.8) with SMTP id KAA21142 for ; Fri, 8 Oct 1999 10:38:15 -0700 (PDT) Received: from [10.1.2.36] by mv.egroups.com with NNFMP; 08 Oct 1999 18:38:10 -0000 Mailing-List: contact wa-ed-deform-owner@egroups.com X-Mailing-List: wa-ed-deform@egroups.com X-URL: http://www.egroups.com/list/wa-ed-deform/ Received: (listserv 1.262); by qk; 08 Oct 1999 17:38:09 -0000 Reply-To: wa-ed-deform@egroups.com Delivered-To: listsaver-egroups-wa-ed-deform@egroups.com Received: (qmail 23614 invoked from network); 8 Oct 1999 17:37:56 -0000 Received: from mail4.halcyon.com (206.63.63.62) by qh.egroups.com with SMTP; 8 Oct 1999 17:37:56 -0000 Received: from bolero.rahul.net (blv-pm104-ip15.halcyon.com [206.63.32.135]) by mail4.halcyon.com (8.8.8/8.8.8) with SMTP id KAA21378 for ; Fri, 8 Oct 1999 10:37:48 -0700 (PDT) Message-Id: <199910081737.KAA21378@mail4.halcyon.com> MIME-Version: 1.0 To: From: Date: Fri, 08 Oct 1999 10:36:19 -0600 X-Mailer: Net-Tamer 1.11.2P To: 71524.2205@compuserve.com From: rdyarrow@elnet.com (by way of Fred Battey ) Subject: [wa-ed-deform] new-unwittingly OCT/ISBE highlights pitfalls of performance Mime-Version: 1.0 Content-Type: text/plain Content-Transfer-Encoding: 7bit Status: When I lobbied in Springfield to eliminate the performance assessment questions on the new ISAT (they were ultimately reduced in number but not eliminated), I used performance assessment questions developed by ISBE back in 1994 to demonstrate how subjective, unrigorous, potentially biased, and downright silly performance assessment questions can be. Many of the state senators were amazed that the ISBE examples of performance assessment activities were seriously considered rigorous activities to measure learning state standards. When these senators thought of rigorous learning standards as applied by 11th graders, they hadn't been thinking of Mickey Mouse activities like these. I assumed that after the senators' reaction to these ISBE generated materials, that ISBE would be embarrassed and quickly shelve those social studies performance assessments that someone was paid royally to develop. Wrong!! You can imagine my surprise today, when during a routine check of the ISBE website I saw that ISBE has just recently (in October) put those SAME silly performance assessment activities on their website. These samples are provided to teachers as examples of rigorous assessment activities to determine whether students in 11th grade classrooms are meeting the state standards. Some folks never learn. You can visit this site and see for yourself what the entire flap about performance assessment is all about (and a bigger flap about subjectivity and increased bias). The website for the performance assessment examples is: http://www.isbe.state.il.us/isbesites/aass/frames.htm Be sure to click on the buttons "Student samples" and "Assessments" at the bottom of the page before clicking on the specific numbers on the right side bar. The assessment questions won't make sense until you have visited the "rubrics" section to see how the three scores given to each student product (whether it be a poster, an essay, or an oral report). The rubric details what the scores that are assigned to each student's product "mean." Each student receives three scores for his or her product . The first score is for knowledge (knowledge of evidence from the social sciences: facts/supporting details; themes/issues: and concepts/ideas) , the second score is for reasoning (analysis, evaluation, and synthesis of evidence), and the third score is for communication (demonstrates knowledge and reasoning through oral, written, visual, dramatic, or mixed media presentation). A student earns a score between 1 and 6 for each of the three areas. You have to see the actual rubric and how they determine how to assign a "5" vs a "4" or "6" to believe it for yourself. After encountering her first rubric last year in eighth grade, even my daughter immediately recognized the subjective nature of awarding points based on such unclear criteria. Let me give you an example, although I urge you to visit the site and see for yourself. Of course if you are a parent of a child in public school, you see these kinds of rubrics routinely. ex. For the "reasoning" score, a student will earn a "5" if he or she has identified and logically organized most of the relevant evidence, used appropriate and critical thinking skills and habits of mind to analyze, evaluate, and synthesize evidence, and reached informed conclusions based on the evidence. That student will earn a "4" in reasoning if he or she has identified and organized some of the relevant evidence, used partial critical thinking skills and habits of mind to analyze, evaluate, and synthesize evidence, and reached informed conclusion based on the evidence. That student will earn a "3" in reasoning if he or she has identified some of the relevant evidence but omits other evidence, used partial critical thinking skills and habits of mind to analyze, evaluate, and synthesize evidence, and reached informed conclusions based on the evidence. and so on...... You try and think how objectively you could apply this type of criteria, assigning three scores to each paper, video product, poster, or oral presentation. I call it "pretending to be objective," When you wonder how this year's ISAT scores became so convoluted and questionable, remember that we were not able to eliminate all performance assessment questions (typically graded by raters in the basement of some mall, located near to some college campus). It's just one more factor added into a list of ingredients that make the validity and reliability of this year's ISAT suspect. Do performance assessment activities have a place in the classroom? Yes, after new material has been rigorously taught, well-designed performance assessment activities can provide the student with opportunities to apply and write about what he or she has learned in a new context. Unfortunately, in many of today's classrooms where the philosophy is that students teach themselves and each other in a more meaningful fashion than if a teacher were to directly teach the material, performance assessment, otherwise known as authentic assessment consumes a disproportionate amount of time. Watching my daughter transfer from the public school she attended last year to the Catholic High School she now attends, I notice that she has gone from an environment where ongoing performance assessment activities seemed to consume more than 50% of the classroom time......to an environment where they are used only occasionally (except in science where the end product of labwork could be considered performance assessment.) Because she is no longer engaged in making posters or writing songs about her work, her homework load has tripled. Reading books and reviewing class notes in order to retain the content takes considerably more time than slapping together another poster or Power Point presentation. I would hope that a teacher would put together more rigorous performance assessment activities than these for 11th graders since all of them could be completed by students ranging from 4th to 8th grade. If you look at the student samples and the scores assigned, you'll see very mediocre work. Indeed, you'll see some appalling work products earn a fair number of points. When you look at the sample questions, be sure to note that because of the variety of presentations that are encouraged (in order to accommodate multiple intelligences), clear accurate writing is never stressed as a priority. Also be sure to look at the following examples: #1. This cartoon provides so much information, that a student could create a product without knowing anything about voting rights. If the student can read the cartoon pictures, enough information is provided to concoct a credible bull..... answer. I would find the caricature of a southern sheriff offensive if I were from the south. The student samples are unimpressive. What do we expect from an Illinois student? #7. Once again the cartoon provides so much information about the three branches of government and their functions, that one could remember nothing and craft together an answer. We expect 8th graders to know more than this question expects. This question exemplifies how advocates of performance assessment (authentic assessment) make it sound so rigorous, when often the process is far easier than a well constructed multiple choice exam. #9. I can't help but wonder how teacher bias would effect the scoring here. What if the teacher is a "liberal" (and I can think of widely divergent descriptions of that concept) and feels that the student's description of a "conservative" is inaccurate and more typical of a "reactionary?" #27. I find this marketing of a soft drink question appalling. My children did a similar activity in a science "experience" at Fermi Lab when they were in fourth grade. You can read the rest....some of the assessment activities are better than others, but I find none indicative of a product reflecting high standards. Reminds me of the time I had a part time job as an administrative consultant at a fairly prestigious high school in the south suburbs and was assigned to write objectives for the biology and chemistry courses. Even though I told the administrator that science was my weakest subject area and I felt no more qualified to write objectives for their courses then the man in the moon, I was required to complete that task. I couldn't believe that he was satisfied with my end product, because I had selected the vaguest, most simplistic kind of objectives that I could take from the course books. In no way, could I write questions that would require the students to demonstrate a high level of proficiency. When the person writing the objectives, or the test questions doesn't have that knowledge, it will not come out in the final assessment product. I can't help but wonder whether that high school science dept. still uses those objectives and sincerely hope that the teachers had the sense to jettison them. Since ISBE has just chosen to display these social studies questions to guide teachers of 11th graders in Illinois, we know that they have not yet jettisoned this fiasco. For some perspective, compare the 11th grade performance assessment questions at the ISBE website to these following simple essay questions at the end of an 8th grade American History Book: 1. (no picture cues) For each branch of governement, give an example of how its power checks or balances each of the other two branches of the governement 2. Compare and contrast political tactics used to get the Constitution ratified with political tactics used today. 3. Explain how seven factors of the economy affected the development of the West. (this question presupposes students know that the seven factors are: natural resources, labor equipment and tools, transportation, money and credit, demand for product, and profit) 4. In the early 1800's differences between the North and South in their positions on tariffs and slavery did not lead to war. Why not? After reading these questions for middle schoolers, ask yourself whether you think the more subjective questions for Illinois 11th graders are rigorous. Wouldn't it be interesting to have teachers of 11th graders in countries around the world visit this web site, look at the student products and give their reactions!!!! It is time for a reality check. Mary ------------------------------------------------------------------------ eGroups.com home: http://www.egroups.com/group/wa-ed-deform http://www.egroups.com - Simplifying group communications .