Dale-Chall Readability Formula (from Readability Revisited: The New Dale-Chall Readability Formula, Jeanne S. Chall and Edgar Dale, Brookline Books, 1995 ISBN 1-57129-008-7). This formula has been widely used to determine reading levels of literature and textbooks and most recently it was used to evaluate the difficulty of the reading test in Tex The methodology used for a book over 200 pages in length is to take 100 word >>>samples at 50 page intervals and apply the formula to these samples. I began >>>on page 7 of the book and took samples on pages 57, 107, 157, and 200. >>>Using the formula, one counts the number of complete sentences (fewer >>>sentences in the 100 word sample indicates more difficult writing) and then >>>counts the number of unfamiliar words. Unfamiliar words are any which do not >>>appear in the list of 3000 words known by students in Grade 4 (the list is >>>included in the book). +OK 8977 octets Received: from smtp03.nwnexus.com (smtp03.nwnexus.com [206.63.63.41]) by mail1.halcyon.com (8.8.8/8.8.8) with ESMTP id BAA09675 for ; Fri, 5 Feb 1999 01:29:42 -0800 (PST) Received: from ripple.dundee.net (ripple.dundee.net [206.249.104.12]) by smtp03.nwnexus.com (8.8.8/8.8.8) with SMTP id BAA00841 for ; Fri, 5 Feb 1999 01:29:39 -0800 (PST) From: rdyarrow@elnet.com Message-Id: Date: Thu, 04 Feb 1999 23:07:39 -0600 To: "ClearingHouse" Subject: [education-consumers] NEW ILLINOIS ISAT STATE TEST Mime-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Reply-To: rdyarrow@elnet.com Precedence: bulk Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by mail1.halcyon.com id BAA09675 Status: ===================================================================== Since Stan sent the ISAT reading selections to LEXILE and received the readability scores, I decided to follow Elaine's example and use the Dale-Chall Readability Formula that she used to analyze the "Who Put that Hair in My Toothbrush" book that the teacher was reading to my daughter's eight grade class. I applied the formula to the same sample passages we sent to LEXILE. I have included the reading levels that LEXILE gave us as well as the reading levels I obtained from the Dale-Chall Readability FOrmula. For each selection, I randomly chose two samples based on "shut my eyes and start at the place where my finger lands." The results show the very problems I was concerned about. 1. 3rd grade reading ISAT sample "Unlocking Secrets of Dolphin Communication" LEXILE: We didn't submit because it appeared to be challenging and we could only submit three samples free of charge. DALE-CHALL: Received between an eighth through ninth grade reading level. The third grade selection was actually more difficult than the fifth and eighth grade sample passages. My hunch is that because this selection had more photographs, it was perceived to be appropriate for younger readers. The passage contained more technical information as well as more unfamiliar words. Sentences are approx. same length as in 5th and 8th grade samples. 2. 5th grade reading ISAT sample "Wilma Rudolph: Against the ODDS" LEXILE: Middle of fifth through beginning of sixth DALE-CHALL: The average of the two samples came to approx. a sixth grade reading level 3. 8th grade reading ISAT sample "The Man Who Listens To Horses" LEXILE: End of fifth through middle of sixth DALE-CHALL The average of the two samples came to approx a fifth grade reading level. 4. 10th grade reading ISAT sample "Supermarkets Build Sales By Beguiling Shopper's Kids" LEXILE: Range end of eleventh through beginning of College Junior - Senior DALE-CHALL The average of the two samples came to approx. a tenth through eleventh grade reading level. The passage was not that difficult, but because it contained the names of a number of different companies, the unfamiliar word count shot up...raising the grade level. Since in the past five or six years, I am seeing middle school book fairs with mostly 3rd and 4th grade books, eighth graders reading books I used to see in 4th grade, high schoolers reading abridged, dumbed down versions of the "real literature," and school learning centers where the librarians have removed all of the grade level designations on books, I have become very interested in who determines what the "typical" child reads at a specific grade level. With most of our suburban schools having thrown out their school curriculum guides and book choice left to individual teachers, the end results are without question "dumbed down." I was even more concerned when I started digging up the old curriculum guides from 1920 and 1890. Can a society gradually lessen its expectations of children and no one even notices as the years slide by???? Last year when I met with Eunice Greer, who was responsible for developing the new ISAT, one of my first questions was how the reading selections for the IGAP would be chosen. Given that her response was that reading selections on the new ISAT would be selected by taking reading material from books and magazines that children of the targeted grade level routinely read, I couldn't help but wonder how closely the selections would match standard calibrated readability levels of at least ten years ago. My concerns increased when she voiced her philosophical stance against applying a readability formula and when it became clear that the new ISAT test would not undergo the rigorous testing period that the previous IGAP had undergone (3 years with correlations to the Iowa Achievement Test.) How can one compare the reading of third graders from year to year when each year they are reading material of different levels of difficulty? What could prevent this from happening if the new test was written under the pressure of this one and then never properly tested? When we found the ISAT reading sample tests on the web, this proved to be a good opportunity to check those reading selections. Of course, nothing guarantees that the reading selections on the actually ISAT given this year match the samples. When literature is picked the way these reading selections were, anything could happen. The Dale-Chall book which has been recently revised describes how the concept of "readability" is unpopular with many educators. Recently I received an email from a colleague of mine with some caveats about readability scores. I will end by including her posting because it emphasizes the importance of high-stakes tests going through the proper testing themselves. Because the ISAT was written and given in such a short time period, we do not have the retest reliability and construct validity that allows us to put our faith in this test as a valid measure. We just don't have that level of demonstrated accountability with the new ISAT. I'll put a few comments IN CAPS as to how these readability issues as I understand them impact the sample ISAT reading passages. Shirley's reminders on readability: Science passages at a 5th grade level will have a higher readability level because of long words. But I would not want these words to be missing from a science passage. BECAUSE THE THIRD GRADE PASSAGE IS A SCIENCE PASSAGE, ITS READABILITY IS HIGHER WITH THE MORE TECHNICAL TERMS. I FIND IT STRANGE THAT A THIRD GRADE TEST READING SELECTION WOULD BE SO MUCH MORE DIFFICULT THAN THE FIFTH OR EIGHTH GRADE SELECTIONS. EVEN THOUGH THE PASSAGES FOR THE EIGHTH GRADE SELECTION DOESN'T HAVE PHOTOS SPRINKLED THROUGHOUT LIKE THE THIRD AND FIFTH GRADE ONE, THE TEXT ITSELF IS CLEARLY MORE DIFFICULT IN THE THIRD GRADE ONE (EXPECIALLY THE CAPTION ON ECHOLOCATION.) Other issues than readability play into this. Text structure is important. Is the text arranged in a way that students can overtly see its purpose (explanatory, compare/contrast, persuasive)? NONE OF THESE PASSAGES HAVE ANY SUBTLETY. ALL OF THE PASSAGES ARE STRAIGHTFORWARD. A second issue is vocabulary. Maybe the words are short, so they get a low readability score, but the words are obscure and studnets have never heard of them. ONLY OBSCURE WORD WAS "DUN" IN THE EIGHTH GRADE PASSAGE) AND ITS MEANING WAS IMMEDIATELY CLEAR. What about syntax? GOOD QUESTION I CAN'T ANSWER. You can see how convoluted this gets., There is no easy answer. Test writing institutes or companies should test and test and test their passages to be sure that similar passages elicit the same scores that they have retest reliability,construct validity, etc. But you know all of that. Be careful to not put too much faith in readability formulae. Shirley ===================================================================== EDUCATION CONSUMERS CLEARINGHOUSE networking and information for parents and taxpayers on the internet Subscriptions & Archives: http://education-consumers.com or You are currently subscribed to education-consumers as: arthurhu@halcyon.com TO UNSUBSCRIBE: Send a blank email to leave-education-consumers-989462S@lists.dundee.net ===================================================================== For less mail, click on the following link and choose 1) a daily digest, 2) a daily list of subjects, or 3) no mail (read postings on Web) http://lists.dundee.net/scripts/lyris.pl?enter=education-consumers For more help & info: http://www.lyris.com/help or . +OK 7329 octets Received: from smtp05.nwnexus.com (smtp05.nwnexus.com [206.63.63.43]) by mail1.halcyon.com (8.8.8/8.8.8) with ESMTP id DAA21134 for ; Fri, 5 Feb 1999 03:54:09 -0800 (PST) Received: from ripple.dundee.net (ripple.dundee.net [206.249.104.12]) by smtp05.nwnexus.com (8.8.8/8.8.8) with SMTP id DAA18824 for ; Fri, 5 Feb 1999 03:54:07 -0800 Message-Id: Date: Fri, 05 Feb 1999 06:42:43 -0500 To: "ClearingHouse" From: "J. E. Stone" Subject: [education-consumers] Comment: Reading levels research and NEW ILLINOIS ISAT STATE TEST In-Reply-To: Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Reply-To: "J. E. Stone" Precedence: bulk Status: ===================================================================== Mary wrote: >Since in the past five or six years, I am seeing middle school book fairs with >mostly 3rd and 4th grade books, eighth graders reading books I used to see in >4th grade, high schoolers reading abridged, dumbed down versions of the "real >literature," and school learning centers where the librarians have removed all >of the grade level designations on books, I have become very interested in who >determines what the "typical" child reads at a specific grade level. With most >of our suburban schools having thrown out their school curriculum guides and >book choice left to individual teachers, the end results are without question >"dumbed down." I was even more concerned when I started digging up the old >curriculum guides from 1920 and 1890. Can a society gradually lessen its >expectations of children and no one even notices as the years slide by???? Here is a post from the ClearingHouse archive that addresses Mary's concern: Date: Sat, 12 Oct 1996 13:55:16 -0400 From: "J. E. Stone" Subject: Very significant study of dumbed down schoolbooks Sender: owner-education-consumers@tricon.net FYI: Subscribers to the Education Consumers Clearinghouse and other interested parties Steve Kossor , a school psychologist in PA (and Clearinghouse subscriber), brought to my attention a study claiming to have found that 8th grade schoolbooks of today are written at a reading level that is something less than that found in 5th grade schoolbooks prior to 1946. The study to which he referred is: Hayes, D. P., Wolfer, L. T., & Wolfe, M. F. (1996). Schoolbook simplification and its relation to the decline in SAT-Verbal scores. AMERICAN EDUCATIONAL RESEARCH JOURNAL, 33(2), 489-508. The AMERICAN EDUCATIONAL RESEARCH JOURNAL is the flagship journal of the American Educational Research Association and is arguably one of the most selective journals in the field of education. The findings of Professor Hayes, et al, are remarkable in both substance and methodology. Using a computer based assessment of large samples of reading material, Hayes and his colleagues were able to appraise the reading difficulty of reading materials from as early as 1665. Using the reading levels of widely circulated newspapers as a benchmark (a remarkably stable standard of literacy over a 300 year period), he was able to detect a sharp decline in the reading difficulty level of schoolbooks following WW II owing to the simplifications made by publishers at the behest of the professional education community. The consensus among American (and not British) educational experts was and is that such simplifications could be made without adverse educational effect. Professor Jeanne Chall is one scholarly voice that has spoken strongly against this conclusion. Of primary importance, Hayes, et al, found that the remarkable drop in SAT-Verbal scores 1963-1979 may well have been the product of the postwar dumbing down of elementary and secondary school texts. His finding implies that not only does simplified reading material result in a lessened vocabulary but a lessened fund of knowledge and correspondingly lessened aptitude for acquiring additional knowledge. His findings nicely complement Professor E. D. Hirsch's "cultural literacy" hypothesis. Both imply that the lack of core knowledge of concepts, information, and words is at the heart of student inability to gain from advanced levels of schooling. In other words, time and money spent on remedial and postsecondary education is bound to have marginal effects compared to that which could be accomplished by stronger academics at the earliest levels of schooling. Hayes also raises a point noted repeatedly by Professor Douglas Carnine at Oregon: "Finally, we find it anomalous that no drugs can be sold in the United States without first demonstrating, by experimental tests and clinical trials, their efficacy and safety, while publishers and schools can freely impose simplified readers and related schoolwork on children without having to produce experimental evidence on [their] efficacy and safety . . . ." The evidence afforded by Hayes adds significantly to the observations of Hirsch, Carnine, and so many others, and all point to a common conclusion: The schooling crisis in the U. S. is not so much the product of economic or societal change as it is an effect induced by wrongheaded pedagogical thinking. Again, I find support for a strengthened consumer voice and consumer protection such as that which might be afforded by the Clearinghouse. A final point: Clearinghouse subscribers have recently raised concerns about the dumbing down of achievement tests. Without a scientifically convincing measure of their difficulty, comparisons of tests over several generations is a challenging problem. Perhaps, the Hayes, et al LEX analysis could be used to assess the tests used over the years including the recently released Terra Nova. Regards, John J. E. Stone, Ed.D. Education Consumers Clearinghouse P.O. Box 4411 Johnson City, TN 37602 phone & fax (call first) 423-282-6832 e-mail professor@tricon.net EDUCATION CONSUMERS CLEARINGHOUSE ===================================================================== EDUCATION CONSUMERS CLEARINGHOUSE networking and information for parents and taxpayers on the internet Subscriptions & Archives: http://education-consumers.com or You are currently subscribed to education-consumers as: arthurhu@halcyon.com TO UNSUBSCRIBE: Send a blank email to leave-education-consumers-989462S@lists.dundee.net ===================================================================== For less mail, click on the following link and choose 1) a daily digest, 2) a daily list of subjects, or 3) no mail (read postings on Web) http://lists.dundee.net/scripts/lyris.pl?enter=education-consumers For more help & info: http://www.lyris.com/help or . +OK 3601 octets Received: from smtp06.nwnexus.com (smtp06.nwnexus.com [206.63.63.45]) by mail1.halcyon.com (8.8.8/8.8.8) with ESMTP id IAA03279 for ; Sat, 6 Feb 1999 08:28:31 -0800 (PST) Received: from ripple.dundee.net (ripple.dundee.net [206.249.104.12]) by smtp06.nwnexus.com (8.8.8/8.8.8) with SMTP id IAA20653 for ; Sat, 6 Feb 1999 08:28:15 -0800 (PST) From: rdyarrow@elnet.com Message-Id: Date: Sat, 06 Feb 1999 10:25:43 -0600 To: "ClearingHouse" Subject: [education-consumers] Readability Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Reply-To: rdyarrow@elnet.com Precedence: bulk Status: ===================================================================== John's idea about analyzing readability of more questionable "tests" develped these past few years is excellent. I had forgotten how easy is is to apply the Dale-Chall Readability measurement. Anyone could read the firt 20 pages of their book and begin analyzing text. It's a bit tedious counting words in the 100 word samples, but it isn't difficult. As I was doing it, memories of analyzing reading tests in such a manner when I was in el ed training 20+ years ago came flooding back. As part of our training, we all had to learn how to apply readability scales. At Elaine McEwan's suggestion I ordered the book I used from Amazon.com and had it within two days: Dale-Chall Readability Formula (from Readability Revisited: The New Dale-Chall Readability Formula, Jeanne S. Chall and Edgar Dale, Brookline Books, 1995 ISBN 1-57129-008-7). The LEXILE submission was more difficult and only one parent on the entire Illinois Loop was able to submit the reading sample in a format which they could interpret. You have to have the files in PDF format (or something like that) and it proved a stumper for most of us. After doing this, I am convinced that readability is a powerful tool that we could use more often to unmask reading assessments which might be flawed...or when teachers have their students read material way below their level. When my daughter graduates at the end of the year, I plan to buy copies of "Who Put That Hair In My Toothbrush" for all of the school board member, which I will give to them at a meeting, relating how my daughter's teacher read this 3-4th grade level book to the eighth grade class for 2 1/2 months. I will then have a page of 8th grade text from 1920 so that they can compare the two before I once more ask them to reinstate a required reading list for each grade level. Just the facts ma'am. Mary ===================================================================== EDUCATION CONSUMERS CLEARINGHOUSE networking and information for parents and taxpayers on the internet Subscriptions & Archives: http://education-consumers.com or You are currently subscribed to education-consumers as: arthurhu@halcyon.com TO UNSUBSCRIBE: Send a blank email to leave-education-consumers-989462S@lists.dundee.net ===================================================================== For less mail, click on the following link and choose 1) a daily digest, 2) a daily list of subjects, or 3) no mail (read postings on Web) http://lists.dundee.net/scripts/lyris.pl?enter=education-consumers For more help & info: http://www.lyris.com/help or . +OK 4767 octets Received: from smtp06.nwnexus.com (smtp06.nwnexus.com [206.63.63.45]) by mail1.halcyon.com (8.8.8/8.8.8) with ESMTP id KAA21362 for ; Sat, 6 Feb 1999 10:01:59 -0800 (PST) Received: from ripple.dundee.net (ripple.dundee.net [206.249.104.12]) by smtp06.nwnexus.com (8.8.8/8.8.8) with SMTP id KAA24483 for ; Sat, 6 Feb 1999 10:01:46 -0800 (PST) Date: Sat, 6 Feb 1999 12:01:37 -0600 (CST) From: Leon Todd To: "ClearingHouse" cc: 71524.2205@compuserve.com, ClearingHouse Subject: [education-consumers] Readability In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Reply-To: Leon Todd Precedence: bulk Status: ===================================================================== The only reason these exit proficiency exams are there, and the only reason they keep raising the bar and insisting on rigorous (ever look of the word rigorous?) standards, is to discredit public education that must take care of all of the baggage of sociey, dysfunctional families and incompetent universities preparing teachers to teach what doesn't work. Look at what they are doing: they have us arguing about graduation requirements and distracting us from focusing on core knowledge curriculum, phonics or getting rid of STW as it is currently constituted, all curriculum oriented subject matters. It curriculum first. Of course your test scores suck if your curriculum and family cultures suck and no scale is going to change bad test scores that result from bad curriculum. I apologize to all for letting my hair down! ------------------------------------- On Sat, 6 Feb 1999 rdyarrow@elnet.com wrote: John's idea about analyzing readability of more questionable "tests" develped these past few years is excellent. I had forgotten how easy is is to apply the Dale-Chall Readability measurement. Anyone could read the firt 20 pages of their book and begin analyzing text. It's a bit tedious counting words in the 100 word samples, but it isn't difficult. As I was doing it, memories of analyzing reading tests in such a manner when I was in el ed training 20+ years ago came flooding back. As part of our training, we all had to learn how to apply readability scales. At Elaine McEwan's suggestion I ordered the book I used from Amazon.com and had it within two days: Dale-Chall Readability Formula (from Readability Revisited: The New Dale-Chall Readability Formula, Jeanne S. Chall and Edgar Dale, Brookline Books, 1995 ISBN 1-57129-008-7). The LEXILE submission was more difficult and only one parent on the entire Illinois Loop was able to submit the reading sample in a format which they could interpret. You have to have the files in PDF format (or something like that) and it proved a stumper for most of us. After doing this, I am convinced that readability is a powerful tool that we could use more often to unmask reading assessments which might be flawed...or when teachers have their students read material way below their level. When my daughter graduates at the end of the year, I plan to buy copies of "Who Put That Hair In My Toothbrush" for all of the school board member, which I will give to them at a meeting, relating how my daughter's teacher read this 3-4th grade level book to the eighth grade class for 2 1/2 months. I will then have a page of 8th grade text from 1920 so that they can compare the two before I once more ask them to reinstate a required reading list for each grade level. Just the facts ma'am. Mary ===================================================================== EDUCATION CONSUMERS CLEARINGHOUSE networking and information for parents and taxpayers on the internet Subscriptions & Archives: http://education-consumers.com or You are currently subscribed to education-consumers as: arthurhu@halcyon.com TO UNSUBSCRIBE: Send a blank email to leave-education-consumers-989462S@lists.dundee.net ===================================================================== For less mail, click on the following link and choose 1) a daily digest, 2) a daily list of subjects, or 3) no mail (read postings on Web) http://lists.dundee.net/scripts/lyris.pl?enter=education-consumers For more help & info: http://www.lyris.com/help or . From: rdyarrow@elnet.com Date sent: Sun, 07 Feb 1999 17:04:12 -0600 To: Arthur Hu Subject: Re: [education-consumers] NEW ILLINOIS ISAT STATE TEST Elaine sent me the following letter several months back which first interested me in doing this myself. By taking the information from her posting on the Dale)Chall book, you can go to Amazon.com and order it (It arrived at my house two days later.) I cannot stress how easy it is to apply the information. >>>Mary, >>> >>>I was so intrigued by how much time the 8th grade class you described has >>>invested in the study of "Who Put That Hair in My Toothbrush? that I decided >>>to purchase the book and apply the time-tested Dale-Chall Readability >>>Formula (from Readability Revisited: The New Dale-Chall Readability Formula, >>>Jeanne S. Chall and Edgar Dale, Brookline Books, 1995 ISBN 1-57129-008-7). >>>This formula has been widely used to determine reading levels of literature >>>and textbooks and most recently it was used to evaluate the difficulty of >>>the reading test in Texas. >>> >>>The methodology used for a book over 200 pages in length is to take 100 word >>>samples at 50 page intervals and apply the formula to these samples. I began >>>on page 7 of the book and took samples on pages 57, 107, 157, and 200. >>>Using the formula, one counts the number of complete sentences (fewer >>>sentences in the 100 word sample indicates more difficult writing) and then >>>counts the number of unfamiliar words. Unfamiliar words are any which do not >>>appear in the list of 3000 words known by students in Grade 4 (the list is >>>included in the book). The more unfamiliar words in the sample, the more >>>difficult the reading is judged to be. >>> >>>Who Put That Hair in My Toothbrush? received between a 3rd and 4th grade >>>reading level score unless one includes the first sample on page 7 where >>>there were 14 sentences and only 1 unfamiliar word. That put the reading >>>level closer to 1st grade. >>> >>>Of course the main character is twelve and her brother is fourteen so their >>>experiences are certainly relevant to an 8th grade class. The age level >>>printed on the back cover by the publisher is age 10 and up. SO.....while >>>this might qualify for "free reading" for an 8th grader, to consider it as >>>"reading curriculum" for more than two months is beyond belief. I wonder if >>>the principal of the school who is desperately trying to raise reading >>>achievement has a clue about what's going on! >>> >>>Elaine >>> >>>Elaine K. McEwan-Adkins >>>The McEwan-Adkins Group >>>(520) 544-4088 >>>(520) 544-8764 >>>http://www.elainemcewan.com >>> >