Lake Wobegon Effect Date sent: Mon, 9 Mar 1998 22:44:48 -0500 From: "Richard G. Innes" <70224.434@compuserve.com> Subject: Re: Terra Nova from Charlene To: ed-consumers main listing Charlene (and the LOOP/ECC), <> I am not a testing expert, either, but I have looked at some of the numbers. More importantly, what you are saying is quite interesting and "unusual." Terra Nova is marketed as a nationally-normed test. The norming was done about two years ago, and does not have to be repeated to generate national comparison scores. If what you say above is correct, then it could be your school system really isn't going to get a national comparison score. Your results will only compare your kids to other kids in your state alone. If so, then if your state is overall weak compared to the rest of the country, you will have no way to know that. That could be a nice dodge for your state department of education, but it would be very misleading to parents. However, I really need to know more about what is going on in Tennessee to really understand what is being proposed. ((Dr. Stone, have you heard this?)) <> This refers to a mythical place from Garrison Kellor's PBS comedy program that often talked about Lake Woebegon where every student was above average. In the late 1980's a non-educator discovered that every state in the country was reporting that its testing scores were above the national average. Obviously, that wasn't possible. It turned out that when nationally-normed tests were given more than once, teachers got better at remembering the questions (which don't change) and making sure their students were well prepared for just those specific questions. Scores thus became inflated, and every school and system tested out "above average." This is still a problem today with just about any test that does not change questions, but there are some work-arounds. First, better tests are available in multiple forms. While each form has different questions, they are all normed at the same time, so scores, at least in theory, are comparable with high accuracy from one form to the other. The different forms, however, make it hard for teachers to look good by teaching to just a narrow set of questions. A school system that really wants to take a hard look at itself will demand different test forms each year (and a system that wants to hide something will make darn sure it does not change testing forms each year). Kentucky tried another approach. They changed a large number of questions each year, and then tried to develop a very exotic formula to "equate" the different tests so scoring would be even. To date, that has not worked (in fact, it has been somewhat of a disaster, in my opinion, not necessarily shared by others). I hope that helped more than confused. Richard Innes EDUCATION CONSUMERS CLEARINGHOUSE