z56\doc\web\2002\05\testab.txt PORTFOLIO ASSESSMENT FOUND TO BE UNRELIABLE, INVALID TITLE Developing a valid and reliable portfolio assessment in the primary grades: Building on practical experience. AUTHOR Shapley, Kelly S.; Bush, M. Joan AFFILIATION Office of Institutional Assessment & Evaluation, Southwest Educational Development Lab, Austin, TX, US, 1 SOURCE Applied Measurement in Education. 1999 Vol 12(2) 111-132 "the portfolio assessment did not provide high quality information about student achievement as hoped. The reliability of the scores was low, and the portfolio contents did not provide a valid sample of students' work. " STUDENTS WHO FLUNK KIRIS ACE THE ACT TITLE Kentucky Performance Assessment of Mathematics: Do the numbers add up? 13% of those judged less than proficient had ACT mathematics scores of 22 and above. The results indicate that the Kentucky constructed response tests do not adequately discriminate differences in mathematics abilities, as measured by the ACT, AUTHOR Strong, Sue; Sexton, Larry C. AFFILIATION Eastern Kentucky U, Counseling & Educational Psychology, Richmond, KY, US SOURCE Journal of Instructional Psychology. 1997 Sep Vol 24(3) 202-206 PERFORMANCE ASSESSMENT INVALID, DEFECTIVE, DANGEROUS TITLE Performance assessment for state accountability: Proceed with caution. because it lacks genuine reliability and validity, its use for high-stakes decisions including State accountability is dangerous. ...became a means of changing behavior rather than a mechanism for gathering information ... prepared to undertake it without sufficient quality control" (S. Dunbar et al, 1991). (PsycINFO Database Record (c) 2002 APA, all rights reserved) AUTHOR Strong, Sue; Sexton, Larry C. TITLE Developing a valid and reliable portfolio assessment in the primary grades: Building on practical experience. ABSTRACT In response to assessment guidelines defined by the Improving America's School Act of 1994, a reading/language arts portfolio assessment was included in the Dallas Public Schools' Title I Local Education Agency Plan as an optional student assessment for students in prekindergarten to 2nd grade. The authors examined the extent to which the 1995-196 portfolio assessment met appropriate technical standards for its intended uses. Classroom teachers scored their own students' portfolios, and then 42 teachers participated as 2nd raters to re-score a random sample of portfolios from 10 schools. 150 portfolios were judged on instructional goals. After 3 yrs of development and gradual improvement, the portfolio assessment did not provide high quality information about student achievement as hoped. The reliability of the scores was low, and the portfolio contents did not provide a valid sample of students' work. (PsycINFO Database Record (c) 2002 APA, all rights reserved) AUTHOR Shapley, Kelly S.; Bush, M. Joan AFFILIATION Office of Institutional Assessment & Evaluation, Southwest Educational Development Lab, Austin, TX, US, 1 SOURCE Applied Measurement in Education. 1999 Vol 12(2) 111-132 ________________ STUDENTS WHO FLUNK KIRIS ACE THE ACT TITLE Kentucky Performance Assessment of Mathematics: Do the numbers add up? 13% of those judged less than proficient had ACT mathematics scores of 22 and above. The results indicate that the Kentucky constructed response tests do not adequately discriminate differences in mathematics abilities, as measured by the ACT, AUTHOR Strong, Sue; Sexton, Larry C. AFFILIATION Eastern Kentucky U, Counseling & Educational Psychology, Richmond, KY, US SOURCE Journal of Instructional Psychology. 1997 Sep Vol 24(3) 202-206 TITLE Kentucky Performance Assessment of Mathematics: Do the numbers add up? ABSTRACT Comparisons were made between mathematics abilities as measured by the Kentucky constructed response assessment with the more traditional format of the multiple choice items of the ACT. There was considerable overlap of ACT scores across the 4 Kentucky categories of Novice, Apprentice, Proficient and Distinguished. 27% of the high school seniors judged as Novice on the Kentucky assessment had ACT mathematics scores which ranged from 18-36. 13% of those judged less than proficient had ACT mathematics scores of 22 and above. The results indicate that the Kentucky constructed response tests do not adequately discriminate differences in mathematics abilities, as measured by the ACT, at either the upper or lower levels of the rating scale. Gender differences in performance were found to be the same as traditional assessments: Boys outperformed girls in the Kentucky assessment in mathematics. AUTHOR Strong, Sue; Sexton, Larry C. AFFILIATION Eastern Kentucky U, Counseling & Educational Psychology, Richmond, KY, US SOURCE Journal of Instructional Psychology. 1997 Sep Vol 24(3) 202-206 __________________ PERFORMANCE ASSESSMENT INVALID, DEFECTIVE, DANGEROUS TITLE Performance assessment for state accountability: Proceed with caution. because it lacks genuine reliability and validity, its use for high-stakes decisions including State accountability is dangerous. ...became a means of changing behavior rather than a mechanism for gathering information ... prepared to undertake it without sufficient quality control" (S. Dunbar et al, 1991). (PsycINFO Database Record (c) 2002 APA, all rights reserved) AUTHOR Strong, Sue; Sexton, Larry C. TITLE Performance assessment for state accountability: Proceed with caution. ABSTRACT The cry for accountability and authenticity in education is louder than the cry for reason and a scientific approach. Testimonials about the value of Authentic assessment in all its forms outweigh and overwhelm the quiet and more rational pleas for genuine test reliability and validity. Here the literature related to the major problems of performance assessment is reviewed. These include bandwagoning, learning vs accountability, validity vs face validity, high stakes decisions, sampling and transferring knowledge, testing time, cost, and fairness. The problems are explored and presented in table format with the names of their significant researchers. Measurement experts conclude that performance assessment may be appropriate for some types of instruction in narrowly defined curriculum. However, because it lacks genuine reliability and validity, its use for high-stakes decisions including State accountability is dangerous. (PsycINFO Database Record (c) 2000 APA, all rights reserved) AUTHOR Strong, Sue; Sexton, Larry C. AFFILIATION Eastern Kentucky U, Richmond, KY, US SOURCE Journal of Instructional Psychology. 1996 Mar Vol 23(1) 68-74 TITLE A validity of the Kentucky Performance Based Assessment System with National Merit Scholars and National Merit Commended. ABSTRACT This study explored the validity of Kentucky's Performance Based Assessment System with National Merit Scholars and National Merit Commended. A significant percentage of these scholars were judged by the Kentucky tests as nonmastery in reading, math, science and social studies as well as in their writing portfolio. The results indicated that this assessment system is highly suspect as a valid measure for high achieving scholars. "During the 1980's, testing became a cornerstone of the education reform movement. Testing burgeoned: the consequences for performance on tests were greatly increased: and assessment increasingly became a means of changing behavior rather than a mechanism for gathering information ... Now the nation stands poised on the brink of yet another wave of test-based reform, and again we appear prepared to undertake it without sufficient quality control" (S. Dunbar et al, 1991). (PsycINFO Database Record (c) 2002 APA, all rights reserved) AUTHOR Strong, Sue; Sexton, Larry C.