Collateral Damage: How High Stakes Testing Corrupts America's Schools
by Gary Natriello
Title: Collateral Damage: How High Stakes Testing Corrupts America's Schools
Author(s): Sharon L. Nichols and David C. Berliner
Publisher: Harvard Education Press, Cambridge
ISBN: 1891792350, Pages: 200, Year: 2007
If you are looking for a synthesis that pulls together a good deal of the background literature on high stakes testing, combines it with some of the latest research on testing practices, and the myriad press accounts of the reactions to NCLB testing policies from teachers, administrators, parents, and students, then look no further than Collateral Damage: How High-Stakes Testing Corrupts AmericaĂ˘€™s Schools by Sharon Nichols and David Berliner. Nichols and Berliner have marshaled a wealth of information to support their contention that high-stakes testing, far from having the uplifting effects promised by advocates and the policy makers who have embraced it as a reform lever for U.S. schools, is undermining the educational system and corrupting all those who are touched by it.
Nichols and Berliner lay out their case in seven well organized and accessible chapters. In Chapter 1 they highlight some long-standing criticisms of high-stakes testing and systematically dismantle ten claims commonly made in support of the policy, claims such as Ă˘€śScoring well on high-stakes tests leads to feelings of success by students, while doing poorly on such tests leads to increased effort to learnĂ˘€ť (p. 9). They continue by considering the reasons for the rise of high-stakes testing, linking it to the increasing prominence of business and accountability, the growing belief in the importance of a highly educated workforce, the desire to maintain privilege, and the American love of public spectacles. Chapter 1 concludes with an introduction to what Nichols and Berliner refer to as CampbellĂ˘€™s law, named for Don Campbell. The law, which serves as a thread throughout the book, states that Ă˘€śthe more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it was intended to monitorĂ˘€ť (p. 23). Readers doubting the force of the law can consider the examples presented in the book or merely consider the federal budget as an important social indicator long since distorted beyond its capacity to shed light on the real actions of the federal government.
Chapter 2 goes directly to the claim of the corrupting effect of high-stakes testing by considering the various forms and sources of cheating that have emerged in its wake. There is cheating by adults, teachers and administrators, and this cheating takes place prior to the test as when students are given inappropriate information on the content of the question, during the test as when students are given the answers to test questions, and after the test as when teachers or administrators change student answers to test questions. Of course, there is also cheating by students, sometimes aided by educators. To some extent cheating is sustained by the failure to confront cheaters and by the sense that resistance to the tests in whatever form may be justified.
Chapter 3 takes on the special issue of excluding students from school in an effort to shape the pool of test takers and thereby shape the performance profile of the school on the high-stakes test. Nichols and Berliner report examples of school districts that withdraw students from their rolls to prevent them from pulling down the overall statistics for the school. Some of these efforts entail direct communications with students to, in effect, Ă˘€śgo away.Ă˘€ť The chapter also considers the situation for special education students and students with limited English proficiency, and it portrays students in these categories whose educational careers have been negatively affected by the testing requirements of NCLB. Even more troubling is what Nichols and Berliner view as a growing lack of caring and compassion as educators are forced to implement the dictates of regulations developed by distant bureaucrats who lack knowledge of local conditions. Of course, there is one group of students who are getting increased attention as a result of high-stakes testing. These are the students who are close to passing the test. Such students receive disproportionate attention as they are coached and drilled over the passing line while those well below the line and those clearly past the line are left to their own devices as part of the educational triage that high-stakes testing sets in motion.
If you thought that the corrupting influences of high-stakes testing were limited to local schools and districts, Chapter 4 makes the case that states are directly involved in manipulating performance data beginning with misrepresentations of graduation rates and dropout statistics. Some states are also engaged in setting low passing scores to maximize the proportions of students achieving Ă˘€śproficiencyĂ˘€ť in order to escape some of the provisions of NCLB. Added to these patterns of deliberate manipulation, are a variety of errors in data reports stemming from test-scoring errors, test-reporting errors, and test-taking errors made by students. Such errors are serious because the consequences for individual students of high-stakes testing are serious.
Nichols and Berliner take up the issue of validity in Chapter 5 and identify a host of validity problems that seem endemic to high-stakes testing. Content validity cannot be achieved in most states where the high-stakes tests have not been aligned with the curriculum standards. Construct validity is compromised when the high-stakes environment leads to extensive efforts to drill students on items very similar to those used on the test. Criterion validity has not been established for the high-stakes tests employed by states because there are few studies of what such tests predict. Nichols and Berliner raise serious concerns about the consequential validity of high-stakes tests when a single indicator is used to make decisions regarding high school graduation in a context in which high school graduation is related to lifeline earnings and employment opportunities.
Chapter 6 discusses the impact of high-stakes testing on teacher and administrator morale, teacher and administrator stress, and the tendency of teachers and administrators to leave the field of education. Nichols and Berliner also consider the stress and anxiety that high-stakes testing imposes on students as well as efforts to reduce such stress and attempts to bribe students to increase their test scores. The chapter concludes with a discussion of the de-skilling of the teaching profession that high-stakes testing seems to engender by removing decision-making responsibility from teachers and consigning them to roles as technicians or trainers whose days are spent drilling students on test items.
Chapter 7 concludes the book with a call for a moratorium on high-stakes testing programs. Nichols and Berliner review the conditions deemed necessary for all high-stakes testing programs by the American Educational Research Association, and conclude that most contemporary high-stakes testing programs fail to meet these professional standards. They continue by offering a number of alternative approaches to the high-stakes tests that dominate U.S. education at the current time. These alternatives include a richer set of formative assessment strategies embedded in ordinary classroom activities coupled with low-stakes external assessments. They also recommend the creation of an independent inspectorate that would conduct school visits to monitor professional standards in the conduct of schooling, a system of end-of-course examinations created and scored by teams of teachers, and a system of student performance tests, including project and portfolio presentations before teams of external judges.
Nichols and Berliner have set forth a challenging but moderate position on high-stakes testing that should appeal to professional educators and educational researchers because it seeks to align assessment policy with the professional standards of practitioners and the professional standards of researchers. Their analysis makes it clear that current high-stakes testing policies fail to conform to widely recognized professional standards of the research and assessment communities and that such policies serve to undermine the professionalism of the educators upon whom we rely to operate our schools.
Supporters of high-stakes testing are unlikely to be convinced by the case put forward by Nichols and Berliner, not because the case is unsupported, but because the same evidence is interpreted quite differently. For example, supporters of high-stakes testing are likely to view the teacher, administrator, and student stress reported by Nichols and Berliner as a plus since it indicates that the policy is encouraging everyone to redouble their efforts. As for those teachers and administrators leaving the field because of high-stakes testing, supporters are likely to view this as an opportunity to re-staff with a heartier cohort attuned to a more ambitious achievement agenda. In the case of special populations, supporters of high-stakes testing are likely to view including them in the testing regime as more respectful of their capabilities to perform. What Nichols and Berliner view as a narrowing of the curriculum and a reduction of teachers to trainers, proponents of high stakes testing are likely to view as a focusing on the most important content and a reduction in the off-task behaviors of teachers who heretofore were free to follow their own whims in the classroom. So, proponents of high-stakes testing are unlikely to be moved by this book.
Another group unlikely to be moved includes those who adopt a more critical perspective on assessment than Nichols and Berliner. Although individuals in this camp would appreciate the call for a moratorium on high-stakes testing, they would go further and question the over-reaching of governmental and professional actors into the personal lives of citizens to render and record judgments of individual capabilities, even if based on alternative assessment strategies. Individuals in this camp would agree with Nichols and BerlinerĂ˘€™s observation that the setting of proficiency standards is a social and not a scientific act, but they would add that it is not an accident that such standards are inevitably set to allow some to pass and others to fail (and never all to pass or all to fail) since the role of state-sponsored schooling in the industrial era has been to provide a theory of personnel to undergird the rationing of social and economic opportunity. Finally, those with a more critical perspective on assessment might nudge Nichols and Berliner to bring their criticisms of high stakes testing in the k-12 sector to bear on higher education where the exclusion of students with low scores is lauded, where there are entire departments (Admissions) dedicated to manipulating test scores and institutional profiles, and where the limited evidence of the criterion validity of the major test is produced by the sponsor of the test (pp. 104-106).
Overall the comprehensiveness, balance, and accessibility of this volume make it a good candidate for reading by educators, school boards, and policy makers. It is sure to create opportunities for rich discussions of an issue that is increasingly salient in public discourse on education. Moreover, the careful scholarship that informs this work makes it suitable for adoption in courses on assessment, policy, and school reform. I will be adding it to my required reading.
Teachers College Record
INDEX OF NCLB OUTRAGES