Review: Collateral Damage: How High-Stakes Testing Corrupts America's Schools
Book Review: Nichols, Sharon L. & Berliner, David C. (2007). Collateral Damage: How High-Stakes Testing Corrupts America's Schools. Cambridge, MA: Harvard Education Press.
The authors give powerful examples of all the ways individuals, schools, and states can--and do--cheat the accountability system but refuse to play the "Gotcha!" game so beloved by the media. Instead, the authors ask the question that the media ignore: "We should be asking why so many competent and decent professionals think the system they are in is so unfair to their students, their schools, and themselves and, as a result, feel justified in doing direct test preparation, violating standardization procedures, and cheating." (p. 52)
Chapter 3, Excluding Students from Education By Design and By Neglect, the Crisis of Caring in Our Schools, and the Special Case of the "Bubble Kids," describes institutional kinds of cheating. The chapter gives examples of schools and entire systems that push kids out of school to improve the test-taking pool and schools that put great effort and attention on those students deemed close to passing the test. These kids get special attention and tutoring while students doing less well get little extra help and more able students are pretty much ignored because their success is a given. British researchers call this educational triage, dividing students into three groups: non-urgent cases, suitable cases for treatment, and hopeless cases.
Chapter 4, States Cheat Too! How Statistical Trickery and Misrepresentation of Data Mislead the Public, describes cheating at the district and state level. Here we get fudging on the number of high school dropouts. Nichols and Berliner name prominent names here.
As Nichols and Berliner document, Houston was not alone.
In their discussion of cut scores, the authors point out something that cannot be repeated enough: Cut scores on tests, determining who is proficient and who is not, are political decisions. They are not scientific or psychometric decisions. (p. 87) Repeat: Cut scores are political decisions. I dream of the day one million teachers march on Washington, carrying banners: Cut scores are political decisions. Or maybe the signs should just say, We know it's politics, Mr. Kennedy.
This chapter also includes a number of stories--ranging from students refusing to take required state tests to teachers refusing to give them to shoddy test questions, test scoring mishaps, and the dubious qualifications of people grading the tests. The stories have been told before, but because of the hard core secrecy surrounding the tests, we repeat the only stories we are able to get our hands on. Teachers have become so intimidated by threats of dismissal that they don't tell what they know. The real scandal here is that because of test secrecy and layers of bureaucratic intimidation, even educators don't know the depth of the problem and they are afraid to reveal what they do know. In a number of states, teachers are threatened with losing their jobs if they even look at the test. One must ask how professional it can be that teachers are forcing children to take tests that are likely to be both developmentally inappropriate and filled with error. One can wonder how the teacher unions and professional organizations can defend their silence about teachers blindly handing out secret tests with dubious validity, tests that make children vomit.
Here is a small example from a McGraw-Hill test given to New York fourth graders some years back. Children were asked to read a passage about a chance meeting between a young girl named Julie who wandered away from her class field trip and a wispy-haired man in Princeton, a man who wore no socks. An afterword informs young readers that the man was Albert Einstein, whose Special Theory of Relativity "is sure to play a big role in human expeditions to the stars." It also explains that the story is based on a real incident involving Mary Budd Rowe, "an education professor at Stanford University in Palo Alto, California." The passage ends with the rhetorical question, "Don't you think she was a great person to be teaching teachers?"
Does any fourth grader in the country care who teaches teachers? For starters, fourth graders are worrying about
* Why Julie wandered away from her school group
* Why Julie talked to a stranger
* Why Julie changed her name to Mary Budd Rowe.
And then there's the problem that two of the three test questions focus not on the story but on the Afterword, with its discussion of the Special Theory of Relativity
Here is one of the three tedious writing prompts accompanying this item: "Pretend you are either Julie OR Einstein. Write a letter to a friend describing your meeting at the fountain, and what you thought about the person you met. Use details from the story and the 'Afterword' in your letter." What fourth grade boy is willing to pretend he's a girl? This leaves him the job of writing in the persona of Einstein.
D. H. Lawrence once observed, "Without secrecy there would be no pornography." Surely this applies as much to the testing industry as to obscenity. Without secrecy, there could be no pretense that tests marketed as measurements of reading ability aren't actually measuring something else entirely. But the State colludes with test publishers to keep these tests secret because without secrecy, the state loses the leverage that helps it do the bidding of the corporate-politico alliance that has been calling for high standards and testing since the 1980ies, and alliance bent on keeping teachers and students scared and subservient. They call it "rigor," a word worthy of study.
If scholars could employ methods of discourse analysis and talk with schoolchildren about why they chose the answers they did, we would all learn a lot about the pervading adult perspective of the tests, about cultural and class assumptions that dominate the tests. We would also see how convincing children's "wrong" answers are and how the reading tests aren't about reading ability at all and how the math tests are complicated by many other factors besides facility with numerical relationships. Imagine asking Massachusetts, New York, Florida, Texas, and California fourth graders why they answered questions the way they did. The whole lid would come off state-run accountability systems. And corporate-politicos would scream that our position in the global economy would wither. Never mind that among industrialized nations, we already rank 20 out of 21 in child well-being.
Ask fourth graders why they answered questions the way they did and public schools across America could no longer be the scapegoat for a host of bad business decisions. There's a reason the Business Roundtable is the chief cheerleader for the renewal of NCLB. So-called accountability in education is, of course, modeled on what passes for accountability in corporate America, i.e., an increase in productivity. As Kathy Emery and I point out in Why Is Corporate America Bashing Our Public Schools? (Heinemann 2004 ), from the hand-holding at the 1989 corporate-politico education summit in Charlottesville through the passage of the No Child Left Behind Act in January 2002, the Business Roundtable has been pushing tests as the chosen way to measure school productivity. The Business Roundtable doesn't veer from its message, which is echoed in newspaper headlines and on many editorial pages ranging from the New York Times to the rural weeklies: school effectiveness (and hence teacher quality) is rated up or down according to student scores on tests. Any attempt to explain that schools are concerned, not with widgets but with students in all their variety, is met with the rejoinder "No excuses."
In a list of ten arguments offered by supporters of the NCLB legislation, Nichols and Berliner posit number five as the theory of action behind the law, the very heart of NCLB: Teachers need to be held accountable through high-stakes tests to motivate them to teach better and to push the lazy ones to work harder. Nichols and Berliner counter that, based on their first-hand observation in hundreds of schools, they believe the percentage of lazy teachers "is considerably smaller than the percentage of lazy politicians who do not read the legislation they support." Hoo-hah! Let's put this up in faculty rooms across America.
I admit that, even though the authors assert that "validity is the most important characteristic of a test," for me, worrying over the distinctions between content validity, construct validity, criterion validity, and consequential validity is truly academic. Unless and until we make tests public, allowing educators to talk to children about why they responded the way they did, then I say all this is spinach and I say to hell with it.
That said, in Chapter 5, What Happens to Validity When Indicators are Distorted and Corrupted, the Many Ways That High-Stakes Testing Promotes Such Distortion and Corruption, and How Those Practices Lead to Confusion About What Test Scores Mean, Nichols and Berliner pose a question that should be on the lips of everyone who plans to spend the next couple of decades in this country:
I thank the authors for caring about the students at the World of Opportunity in Birmingham, Alabama, a school with which I have longstanding connections. I thank them for showing readers that we all have a stake in the horrendous policy of pushing kids out of school. And I would extend their question to the assaults on younger children, also documented in the book. Do we want to live with the consequences of children growing up without art, music, and recess, children who have been condemned to a school life of relentless test prep? Children who have grown up hearing the constant message that they don't measure up? That they aren't good enough?
The school board in my village is currently discussing a policy of depriving middle schoolers entrance to school dances if they aren't up-to-date on their homework assignments. Surely the obsession with homework would not be so prevalent if it weren't for the obsession with making adequate yearly progress on the tests required by NCLB. Vermont changed its whole testing product and procedure in order to march behind the U. S. Department of Education drumbeat. We don't yet have a high school exit exam, but one can wonder how long our politicos will hold out. I think of the slightly altered version of Nichols and Berliner's question and I know I don't want to live next door to citizens who were demeaned and diminished as primary graders, excluded from school dances as middle graders, and denied high school diplomas. Such kids will grow up to be very angry adults. Children trained in the Business Roundtable model of a dog-eat-dog world of maximized global economy profits don/t get educated for the common good. If for no other reason than our own well-being, we must all care about this.
The book ends with a bang. The authors call for an immediate moratorium on the use of high-stakes testing and they give a dozen strong reasons why NCLB cannot be defended. They do this to "stop the wreckage of our public educational system." Throughout, they have shown that if we allow our public educational system to be wrecked, our very social fabric will crumble.
About the Reviewer
Susan Ohanian, a longtime teacher, is a Fellow at the Vermont Society for the Study of Education. She is a co-founder of Educator Roundtable. In addition, she is a free-lance writer whose articles have appeared in periodicals ranging from the Atlantic and Washington Monthly to Phi Delta Kappan and Education Week. Susan is the recipient of The George Orwell Award for Distinguished Contributions to Honest and Clarity in Public Language, National Council of Teachers of English, 2003; The Kenneth S. Goodman "In Defense of Good Teaching" Award, College of Education, University of Arizona; and The John Dewey Award for Extraordinary Contributions to the Education of Young People In America (2006).
FAIR USE NOTICE
This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available in our efforts to advance understanding of education issues vital to a democracy. We believe this constitutes a 'fair use' of any such copyrighted material as provided for in section 107 of the US Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit to those who have expressed a prior interest in receiving the included information for research and educational purposes. For more information click here. If you wish to use copyrighted material from this site for purposes of your own that go beyond 'fair use', you must obtain permission from the copyright owner.