Making the Grades: My Misadventures in the Standardized Testing Industry by Todd Farley
Polipoint Press (October 1, 2009)
The buzz is starting over Todd Farley's new book Making the Grades: My Misadventures in the Standardized Testing Industry. Jonathan Kozol pronounces it "Dynamite!" Dan Brown tags it "hilarious." Me? I worry about the problem of the unreliable narrator.
This a fellow who brags about his climb to the top of the gravy train in a 15-year-career in the testing industry, starting off as a test corrector and progressing to positions of trainer, fixer, and finally, a consultant tapdancing on the hearts and souls of children forced to take high stakes tests.
Farley seems to have worked for all the big ones--Pearson, NCS, Riverside, and Educational Testing Service. And he brags about how he cooked the books, everywhere he went saying, "The eraser was my only friend." Unable to do things legitimately, he "simply fudged the statistics," calling himself the Man with the Omnipotent Eraser. After all, he reminds us, he was only a temporary employee taking shortcuts.
In a refutation of 'the buck stops here,' not to mention Kant's categorical imperative, throughout the book we are told that the blame lies elsewhere.
The marketing materials accompanying the book state that "When Farley accepted a job at a standardized test scoring company, after a mere four hours his first day on the job, he realized what a joke standardized testing really is." Joke? Surely not a joke to the neighborhoods whose schools are turned into test-prep factories, to the teachers driven out of the profession, to the children for whom the very concept of childhood is put at risk.
I admit that early on Farley's smart-guy-among-the-philistines smugness got on my nerves so much that I almost forgot to laugh at the quite hilarious accounts of test-grader incompetence and idiocy. Not to mention his demolition of the rubrics held in such high regard by the testing industry. Usually, anyone who skewers rubrics is my pal.
Throughout, Farley invites us to share his view of his own importance--from not having to share a company-provided room at Embassy Suites to jetting "off to Princeton to work with the bigwigs at ETS," In his words, "quite a coup for me." I find it troubling that the fellow who saw so many flaws and downright chicanery in the testing industry continued to work there for 15 years, bragging all the while about being the wise one among misfits and incompetents.
Farley used the chance to write multiple-choice questions for the California High School Exit Exam (CAHSEE) at $25 a pop, and short stories at $500 to pump up his resume, which he parlayed into a full-time job at ETS, "The mother ship of the assessment world." There he gloried in a huge office with two desks, where he got fat bonuses and fat raises even though the work was so middling he spent half the day off task. He mentions a six-figure salary. Truth in disclosure: Although I once quit a job after the CEO offered me a bribe worth a yearÃ¢€™s salary to do something legal but contrary to the best interests of the students in the program I worked in, IÃ¢€™ve never come close to earning a six-figure salary.
After studying scorer responses to student short answers on NAEP samples, I'm ready to believe almost anything about the test scorers, and yet FarleyÃ¢€™s tales of foreigners who donÃ¢€™t understand English and basic incompetents cause me to pause. Believable or comic exaggeration? Each reader will have to decide. He does tell a good story.
Farley did quit ETS after three years. Did a crisis of conscience over what he reports as fudging of NAEP results cause him to leave? No, he was annoyed by the company pressure to contribute to United Way every year.
Farley waits 222 pages--until the Epilogue--to offer any reflection on what damage the testing industry is doing. Until then, this is a tale of "I'm smart and they're dumb.'
In the epilogue Farley says that the dirty, little secret of the standardized testing industry is that "the people hired to read an score student responses to standardized tests are. . .the dregs of the working world, the 'uglies and unhirables'. . . .scorers capable of doing a good job moved on to other careers, but the inept settled in for the long haul."
Funny thing. Farley was one of them for 15 years.
Maybe it doesn't matter that we have so little reason to trust the narrator, who gleefully accounts how he cheated to get the numbers right. Maybe it's enough that this account will bring needed attention to an out-of-control industry that hurts our children and causes grave damage to the very fabric of public education.
Maybe the point here is that even when a book is written by a rat, it may be useful.