Flunking the Test: New Generation
Ohanian Comment: This article was part of a May 2004 series "Flunking the Test" that won a first-place award for education writing in the 71st annual National Headliner Awards.
And it is well worth reading again. Three cheers! The article has some depth. The reporters didn't settle for cheerleaders for the so-called newer and better tests. They contacted savvy people like Clifford Hill, co-author of the book I've praised so often on this site that I should get an agent's commission. Children and Reading Tests. Read it. Hill says all the hooplah about new tests is old wine in new bottles. We were told there would be new kinds of testing, but they are doing things no differently.
This said, be careful what you wish for. The new tests, as described by the cheerleading experts, could well be more restrictive than the current ones. There's plenty to think about here, most of it not good.
A top researcher for one of the nation's biggest testmaking companies hopes this is the end of an era in standardized testing. Today's pencil and paper multiple-choice tests - largely unchanged in design for decades - will be replaced in a few years by more interactive tests that evaluate student achievement as part of daily lessons, said Randy Bennett, who studies new test-building ideas for the Educational Testing Service, the maker of the SAT college entrance exam and state achievement tests nationwide.
Today's standardized tests are becoming antiquated, he said. "The kinds of questions the tests tend to present don't measure all of the thinking processes that are important to be proficient in a subject area," Bennett said.
Some believe the nation is already crossing a divide to a new generation of tests, using new technology to better evaluate students. "These tests are as a whole mediocre. But they're getting better. They're getting more aligned with standards," said John Katzman, founder of the Princeton Review, a New York-based test preparation company. U.S. Rep.
John Boehner, R-West Chester, co-sponsored the No Child Left Behind act and is one of its most vigorous defenders. He argues that testing has helped make student achievement the primary goal in education. "The simple fact is that as a result of No Child Left Behind, the public education system in focusing on disadvantaged children like never before," Boehner said on the law's two-year anniversary earlier this year.
"Overall, the law is working very much as envisioned. There has been predictable grumbling by the education establishment as it has gradually realized the Bush administration has no intention of watering down the law ... . But virtually no one has suggested we should return to the days in which achievement gaps were subsidized and hidden from public view."
And yet, critics complain that standardized tests are not up to the task of answering tough questions like whether third graders can read or if a high school senior is ready to graduate.
"Today's tests contain many short and elemental items that test a small skill," Bennett said. "The implication is that proficiency in a subject is largely a matter of how much you know. If we've learned anything from 30 years of research in cognitive science, it's that the amount of knowledge is important, but also important is how it's organized. If it's not organized to be efficiently used, a person is not going to be effective in a subject."
Today's standardized math tests are a case in point. Multiple-choice questions can test how much a student knows about math, but it won't necessarily show how well that student can apply those concepts to real situations. Consider an example of two ways to test knowledge of a subject. Suppose a traveler needed a taxi ride from Dayton to Cincinnati and gave two drivers a standardized test to decide who would get the job. A traditional test might ask both drivers to put three routes in order of fastest to slowest. But a second test instead could ask each driver to name the alternative routes they might use if there were traffic jams along the way. Even though both drivers know the same three routes, driver B can spell out more clearly when to use each route.
Critics of today's tests said the exams are too much like the first test the taxi drivers took and not enough like the second. As a result, students' scores may not tell the whole story of their proficiency in the subject.
And when these tests are used for high-stakes decisions - such as whether a student should graduate from high school or advance to the next grade level - the need for good assessment is paramount. "You're going to need good tools," said Thomas Lasley II, dean of the University of Dayton's School of Education and Allied Professions. "As the tests advance, the good techniques will drive out bad techniques."
Old school testing habits die hard. States like Ohio are attempting to move away from nationally-normed tests in favor of "criterion-based" tests that are tied directly to standards in each subject. But so far the new tests have many of the old problems, said Columbia University researcher Clifford Hill, who co-wrote Children and Reading Tests, a book about the flaws of elementary school reading tests . "It's old wine in new bottles." he said. "We were told there would be new kinds of testing, but they are doing things no differently."
Criterion-based tests should be filled with questions that measure a specific skill spelled out in the state's list of what they want kids to know at that grade, Hill said. But the new tests he has seen largely use the same techniques of nationally-normed tests, which are designed to compare kids to each other, he said. These tests seek to trip kids up so student scores will be spread out, not bunched up. The criterion-based tests are not for comparisons, but are simply supposed to measure if students have learned the standards.
Too often, Hill said, scores don't reflect students' true proficiency as a result. "We were kind of sold a bill of goods with the idea these new tests would be really, really different," he said. "But when you go to reading comprehension, its pretty much the same old stuff going on. It hasn't changed much."
Katzman of the Princeton Review said it may take time for question writers to adapt. "Old habits die hard for people who write test items, even in a criterion-based world," he said.
New tests, using today's technology like computers and the Internet, might provide more avenues for testing different types of student understanding of subjects such as reading and math.
Bennett envisions a future in which kids do assignments on computer, and classroom tasks include testing as part of a lesson. "Why test a kid once for three hours when you could have info that builds up cumulatively across the year?" Bennett said. "The trick is how to integrate this information in a fair and accurate way - to combine it all into a meaningful judgment about that student's level of knowledge and skill. It doesn't mean we need to be doing testing all the time, but we can be doing it more frequently and doing it through the kinds of tasks students should be doing anyway."
Testing overhaul coming, experts say With the explosion of new tests, Katzman said the day is not far off when testing programs will effectively evaluate students based on what they've learned in individual classes. "I think we'll see the end of the SAT," he said. "There will be an evolution from one-size-fits-all testing. States will offer options, say 10 math curriculums. Schools can choose one and when they buy the curriculum, the testing component comes with it."
He said colleges would rely on those tests, along with other measures, to make judgments about student proficiency, rather than relying on a national, standardized test like the SAT admissions exam.
Ironically, Ohio was moving toward just such a program before NCLB, the federal No Child Left Behind act, became law in 2002. A commission appointed by Gov. Bob Taft recommended the state consider requiring high schoolers to pass "end-of-course" exams tied to the classes they took during their high school career as an alternative to a single graduation test. But when that program did not fit the testing requirements of the No Child Left Behind act, Ohio scrapped it. "NCLB is martial law," a disappointed Katzman said. "States that were doing innovative stuff took a step backwards when NCLB came along."
Mark Fisher and Scott Elliott
Dayton Daily News
INDEX OF NCLB OUTRAGES