Test Failed; Students Didn't, State Says
Oregon will scrap results of this year's math problem-solving test for high schoolers because the Oregon Department of Education inadvertently made the test too hard, officials said Thursday.
The main proof the test was too difficult, they said: Roughly 80 percent of sophomores failed, compared to about 50 percent in prior years.
The upshot for students who took this year's test: Those who failed, some 33,000 students, will have to retake a problem-solving test as juniors.
For schools, the state's decision is a big relief. They won't be downgraded on federal performance standards based on the steep drop in their students' problem-solving scores. Under the federal No Child Left Behind law, Oregon schools have to get 39 percent of their students to pass math tests or they are tagged as low performers.
For this year only, schools' performances at teaching math will be judged solely on the state multiple-choice math test. The passing rate for sophomores this year on that test was consistent with past years', state officials said.
School-by-school results on that test, plus the state reading test, will be released in early August, state officials said. The list of low-performing schools will follow in mid-August.
Results on the problem-solving test, which requires students to solve one complex math problem and show their work, won't count on the school report cards.
Oregon Superintendent Susan Castillo made that call this week, and the U.S. Department of Education gave its approval.
"It's not that our kids are dumber, and it's not that our teachers didn't do the job they needed to do. It's the test. The difficulty of these (test questions) was not in line with what we'd done before," Castillo said. "We want to maintain the highest possible credibility of our testing system. And we want to make sure we are doing the fairest thing for kids and schools."
State testing officials say they didn't adequately evaluate the difficulty of this year's problem-solving questions before giving the test because they were trying to save money.
In hindsight, they said, that was a mistake. For an additional $40,000, officials could have tested another batch of math problems to pick questions that more accurately matched past tests. Instead, they spent about $350,000 to create, administer and score a test whose results they cannot use.
Normally, said Cathy Brown, the state's math testing specialist, Oregon has tried out up to two dozen potential questions each year on a broad sampling of sophomores. Schools volunteer to have their students in the trial. The state then picks the best questions to use on real tests, based on which questions most closely match the difficulty levels of previous tests.
But the Legislature cut the state testing budget in 2001, so the state skipped a year of test trials. For the 2004 test, state officials chose questions they'd tested in 2002 but opted not to use in 2003.
Results from the trial tests led state officials to think the questions they put on the 2004 test would be a smidgen more difficult than previous tests. In reality, they proved to be more than twice as hard.
To avoid repeating that mistake, the state now will test potential problem-solving questions on a larger, more representative sample of students by requiring schools to take part in the trials, Brown said.
The state weighed, but rejected, the idea of permanently judging schools solely by multiple-choice tests, not by the open-ended problem-solving test as well, said Doug Kosty, assistant superintendent over testing.
But educators and business officials urged the state not to diminish the importance of the problem-solving test, he said. Teaching students to pass the open-ended test leads to discussion about many ways to solve the same problem and how to communicate with others about math, he said. "The message we received is that this is a valuable instructional tool and you need to keep it in," Kosty said.
Added Brown: "The business community was particularly adamant that they want to know, 'Can the students apply the mathematics they are learning?' "
Each time the high school problem-solving test is given, it works the same way: Students are given a choice of three questions, one centered on probability and statistics, one on geometry and one on algebra.
Students choose one of the questions and take as long as they wish to solve it. More points ride on how well they explain their approach and show their work than on whether they get the right answer.
The tests are graded by Oregon math teachers who are trained to look for the five basic traits -- conceptual understanding, strategy, accuracy, verification and communication. Each test is graded by two teachers, and if they don't agree, a third teacher also grades it.
Fifth- and eighth-graders also took problem-solving tests this year. But because those tests were canceled last year, also to save money, the best questions from the 2002 trial tests were still available. Results from those grades appear sound, and elementary and middle schools will be graded on those results this year, officials said.
Betsy Hammond: 503-294-7623; firstname.lastname@example.org
INDEX OF NCLB OUTRAGES