School test data tardier than ever
Ohanian Comment:You want to know how the kids are doing? Ask a teacher.
These glitches with testing companies are just tips of the proverbial iceberg. Expect real trouble ahead.
By Stephanie Banchero
A series of computer glitches, printing mistakes, labeling blunders and human error by school and testing officials have conspired to seriously delay the state's 2006 school report cards, making Illinois one of only two states that have yet to get student test results to children, parents and schools.
State education officials don't expect that data will be made public until January, the latest it has ever been returned. By state law, the report cards are supposed to be made public Nov. 1.
As a result, Illinois public elementary pupils and high school students still do not know how they did on state-mandated math, reading, science and social science exams they took last spring.
State education officials also have yet to determine which schools ran afoul of the federal No Child Left Behind law, meaning thousands of children who might have been eligible to transfer out of failing schools or receive free tutoring under the federal reform have been denied that opportunity this year.
"This is a very difficult situation for schools, students and parents, and we're working hard to get final reports and results out to schools as quickly as possible," said Becky Watts, chief of staff at the Illinois State Board of Education.
For months, the testing company that administered the exams, Harcourt Assessment, has been blamed for the delays. But state education officials acknowledged this week that problems with the state's new student identification system played a major role in the delay.
District officials statewide entered incorrect or incomplete demographic information on more than 11,000 students, forcing Harcourt to spend months calling hundreds of schools to verify the data.
The debacle, one of the worst cases of testing errors in the country this year, highlights a growing national problem. In the last few years, the nation's testing companies--overburdened by the demands of No Child Left Behind--have made a series of significant scoring errors.
Earlier this year, the College Board revealed that it had mistakenly reported lower scores for 4,000 students and higher scores for 600 students who took the SAT exams. A few months later, Harcourt revealed that it had reported incorrect state test scores for 355 Connecticut high school students.
This year, more than a dozen states were late getting report card data to parents, in large part due to scoring problems. Illinois and Montana are the only two that have yet to make their results public, according to federal education officials.
Many researchers and policy makers argue that the testing industry is underprepared and overworked and cannot handle the estimated 40 million standardized exams given to the nation's public school students. Five testing companies create and score most of the exams.
`The tip of the iceberg'
"This is only the tip of the iceberg," said George Madaus, professor emeritus of education at Boston College, who has researched the testing industry and co-wrote a report on errors in standardized tests. "The volume of testing is really catching up with the industry, and I don't think there is sufficient capacity and sufficient talent to handle the volume. Given the time constraints and the high-stakes nature of these exams, I think we can expect trouble ahead."
Administering and scoring student-achievement exams might seem a simple process, but it involves dozens of steps, and mistakes can be made anywhere along the way.
In Illinois, blunders were made from start to finish.
First, Illinois hired Harcourt two years ago to create new exams for elementary pupils. In the midst of that, the state tried to develop a student identification database that would assign students a number that included vital school and demographic information.
Local school officials were supposed to enter information on approximately 900,000 students, and that information was supposed to be printed on labels affixed to the proper tests.
But school officials made about 11,000 errors when entering that data, a disastrous occurrence that lay hidden for months. Complicating the problem, test development took longer than expected, and Harcourt shipped exams to schools later than usual.
When the tests arrived at schools in March, they were rife with errors. Some test booklets had missing pages; others had duplicate pages. Some schools received empty boxes that should have been packed with exams.
About one-fourth of the state's 896 districts were affected.
Harcourt was forced to charter planes to get tests to districts on time.
"We went to some pretty extreme measures to try to respond to the problems," said Russell Schweiss, director of public relations for Harcourt Assessment. "We wanted to do everything in our power to get it right."
An avalanche of errors
Student testing went off without a hitch, but once Harcourt began scoring the exams, an avalanche of errors followed.
When state officials got the preliminary data in June, they noticed problems. After some digging, Harcourt found a scoring glitch. If a student left an answer blank, the scoring machine moved up all the subsequent answers. If the student left question No. 6 blank, for example, the machine automatically moved answer No. 7 into its place, and answer No. 8 into the space for answer No. 7, and so on, Watts said.
More than 17,000 exams had to be rescored, further delaying the process.
School officials were supposed to receive preliminary data on June 1 but did not get it until mid- or late July, Watts said.
In the meantime, Harcourt and state board officials found that 11,000 students had missing data, including school, race or low-income status.
This information is vital to No Child Left Behind. Under the law, test scores must be broken out by race, income level and special-education status of students. If any of those subgroups fails to meet the federal standard, the school lands on the failing list.
Harcourt was forced to hire people to place hundreds of calls to schools to get the information.
Even now, there are 700 exams that the state cannot match with a student, Watts said.
State education officials expect that schools will receive the finalized school reports in December, and that they will be released to the public in January.
INDEX OF NCLB OUTRAGES