Orwell Award Announcement SusanOhanian.Org Home

NCLB Outrages

Errors may snarl Illinois testing: Officials hire auditor, worry state testscores may show failure of system

Ohanian Comment: The reader is told, "If problems are not figured out—and corrected—by fall, parents and schools won't have an accurate measure of how well children performed on the state's high-stakes achievement exams." As though an "accurate measure" of children's performance ever results from standardized tests.

The article provides the Standardisto party line along with the pro forma caveat from FairTest.

The article insists "This is the first time that the grade school test itself has been called into question." Maybe in Illinois. But maybe the person who cares about public schools would be interested in Harcourt's track record elsewhere. The Chicago Tribune reveals no hint of problems.

The concerned consumer would have to go to a (slow-loading) sample test (pdf file) to find out that Harcourt insists a third grader must look at footnotes in order to answer the multiple-choice questions. A fourth grader has many more footnotes to worry about and needs to know about the literary technique of flashback. Of course, these items are just samples. Actual tests are kept under absolute secrecy. Not even teachers responsible for students' welfare are allowed to examine them.

If one wants to buy a car or a refrigerator or cocoa mulch, one can find out how this product has been rated. But we continue to allow a corporate conglomerate to rate the nation's children with a secret test. Just for starters, people in Illinois might want to check with Harcourt's record in Nevada, Georgia, and California.

By Stephanie Banchero

Wild swings in the scores on this year's elementary school math and reading exams raise new questions about the reliability of the state testing system used to rate schools, apply federal penalties, and even put some kids in summer school.

Now an outside auditor has been brought in to investigate why test results fluctuated up to 10 percentage points from those of the previous year. If problems are not figured out—and corrected—by fall, parents and schools won't have an accurate measure of how well children performed on the state's high-stakes achievement exams.

The inexplicable shifts on the Illinois Standards Achievement Test (ISAT) administered this past spring were found in districts throughout the state and spread across student race and income levels. The most dramatic swings came in reading at the 3rd, 4th, 6th and 8th grades.

State education officials would not give specific details about the size of the fluctuations at each grade level for each exam. But Matt Vanover, spokesman with the Illinois State Board of Education, said some scores increased dramatically while others decreased significantly.

"Typically, you expect to see steady growth or scores holding steady," he said. "We observed changes that are outside of what you would expect to see."

The outside auditor is looking into possible problems with the ISAT construction or possible errors in scoring the exam.

School districts have reviewed preliminary test scores. The state plans to get final results to parents by Nov. 1.

Illinois has been plagued with testing problems since 2002, when No Child Left Behind required states to ramp up testing. In every year except 2005, testing data has been riddled with errors, seriously delayed or contained sizable shifts.

Last year, the high school reading scores dropped about 5 percentage points, the lowest pass rate ever recorded, but an outside audit could not determine the cause of the precipitous decline.

This is the first time that the grade school test itself has been called into question. The state paid Harcourt about $3.2 million this year to create and score the exam.

The state's testing problems worry some local school officials.

"Despite the problems, I've had faith in the testing system—until this year," said Carole Cooper, director of assessment and accountability for Carpentersville School District 300, where reading scores dipped in 26 of 27 schools after increasing in previous years. "I don't think this problem will be worked out until two to three years from now, and by then, we will have moved on to another assessment."

Since the federal law was enacted, many states have had difficulty delivering timely and reliable test results. Meanwhile, they are using the results to sanction schools and in some cases make children repeat a grade.

"This is yet another example of why critics warn against the overuse of standardized tests for high-stakes decisions, like student retention, school ratings and teacher firings," said Bob Schaeffer, with FairTest, a non-profit group that's been critical of the overuse of standardized tests. "These exams are not the absolute, accurate measure politicians say they are."

In Illinois, the state is paying the Center for Educational Testing and Evaluation about $20,000 to investigate this year's ISAT problems. The Kansas-based company is looking into, among other things, whether the construction of the exam was flawed.

Each year the state uses a slightly different version of the ISAT, but creating the exam is a complicated process.

The state keeps a bank of test questions from each year. Every question is tested to make sure it is not flawed. Then, questions are given weights based on the degree of difficulty. Tougher questions are worth more.

Researchers are looking into whether this year's weighting process was flawed. If it was, it is possible to recalibrate and rescore the exam, state officials said.

Some scoring declines were expected.

For the first time this year, the state required students who speak English as a second language to take the regular exams instead of the easier test they used to take.

In Barrington Community Unit School District 220, the reading score for that small group of students plummeted from 71 percent passing last year to 44 percent this year, said Jeff Arnett, district spokesman."We've said all along that we were concerned that these students might have a tough time with this test," Arnett said. "We want to work with the state to arrive at a solution, but we are very concerned about these results."

Vanover said the fluctuations were not isolated to non-English speaking students and there were too few of them to sway the scores.

— Stephanie Banchero
Chicago Tribune


This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available in our efforts to advance understanding of education issues vital to a democracy. We believe this constitutes a 'fair use' of any such copyrighted material as provided for in section 107 of the US Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit to those who have expressed a prior interest in receiving the included information for research and educational purposes. For more information click here. If you wish to use copyrighted material from this site for purposes of your own that go beyond 'fair use', you must obtain permission from the copyright owner.