Connecticut Commissioner Not Sure McGraw-Hill Has Capacity or Commitment To Do Testing Job Right
Ohanian Comment: As the reporter notes, NCLB makes testing errors more high-stakes than ever. Also note that McGraw-Hill seems to be able to correct multiple choice answers--but not essay questions. Don't miss McGraw-Hill's response.
State Finds Errors In Exam Scoring
Mastery Tests To Be Returned To Contractor For Reprocessing
State officials will send thousands of student exams back to a testing company after finding scoring errors that could delay the results of the Connecticut Mastery Test for months.
The state Department of Education suspected a problem when this year's scores from a new testing contractor were considerably lower than those of a year ago.
The stakes are higher than ever because the scores are used to determine whether schools meet standards and avoid sanctions under President Bush's 2002 school reform law, the No Child Left Behind Act. But a delay in reporting results also could be harmful to schools that depend on the scores to adjust lessons or monitor performance of individual classrooms or students.
"Getting results at the end of the year is virtually meaningless," said J.A. Camille Vautour, superintendent of schools in Rocky Hill. "It is, in my mind, perhaps a wasted year of testing."
Officials would only say results, usually reported in January, would be released in the spring. "March? April? May? I don't know," said Tom Murphy, a Department of Education spokesman.
It is only the second time in the 19-year history of the mastery test that the state has rescored tests because of a potential flaw. The test of fourth-, sixth- and eighth-graders was introduced in 1985 and has become the state's chief benchmark of educational progress. In 1992, officials rescored an essay when results appeared to be inaccurate.
In a letter sent Tuesday to public school superintendents, state Education Commissioner Betty J. Sternberg said a new testing contractor, CTB McGraw-Hill, "has experienced a significant number of problems in processing, scoring and producing the 2003 [mastery test] reports."
The state is in the first year of a seven-year, $48 million contract with the California-based company.
In a letter sent Wednesday to CTB McGraw-Hill, Sternberg outlined a series of missed deadlines and other scoring mix-ups. "I am not convinced that your company has the capacity or commitment to meet these challenges," she wrote.
Sternberg said the scoring errors occurred on an essay portion of the test and on reading and mathematics questions that required students to write open-ended answers. There were no apparent problems on the more easily scored multiple choice questions.
Testing experts at the state Department of Education found inconsistently low percentages of students meeting state goals in reading, writing and mathematics - sometimes as much as 20 percentage points below what was expected.
"Obviously, that was a big red flag," said Barbara Q. Beaudin, acting chief of the Bureau of Student Assessment. The department is negotiating a plan with the testing company for a complete rescoring of the exam, which was given to 120,000 children last September.
In addition to being the barometer for meeting the standards of the federal No Child Left Behind Act, the mastery test has been used as a guide for adjusting curriculum, assigning children to summer school remedial classes or placing students in special education classes.
"It's a high-stakes exam for students and schools," Beaudin said. "It's really important to get it right."
Under terms of the contract, the state could impose fines against the company for missing deadlines, but that is still to be determined, Sternberg said. "My first priority is to see that we have accurate data," she said.
Michael Kean, a spokesman for CTB McGraw-Hill, said Connecticut's transition to a new company resulted in inconsistencies in scoring methods but that officials were working to iron out the problem.
He said there have been previous instances of rescoring exams, in some cases when the company's quality control procedures have found anomalies.
Several testing companies, including CTB McGraw-Hill, have made similar mistakes in scoring standardized tests, according to a report last year by the National Board on Educational Testing and Policy based at Boston College.
In 1999, CTB McGraw-Hill discovered a programming error that resulted in incorrect scores for a quarter-million students in Tennessee, Indiana, Wisconsin, Nevada, South Carolina and New York City, the report said.
In New York City, the error resulted in mistakenly sending more than 8,600 children to summer school classes.
In Minnesota, more than 50 students were denied high school diplomas in 2000 because of another testing company's error on the scores of a test required for graduation, the report said.
Robert A. Frahm
State Finds Errors In Exam Scoring
INDEX OF NCLB OUTRAGES