Orwell Award Announcement SusanOhanian.Org Home

Errors in Standardized Tests: A Systemic Problem

Susan Notes: Don't miss this report.

Here are a couple of snippets from a provocative report from the National Board on Educational Testing and Public Policy.
You can access the report at the url below.

Finding Active Error: Reporting Error to the Testing Contractors

In 1999, we began a systematic search for examples of testing errors. Over this three-year
search, we found dozens of errors that were discovered by school officials, teachers, parents, and even students. Testing companies had not discovered most of them. Summaries of errors not discovered by testing contractors are contained in Appendix A. We describe below errors associated with three major publishers: CTB McGraw Hill, National Computer Systems, and
Harcourt Brace. . . .

Moreover, officials from California's DOE reported very little state oversight of the testing program. Gerry Shelton, an official with California's DOE, had lobbied for an $800,000 office that would verify the accuracy of test results. Shelton cautioned,"The department doesn't have any responsibility--because the publisher is running the program. This specific problem could have been "prevented with an hour of time from one of my staff members (Colvin, 1999a, p.2)."
An unnamed source from Harcourt Brace echoed Mr. Shelton: "There is no one in charge. This
thing is out of control.You've got the California Board of Education, competing voices in the state Department of Education, the governor--and 1,100 separate contracts"(Asimov, 1999a, p. 1). A top official from Harcourt Brace warned,"We can't check the results for each of the 1,100 districts. It's invaluable for folks at the local level--to check to see if there's anything that looks suspicious"(quoted in Colvin, 1999b, p. A3).

Despite the scoring problems of 1999, California Governor Gray Davis signed a bill in
October of that year awarding a single state contract to a testing company. The bill called for rewards or penalties for schools based on a list of criteria (test scores prominent among them)and opened the door for school takeovers by the state, retention in grade, and denial of
diplomas on the basis of test scores (Asimov, 1999b). Further, the bill mandated that test
results be published by June 30. These provisions incorporated latent error and so increase the probability of active errors in the future. Indeed, Harcourt officials attributed errors made in 2001 to the cramped time table for reporting results and indicated that in the future they would rather be penalized than send out results that went unchecked (see Appendix C, #13)(Groves & Smith, 2001).

— Kathleen Rhoades and George Madaus
Errors in Standardized Tests: A Systemic Problem



This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available in our efforts to advance understanding of education issues vital to a democracy. We believe this constitutes a 'fair use' of any such copyrighted material as provided for in section 107 of the US Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit to those who have expressed a prior interest in receiving the included information for research and educational purposes. For more information click here. If you wish to use copyrighted material from this site for purposes of your own that go beyond 'fair use', you must obtain permission from the copyright owner.