Orwell Award Announcement SusanOhanian.Org Home

NCLB Outrages

Education tests: Who's minding the scores?

A poll shows less than half of states look for cheating.

By Kellie Patrick and Larry Eichel

In New Jersey, education officials didn't notice that standardized test scores from three Camden schools had risen in 2005 at a seemingly inexplicable rate - and thus might be worth investigating - until prodded by reporters.

Had the same sort of test results been recorded in other parts of the country, there's no guarantee they would have drawn the attention of school officials either.

Even with all the money and prestige that's riding on standardized test results in this era of the federal No Child Left Behind law, only about half the states responding to an Inquirer survey do any sort of statistical analysis of scores to identify signs of organized cheating.

"We assume that people are not cheating, unless we see a red flag," said Mary Jane Michalak, a spokeswoman for the Indiana Department of Education, which looks for dramatic test-score changes.

And there is no federal requirement that education departments adopt a more proactive approach.

"We would expect that states would monitor, make sure that people are administering tests appropriately, accurately, honestly," said Henry Johnson, the U.S. assistant secretary for the Office of Elementary and Secondary Education.

No one keeps track of the extent of cheating on standardized tests in this country, whether orchestrated from the top or done by individual students. Experts rely on media reports, and those reports, they say, suggest that irregularities are on the rise - although perhaps not rampant.

"Where there's smoke, there's fire, and we're certainly seeing a lot of smoke," said Thomas Haladyna, an expert on standardized testing who teaches at Arizona State University. "As long as we put a lot of reliance on single tests the way we do, we're asking for this kind of contamination in the results."

The Inquirer survey, to which 30 states responded, found that some states leave the policing to individual districts. Many states do little checking in the absence of specific complaints.

But that's not to say officials aren't worried.

A separate study, done by a test-security company, found that 80 percent of education-assessment directors are spending more time dealing with security issues than they did five years ago. This survey, which got responses from 34 states, was done by Caveon Test Security, a Utah-based firm.

In New Jersey, the Education Department does not routinely analyze results to look for irregularities even though there is a 10-month lag between the time the tests are given and their scores are made public.

The Inquirer first raised questions about unusually high 2005 test scores in Camden in February and triggered an ongoing state investigation. The state has not yet said whether cheating did occur.

New Jersey Department of Education spokesman Jon Zlock said state officials were considering several options, but could not say whether the state planned to hire a test-security firm or routinely analyze test scores.

"Test security is something we take very seriously," Zlock said Friday. "We provide training for school districts on how to keep their tests secure, and we investigate any reports of any breach as soon as possible. We'll continue to strengthen the process as we move forward."

A number of states, including Illinois, South Carolina and Pennsylvania, rely on the firms contracted to write and grade the tests to look for evidence of foul play.

Officials in states that don't routinely review the data - among them Alabama, Connecticut, Massachusetts, Missouri, Montana, Nebraska, Tennessee, Utah and Virginia - say that they haven't seen evidence to suggest that cheating is widespread in their areas.

Nor do they feel the need to devote their limited financial resources to solving a problem they're not sure exists.

"If you do spend on security, are you really going to stop cheating?" asked Robert Schaeffer of the National Center for Fair and Open Testing.

A few states, including some that have been burned by cheating scandals, are proactive.

Nevada has created a security position. Louisiana and Mississippi have laws requiring procedures to ferret out irregularities. California officials carry out checks, including looking for excessive erasures.

Texas, Ohio and North Carolina have hired consultants to conduct statistical analyses looking for unusual year-to-year gains, high numbers of erasures, and students who get easier questions wrong and harder ones right.

The firm getting much of such work is Caveon, which offers both security audits of a state's entire testing process and what it calls "forensic analysis" of the results, a service designed to find irregularities.

"The objective here is to eliminate cheating, not to arrest and string up cheaters," said John Fremer, president of Caveon. "We're not out to injure people. We're out to get them pointed to being honorable and fair in test-taking behavior."

There certainly is cause for concern.

In Camden, state officials are investigating dramatic rises in 2005 test scores at H.B. Wilson and U.S. Wiggins Schools and Dr. Charles E. Brimm Medical Arts High, where the recently fired principal has said he refused a supervisor's pressure to change scores.

At H.B. Wilson, 100 percent of fourth graders passed the math test last year, but only 23.2 percent this year. At U.S. Wiggins, 96.6 percent of fourth graders passed the language-arts test last year and 55 percent this year. Scores dropped 17 percentage points to 74.6 at Brimm.

In Texas, a study by Caveon, obtained by the Dallas Morning News, found "statistical inconsistencies" in 609 of the state's 7,112 public schools on the 2005 Texas Assessment of Knowledge and Skills test, the model for No Child Left Behind. Those inconsistencies by themselves are not proof of cheating.

In Georgia, the Atlanta Journal-Constitution reported last year that about 160 educators had been sanctioned in the previous six years for bending the rules or cheating on standardized tests.

One reason for the uncertainty over the scope of the problem is that it's difficult to prove cheating occurred, unless someone is caught in the act. And disciplinary action, if it occurs, almost always comes privately and slowly.

Consider the case in Pennsylvania of suspected cheating between 2001 and 2005 at two middle schools in the Chester Upland district. Jayne Gibbs, a principal at Parry Middle School there, was fired for allegedly changing test responses in 2005, district officials said.

A year after Gibbs' dismissal, state education officials refuse to say whether cheating took place, explaining that they could go to jail for commenting on the investigation or even confirming it under the state's Public School Code.

If Pennsylvania's Professional Standards and Practices Commission decides that cheating didn't take place, or that it can't be sure, or that no punishment is warranted, the state won't ever be able to talk about it.

And this is a case in which having higher test scores actually has a negative financial impact on the district. With lower scores, the schools would likely qualify for additional aid as "failing" institutions.

Even in states without such gag orders, officials have trouble keeping track. A few years ago in South Carolina, which has been aggressive in test security, a teacher made copies of test questions and distributed them to students. And the fate of that teacher?

"There was some legal action, but it is unclear what happened," said Theresa Siskind, director of South Carolina's office of assessment. "We don't always know."

Some experts on testing say that secrecy isn't sound policy.

"When you make it public, it tells everyone that you're serious about wanting to prevent cheating," said Gregory J. Cizek, a professor of educational measurement at the University of North Carolina.

Experts say there are ways to make misbehavior more difficult: limiting the time test documents are in the schools and limiting access to them; security chiefs in each district and school; security training for teachers and administrators, and not allowing teachers to monitor their own students.

As long as test scores continue to be the benchmark for judging districts, schools and teachers, a powerful incentive to cheat will exist, experts say, as will the need to counteract it.

"In these matters, the defense can never move as fast as the offense," said Schaeffer of the National Center for Fair and Open Testing.
Contact staff writer Kellie Patrick at 215-702-7807 or kpatrick@phillynews.com. Staff writers Dan Hardy and Melanie Burney contributed to this article.

— Kellie Patrick and Larry Eichel
Philadelphia Inquirer


This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available in our efforts to advance understanding of education issues vital to a democracy. We believe this constitutes a 'fair use' of any such copyrighted material as provided for in section 107 of the US Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit to those who have expressed a prior interest in receiving the included information for research and educational purposes. For more information click here. If you wish to use copyrighted material from this site for purposes of your own that go beyond 'fair use', you must obtain permission from the copyright owner.