Illinois Tosses Out Thousands of Tests
Ohanian Comment: It's hard to see what's going on here. This is definitely a story that needs further investigation.
Illinois tossed out more than 80,000 tests used to determine whether schools met academic standards this year in reading and math, inflating scores at nearly 1,400 schools and causing some to be judged on only a fraction of students tested.
So many tests were thrown out in some Chicago schools that district officials said they are investigating whether mistakes were made.
State data showed that at Tonti Elementary on the Southwest Side, for example, 287 children took the state's reading test last spring, but only 51 scores were used to determine whether the school met federal standards for reading. The rest were tossed out. At Hirsch Metropolitan High School on the South Side, more than 90 percent of reading scores were discarded.
Statewide, nearly 7 percent of test scores were thrown out as state education officials calculated whether schools met the critical math and reading standards required under federal education reforms.
Most of the disqualifications occurred because of a new state policy dictating that if a child was not attending a particular school on or before Sept. 30, that student's score is not counted in judging whether the school met standards. Scores also can be judged "invalid" if a student didn't try to answer enough questions, state education officials said.
The attendance policy is designed to avoid penalizing schools with a high population of "mobile" students--the homeless, migrant or poor children who move around a lot and tend to fall behind in their studies. Federal officials allowed states to set such rules as part of No Child Left Behind, the sweeping law that puts unprecedented pressure on schools to ensure children from all backgrounds pass state tests.
However, local educators said test results were skewed so wildly in some cases that mistakes must have been made in how students were labeled.
Data hurt Downstate school
Scott Tonsor, principal at Staunton High School in Macoupin County, said he was infuriated when state data showed that only 14.3 percent of his students met standards in reading. Tonsor was sure about 46 percent of students actually had passed.
He later learned that only 21 of Staunton's 93 reading tests were used to determine whether the school met standards. One test was judged not valid, but 71 were negated because those students allegedly enrolled after Sept. 30, 2002. Tonsor suspects the state made a mistake.
"We don't try to cover up numbers by lying about when kids enrolled," Tonsor said. "Besides, in our case, it would have hurt us to say all these kids enrolled late. I really don't know what is going on or how the state makes their calculations."
Although scores at Staunton fell, the disqualifications in most cases served to raise scores. Attucks Elementary in Chicago achieved a 40 percent passing rate in reading--a critical figure because a 40 percent rate is required in Illinois to avoid a failing label. This year Attucks was among the Chicago schools recognized with a cash award for improved academic performance.
However, state data showed that about 35 percent of the school's reading scores were discounted. If those scores had been included, its passing rate would have been lower: 35.3 percent.
Overall, discarded test scores helped 23 schools avoid a failing label for overall scores in reading, and 22 schools avoided failing in math, according to a Tribune analysis of testing data requested under the Illinois Freedom of Information Act. The Tribune looked at math and reading test data for about 3,800 grade schools and high schools in the state.
In reading, 1,033 schools increased their passing rates by at least 1 percentage point with the help of disqualified scores, and 999 schools improved by that much in math, the data showed. In all, 1,380 schools had increases of at least 1 percentage point in one or both subjects.
But a handful of schools, such as Tonti, saw more dramatic improvement. The school would have had a 46.3 percent passing rate on reading tests if test scores were not discounted. But tossing out the tests boosted the passing rate to 64.7 percent.
Several local school officials interviewed by the Tribune said they had no idea that the state was discounting test scores.
Wanda Shelton, director of administrative services and instruction for North Chicago Community Unit School District 187 in Lake County, said she was still under the impression that the old rules applied--that all student test scores counted regardless of when children enrolled.
With a large number of military families--the district serves children at the Great Lakes Naval Training Center--the district has a highly mobile student population, Shelton said.
At Hart Elementary School, 66 students took the state reading test last spring, but only 25 scores counted when determining if the school met standards. Most of the scores were discounted because students enrolled after Sept. 30, state data showed. With so many scores thrown out, Hart's passing rate rose to 64 percent from 34.4 percent in reading.
Some defend policy
Education officials from various states have argued that the attendance policy makes sense because schools shouldn't be held accountable for mobile students whom they haven't had a chance to educate.
"If you plop a child down into a school on, say, Oct. 15, that child will have missed all the review time and the beginning of some very important new lesson plans," said Gail Lieberman, who serves as the federal liaison for the Illinois State Board of Education and is the point person on No Child Left Behind. "It may not sound like a lot to an adult, but for a child, those few weeks of school are very, very critical."
Though research has shown that statewide passing rates on achievement tests change very little when mobile children are added to the mix, individual schools can be dramatically affected by high numbers of mobile children, Lieberman said.
Prior to No Child Left Behind, which was enacted early last year, about half the states had attendance rules in place that discounted some test scores, said Celia Sims, special assistant in the U.S. Department of Education's Office of Elementary and Secondary Education.
But states including Massachusetts, California and Illinois changed their rules with the advent of the federal reforms, state and federal records show.
In the past in Illinois, individual schools were held accountable for a student's scores, regardless of when that child enrolled. Now the scores of late-enrolled students are included in district and statewide test results--but individual schools aren't held responsible for them.
In the high-stakes world of testing, some educators and child advocates see a danger in deciding that some children's scores don't count at the school level.
"I'm sure if a school's whole emphasis is on test scores rather than quality of education, then certainly kids who aren't counted wouldn't receive the same high level of attention as would other children," said Rose Marie Lorentzen, executive director of Hesed House in Aurora, which provides food, clothing and shelter for the poor and homeless.
Kathy Christie, a vice president at the Education Commission of the States, a prominent education policy and research organization, said some educators who "truly don't care about kids" may not devote as much attention to mobile children, knowing their scores don't count. "But to a good teacher, it shouldn't matter a whit," she said.
School officials suspect errors
In Glen Ellyn in DuPage County, Abraham Lincoln Elementary School saw 32 percent of its reading scores thrown out, mostly because of late-enrolled students. However, the high-scoring school also reported a low student mobility rate, raising questions about how there could be so many late enrollees.
Supt. Jack Barshinger said he was surprised when he saw how many tests had been discounted. Though he assumes state officials might have made a mistake in the calculations, Barshinger said the district would be more diligent next year when filling out student demographic information.
"It really doesn't matter who made the mistake or whose fault it is. The bottom line is parents deserve for this information to be correct," Barshinger said. "I don't want to be part of a system that sends out bad information or accidentally discounts students' test scores."
In Chicago, officials said they are looking into the schools where a high percentage of tests were not counted. Dan Bugler, the district's chief of research, evaluation and accountability, said he suspects mistakes were made either by local school officials or state officials who crunched the numbers.
Bugler said he first noticed the problem when he saw the low number of scores that counted at Tonti Elementary, 5815 S. Homan Ave., and at Hirsch High School, 7740 S. Ingleside Ave., where only seven of 97 tests counted in determining if Hirsch met standards.
"There has been a lot of confusion, in general, over the report card data and we are not sure exactly what happened with these test scores," Bugler said. "But we are looking into it and talking to principals about whether these students actually came after Oct. 1."
Illinois State Board of Education spokeswoman Karen Craven said schools had several opportunities to correct any errors in data they provided to the state. Local schools or districts are responsible for identifying when children are enrolled.
Because of widespread concern about problems with testing data, 450 schools already have been given the opportunity to correct possible errors.
State officials stressed that late enrollee scores are still reflected in districtwide scores, statewide scores, and some of the individual school results published in School Report Cards available to the public.
But Craven acknowledged the report cards aren't clear on when students are counted and when they are excluded. State officials should consider changing the document, she said, to better reflect how many tests are used to determine if schools met standards.
"It is something that we will look into in the future," Craven said.
- - -
Nearly 7 percent of Illinois tests thrown out
The Illinois State Board of Education disqualified more than 80,000 tests used to determine whether schools met standards in reading and math in 2003.
REASONS FOR DISQUALIFIED TESTS
Invalid tests: All incomplete tests
Late enrollee tests: Taken by students who enrolled on or after Oct. 1, 2002
Total tests: 613,979
Total disqualified: 41,636
Late enrollee tests: 30,638
Invalid tests: 10,998
Total tests: 613,259
Total disqualified: 40,078
Invalid tests: 9,436
Late enrollee tests: 30,642
SCHOOLS WITH MOST DISQUALIFICATIONS
In some schools, the majority of reading and math tests were disqualified. The top ten:
SCHOOL LOCATION TESTS DISQUALIFIED Hirsch Metropolitan High School Chicago 92.8% Grand Prairie Elementary School Centralia 90.0% Lovington High School Lovington 88.0% Tonti Elementary School Chicago 82.5% Staunton High School Staunton 77.4% Dugan Alternative High School Chicago 75.0% Greenview Junior High School Greenview 72.2% Beecher City Grade School Beecher City 72.1% Wilson Elementary School East St. Louis 71.2% Odell Grade School Odell 71.1% Source: Illinois State Board of Education Chicago Tribune
Copyright © 2003, Chicago Tribune
Diane Rado, Stephanie Banchero and Darnell Little
Thousands of exams tossed out by state Move raises scores at 1,400 schools
INDEX OF NCLB OUTRAGES