Wisconsin Gives Schools Extra Leeway
Despite increasingly tough standards, the number of Wisconsin schools that will be flagged this year for failing to meet federally mandated reading and math goals will be less than half what it was last year - 51 as opposed to 108 - but not because things are getting better.
Rather, it is the state's controversial calculation method that allows schools to miss the goals by substantial percentages without having it count against them.
For the same reason, only one school district in the state will be flagged for failing to meet the federally mandated standards, whereas last year 30 school districts were listed as failing to make enough progress.
The dramatic shift is due to the use of a statistical tool known as confidence intervals.
In short, confidence intervals represent a range around a given percentage. Think of it as the plus or minus factor often mentioned in polls and surveys because of the limited reliability of data based on points in time.
State education officials and proponents say using confidence intervals represents a fairer way to do business because they take into account the variability of test scores and give schools the benefit of the doubt when it comes to determining how well they are performing on standardized tests.
Critics say confidence intervals can be used to evade accountability called for under No Child Left Behind, the federal law that calls for all students, including those in various subgroups, such as minorities and the poor, to eventually be proficient in various subjects. The use of confidence intervals means schools are less likely to be identified as needing improvement because of the poor performance of a subgroup.
Lawrence A. Uzzell, a No Child Left Behind critic and former U.S. Department of Education researcher who served under President Ronald Reagan, stopped short of saying Wisconsin was trying to game the system through the use of confidence intervals.
"I would just say that citizens should take a very close and skeptical look at the way Wisconsin and other states are using confidence intervals," Uzzell said. "It's very easy to misuse them."
Richard Innes, education analyst at the Bluegrass Institute, a policy analysis group in Kentucky, where that state's use of confidence intervals has drawn criticism, was less restrained in his remarks.
"I'd say they're pulling the same scam we are in Kentucky," Innes said, referring to how Wisconsin - like Kentucky - is insisting on a such high degree of test score confidence that many schools that would have otherwise failed to make "adequate yearly progress" will be counted among those which did.
While proponents of confidence intervals argue that test scores don't always reflect how poorly a school is performing, Innes says they don't always show how well a school is doing, either.
"My personal feeling - we ought to throw (confidence intervals) out and just take the scores as they are published, because there's error going both ways," Innes says of the variability of test scores.
State education officials say they are simply trying not to "overidentify" schools that are failing to meet the federal benchmarks. By insisting on a high degree of confidence in the test scores, says Joe Donovan, spokesman for the state Department of Public Instruction, the state can target its limited resources on schools that are truly in need of assistance.
Confidence intervals will be particularly beneficial for small schools, where a few students can throw off the entire score, Donovan says.
"We have to ensure that we are as accurate as we can be," Donovan said. "That's the reason we're using a 99% confidence interval as opposed to a 95% confidence interval."
There are two ways of looking at the situation, says Eric Key, an associate professor of mathematics at the University of Wisconsin-Milwaukee.
"The charitable way to view this is to say they chose 99% to make sure that anybody who they said was bad, really, really is bad," Key says. "The uncharitable way to view this is to say they chose 99% so they would have to say as few people are bad as possible."
In Wisconsin, without the confidence intervals, any school or district where 47.5% of the students were not proficient in math would have been flagged as being in need of improvement. The same would have been true for any school or district where 67.5% of the students were not proficient in reading.
However, this year, by using confidence intervals, depending on the size of the group being tested, the range of acceptable test score averages increases dramatically.For a group of 40, the percentage of students proficient in reading can range from the upper 40s to the lower 80s, and it will still be as if the group met the 67.5% goal, according to data from the state Department of Public Instruction.
In math, the percentage of students who are proficient can be in the upper 20s to high 60s for a group of 40, and it will still be as if the group met the 47.5% goal, the data show.
The range decreases as the size of the group being tested increases but still gives schools and districts a lot of leeway, critics say.
Jack Jennings, founder and president of the Center on Education Policy in Washington, D.C., says the use of confidence intervals represents a statistically valid tool but that the range of scores deemed acceptable in Wisconsin for the purposes of meeting federal standards is on the high end.
"Confidence intervals are supposed to give you a swing, but it's supposed to be a reasonable swing around a number," Jennings says. "That sounds like a very wide swing," he said of Wisconsin's range of acceptable test score averages for a group of 40, which Jennings calls a fairly large group that doesn't need as wide a confidence interval as Wisconsin provides.
But Wisconsin is not alone in using confidence intervals, nor is it alone in using the 99% confidence intervals. Sixteen other states use 99% confidence intervals, according to data compiled by the DPI.
Fifteen use 95% confidence intervals; a few use 98% intervals, and a handful use a variation.
Jay Beder, a statistician and associate math professor at the University of Wisconsin-Milwaukee, says Wisconsin's use of confidence intervals appears to be reasonable, given the consequences of being flagged as a school failing to make progress.
"I'd rather see them be cautious," Beder said. "The consequence to a school is tremendous.
"So a school that is not performing well, beyond a reasonable doubt, should be examined carefully. But schools where there's some reasonable doubt need to be given some leeway."
INDEX OF NCLB OUTRAGES