Are Schools Passing or Failing? Now There's a Third Choice ... Both
Ohanian Comment; There's another possibility: NAEP may be completely out-of-whack. See my article in the November Substance.
By Michael Winerip
OUR leaders in Washington and the state capitals have not trusted teachers, principals and superintendents to grade and assess their own students rigorously. And so, over the last decade, politicians have enacted many new testing and rating systems - most notably the federal No Child Left Behind Law in 2002 - to ensure that there is an accurate and scientific measure of how students and schools perform.
No more touchy-feely glop. The new age of precision testing has arrived. Thanks to No Child Left Behind, all children must take a state test every year beginning in third grade, and many schools spend much of the year prepping for it. We now have federal and state tests, as well as federal and state rating systems to measure performance precisely.
Unfortunately, it may be that the more we test and the more rating we do, the less we know.
For example, two weeks ago, the latest results of the National Assessment of Educational Progress (NAEP, pronounced nape) were released. It is a federal reading and math exam given to hundreds of thousands of students every few years, and is often called the gold standard of testing.
Sounds definitive, except in most cases it's hard to know what to make of the results, since big discrepancies exist between the scores on this federal test and the annual state tests.
Take Florida, where 30 percent of fourth graders were proficient in reading on this federal test in 2005. Yet on the Florida state test, 71 percent of fourth graders were proficient in reading in 2005. It's a big difference: Are nearly three-quarters of your fourth graders proficient? Or less than a third? And it's typical.
On the 2005 federal test, 33 percent of New York's fourth graders were proficient in reading; on New York's 2005 state test, 70 percent were. In Tennessee, 27 percent of fourth graders were proficient in reading on the federal test; 87.9 percent on the state test.
Nationwide, millions of students may or may not be proficient, depending on which test you favor.
What's more, basic trends on the two sets of tests are often contradictory. In Florida, the federal fourth-grade reading proficiency scores were down two percentage points between 2003 and 2005 (bad news); on the state test they were up 11 points (good news). In New York, on the federal test, fourth-grade reading proficiency was down one point; on the state test, up six points. In short, it's hard to answer the age-old question: Are fourth graders getting smarter or dumber?
"It's a problem," said Wayne Blanton, executive director of the Florida School Boards Association. "It's a case of trying to compare apples and elephants."
Federal officials don't see it that way. "To us, more information is better," said Tom Luce, an assistant secretary in the federal Department of Education. "People say, 'Well, it's confusing.' But I think the American people can deal with two different pieces of information at once."
Mr. Luce said that when residents in states like New York, Tennessee and Florida see such big discrepancies, "they're going to ask questions."
He added, "That's why the NAEP test is there, to shed light."
Right now, the light's looking dim; there are only guesses about what's really going on. Some, like Chester E. Finn Jr., president of the conservative Thomas B. Fordham Foundation, say the states are making their tests too easy, to ensure they get high marks on the No Child Left Behind rating system. Some say the federal test's proficiency level is set too high. And Robert Schaeffer, public education director of the liberal group FairTest, says, "It shows these so-called objective measures are arbitrary, easily manipulated and profoundly political."
There are also major discrepancies between the state and federal systems used to rate schools. Under Florida's state report card system, 66 percent of schools got an A or B in 2005. Under the federal rating system, 64 percent of Florida schools failed to make adequate progress in 2005. So, about two-thirds of Florida schools were succeeding and two-thirds failing at the same time; 825 schools that got an A or B from the state failed under the federal system.
What to do when two highly scientific rating systems produce such skewed results? Fiddle with the numbers. Last spring Florida requested several waivers to ease standards for the federal rating system. Originally, 48 percent of students for 2005 were supposed to be proficient on the state reading test for a school to meet the federal standard. Federal officials agreed to let Florida lower the threshold to 37 percent proficient.
Thanks to this waiver, hundreds of Florida schools, like Lake Alfred Elementary in Polk County, made adequate progress under the federal system for the first time.
Under the federal rating system, the school, as well as each subgroup at the school - whites, blacks, Hispanics, special ed children - must make the 37 percent proficiency. At Lake Alfred, 44 percent of the Hispanic subgroup was proficient in reading. Had there been no waiver, and the cutoff stayed at 48 percent, Lake Alfred would have failed on the federal system.
Lake Alfred also benefited from a statistical quirk. The past two years the school's special ed students failed to make adequate progress. For example, in 2003, only 3 of 34 special ed students were proficient in reading. This year, only 4 of 26 special ed students were proficient, 15 percent, far below the 37 percent proficiency needed. But in Florida, for a subgroup to be counted in the federal rating system, it must have at least 30 students. Because Lake Alfred had 26 this year, the high special ed failure rate didn't count.
IN 2004, because Lake Alfred had failed to make adequate progress under the federal system for two years, students were given letters permitting them to transfer out. This year, students were transferring in to Lake Alfred.
Same school. Same principal. Same teachers.
Eileen Castle has been the principal for 21 years and celebrated with her teachers when they made adequate progress under the federal system. She is proud of the innovative reading and math programs at her high-poverty school, but even so, she knows she made it because of the state waiver and subgroup size exemption - not because Lake Alfred was any better or worse than the previous year.
"It's a numbers game," she said, "and we could have just as easily been labeled failing."
Over the last two weeks, Kim Karesh, spokeswoman for the Tennessee Education Department, has repeatedly been asked by reporters about the big discrepancy between Tennessee state scores and federal scores.
"I've asked these questions myself to federal officials, and the answers don't make a lot of sense," she said. "In education these days, we talk numbers until we're blue in the face. But there's a bigger philosophical question: 'Can you really boil it down to a number?' "
New York Times
INDEX OF NCLB OUTRAGES