Proficiency Put to the Test
Too bad the reporter didn't interview scholars who would reveal how bogus the NAEP is.
Scenario one: Only one-quarter of Tennessee fourth-graders have passable reading skills.
Scenario two: A whopping 80 percent of Tennessee fourth-graders have passable reading skills.
Both statistics are true.
If you're confused, you should be.
The bad news, that most Tennessee fourth-graders aren't reading well, comes from the federal test called the National Assessment of Educational Progress, dubbed the Nation's Report Card, which is given to a sample of fourth- and eighth-graders nationwide.
The higher score comes from the state's own test, known as the Tennessee Comprehensive Assessment Program.
Both tests assess reading and math. Both assess the same grade levels. But both yield different results.
In a report issued last week, Education Week, a national industry newspaper, juxtaposed the NAEP test scores and the state test stores of all 50 states. In most states, students tended to fare much better on their state reading and math tests than on the uniform federal ones.
The gap between the scores in Tennessee is one of the highest in the country, which raises questions from some about the ease and the scoring of the state's tests.
"We're not saying in showing this comparison necessarily that NAEP is right," Lynn Olson, the executive project editor of the report "Quality Counts 2005," said in a conference call with reporters. "But states really need to look at what their standards are for their students and where they're setting their proficiency bars."
The discrepancy between the federal and state test scores hinges on one word: "proficient." Results from both types of tests pronounce that a certain percentage of students scored "proficient," or passed the exam. But there is no universal definition of proficient.
That means it's possible for 26 percent of Tennessee fourth-graders to be deemed proficient on one test, and 80 percent to be proficient on another. The meaning of proficiency also varies from state to state.
"The gap is particularly large in Tennessee," said Mike Cohen, a former assistant U.S. secretary of education and president of Achieve Inc., a Washington, D.C.-based group that helps states with standards-based reform.
Officials with the Tennessee Department of Education contend that state test scores should be higher than NAEP scores because the local test matches the state's curriculum standards.
"The NAEP is a national test, so you do get into some of the regional differences and the curriculum differences," department spokeswoman Kim Karesh said. "When you see the state scores, our tests are specifically developed for what our teachers are teaching here in Tennessee."
Karesh and Jan Lineberger, Tennessee's NAEP coordinator, said the scores on the two tests should not be compared side by side.
"They're fundamentally two different kinds of tests even though they're measuring the same content," Lineberger said. "What NAEP was initially set up to do was (to be) a research piece to look at general trends nationally. When you're talking about a national assessment, it's not tied to a national curriculum. We don't have a national curriculum."
Cohen doesn't buy the argument that state scores are more legitimate because they align with state standards.
"That response has amounted to saying, 'Everything is fine. Our tests measure what we expect kids to learn. This NAEP test is just a little noise,' " he said. "How do you know the standards you set in Tennessee are adequate?"
The stakes tied to state test scores have been raised since 2002 when President George W. Bush signed the No Child Left Behind Act, which requires all students to be "proficient" by 2013-14. Only state test scores, not NAEP results, matter.
Mike Winstead, coordinator of research and evaluation for Knox County Schools, acknowledged that Tennessee has set a lower bar for achievement than other states.
"The NAEP proficiency bar is a high standard," Winstead said. "It's not perfect. The sample's kind of small and randomly chosen. But looking at that, you can see Tennessee has not set as high a definition. They don't demand the rigor as much as other states."
Put simply, one state might call proficient getting 80 of 100 questions correct; another state could set the passing level at 40 of 100 questions.
Winstead said Tennessee has placed its bar at a "minimum standard."
"If our goal is just to pass the state test," he said, "we're probably shortchanging our kids."
To illustrate the differences across state lines, 26 percent of fourth-graders scored proficient in reading on NAEP in both Tennessee and South Carolina. But 80 percent of Tennessee students passed the state test while only 31 percent of South Carolina students passed their exam.
That does not mean South Carolina students are three times less smart than Tennessee children, said Jim Foster, spokesman for the South Carolina Department of Education. It means South Carolina has high standards.
"But there's been a lot of pressure to lower them," Foster said. "The state superintendent's response has been, we don't need to lower our standards. We feel there should be a uniform standard set to NAEP."
Knoxville News Sentinel
INDEX OF NCLB OUTRAGES