Every State Left Behind
Ohanian Comment: As Diane Ravitch points out, Republican and Democratic politicos alike have joined hands with the Business Roundtable in pushing for a national test. However, she would have us assume that the NAEP is valid. Many respected authorities dispute this. I hope my article in the November 2005 Substance will convince any doubters just how wrong-headed NAEP is in its reading assessment.
I agree with Ravitch that we must "invest in human capital." But I say this means raising the minimum wage, making sure every working person receives a living wage. We can't continue to insist that schools must supply the capital that should be the province of families.
I attach recent remarks from Gerald Bracey on NAEP.
OH, THOSE NAEP ACHIEVEMENT LEVELS
Gerald W. Bracey
The 2005 NAEP results will arrive shortly and more tongues will cluck about them this time than in the past. That’s because some reformers have made the NAEP achievement levels—basic, proficient, and advanced--more prominent by calling for them to be used to validate state achievement results reported for NCLB. Such use would be a disaster. The NAEP achievement levels are “fundamentally flawed” to use the words of the National Academy of Sciences (NAS).
In fact, everyone who has studied the NAEP achievement levels has said, essentially, “These things are no damn good.” Those who have studied them include the NAS, the Government Accounting Office, the Center for Research in Evaluation, Standards and Student Testing, and the National Academy of Education. Even the NAEP reports themselves contain a disclaimer quoting from the NAS study: “NAEP’s current achievement level setting procedures remain fundamentally flawed. The judgment tasks are difficult and confusing; raters’ judgments of different item types are internally inconsistent; appropriate validity evidence for the cut scores is lacking; and the process has produced unreasonable results.”
Fundamentally flawed? Judgments inconsistent? Validity evidence lacking? Can you imagine the howls of outrage that would greet ETS or CTB/McGraw-Hill if they dared bring to market an instrument with such basic failures?
So why are we still using the achievement levels? The official story from the U. S. Department of Education is that “a proven alternative to the current process has not yet been identified.” That was written in 1998. One would think that a Department as obsessed with applying “scientifically based research” as the current one would have screamed in horror at the flawed achievement levels and rushed to fix them.
The truth is, though, neither the Department nor anyone else is trying to develop a “proven alternative.” Indeed, many observers believe that the NAEP achievement levels, created by the National Assessment Governing Board under its then-president Chester Finn, were deliberately set too high in order to sustain the sense of crisis created by 1983’s “A Nation At Risk.” There is no rush to develop new achievement level setting procedures because much political hay can be made by alleging that American students are performing poorly.
Here’s what NAS meant by “unreasonable results:” The NAEP achievement level results do not accord with any other performance evaluations, especially results from international comparisons. For example, in the 1996 NAEP science assessment, only 30% of American 4th graders scored proficient or better. In that same year, though, the Third International Mathematics and Science Study found American 4th graders third in the world in science among 26 nations. Such “unreasonable results” as the NAEP-international comparisons discrepancy consistently appear.
The 2005 results will get more attention than usual because of No Child Left Behind (NCLB). Currently, under NCLB, each state uniquely defines “proficient.” To some, this creates a Babel of incomparable results. We need a common yardstick, they say, that will let us compare states. NCLB made state-level NAEP mandatory and it ever so conveniently has an achievement level named “proficient.”
Rod Paige said he would use the NAEP achievement levels to “shame” states into doing better. Others, such as the American Enterprise Institute’s Frederick Hess and Finn have proposed that NAEP be the NCLB test used to evaluate schools in reading and math. And Hess and Paul Peterson of Harvard have developed a procedure to grade all states based on the discrepancy between the percent proficient on the state test and the percent proficient on NAEP.
Few states do well by Peterson-Hess. Only 5 get A’s and only 2 get B’s. The scale doesn’t strike me as particularly sophisticated or accurate. For instance, South Carolina gets an A because there’s little difference between the state test and NAEP: The state is low on both. Connecticut, on the other hand, gets a C-. Yet Connecticut has the nation’s highest proportion of students proficient on NAEP reading. The “Texas Miracle,” on the other hand, disappears. Texas gets an F because it claims that 87 percent of its 4th graders are proficient in reading while NAEP says only 33 percent.
But you can rest assured that when the NAEP results appear, school critics and reporters both will point to the NAEP-state discrepancies and imply that the state is lying about how well its kids are doing. In some quarters, it will be argued that the discrepancies mean we need vouchers and more charter schools.
By Diane Ravitch
WHILE in office, Presidents George H. W. Bush and Bill Clinton both called for national academic standards and national tests in the public schools. In both cases, the proposals were rejected by a Congress dominated by the opposing party. The current President Bush, with a friendly Congress in hand, did not pursue that goal because it is contrary to the Republican Party philosophy of localism. Instead he adopted a strategy of "50 states, 50 standards, 50 tests" - and the evidence is growing that this approach has not improved student achievement. Americans must recognize that we need national standards, national tests and a national curriculum.
The release last month of test results by the National Assessment of Educational Progress, which is part of the Department of Education, vividly demonstrated why varying state standards and tests are inadequate. Almost all states report that, based on their own tests, incredibly large proportions of their students meet high standards. Yet the scores on the federal test (which was given to a representative sample of fourth and eighth graders) were far lower. Basically, the states have embraced low standards and grade inflation.
Idaho claims that 90 percent of its fourth-grade students are proficient in mathematics, but on the federal test only 41 percent reached the Education Department's standard of proficiency. Similarly, New York reports that nearly 85 percent of its fourth graders meet state standards in mathematics, yet only 36 percent tested as proficient on the national assessment. North Carolina boasts an impressive 92 percent pass rate on the state test, but only 40 percent meet the federal standard.
In fourth-grade reading, the gaps between state and national reports are equally large. Georgia claims that 87 percent of its pupils are proficient in reading, but only 26 percent reached that level on the national exam. Alabama says that 83 percent of its students are proficient, but only 22 percent meet the federal standard.
The same discrepancies are found in the scores for eighth-grade reading, where Texas reports that 83 percent met the state standard, but the federal test finds that only 26 percent are proficient. Tennessee and North Carolina both claim that 88 percent are proficient readers, whereas 26 percent and 27 percent, respectively, met that mark on the federal test.
Why the discrepancies? The states function in a political environment. Educational leaders and elected officials want to assure the public that the schools are doing their jobs and making progress. The federal testing program, administered for the past 15 years by an independent, bipartisan governing board, has never been cowed by the demands of parents, school officials and taxpayers for good news.
In the No Child Left Behind law of 2001, Congress left it to each state to develop its own standards and tests, but added that the tests given by National Assessment of Educational Progress should serve as an external gauge of national and state-level achievement. The federal tests are considered the gold standard for good reason: they are the product of a long-term federal investment in research and development. Unlike the state tests, the federal program tries to align its performance standards with international education standards. Many states model their testing on the national program, but still cling to lower standards for fear of alienating the public and embarrassing public officials responsible for education.
The price of this local watering-down is clear. Our fourth-grade students generally do well when compared with their peers in other nations, but eighth-grade students are only average globally, and 12th graders score near the bottom in comparison with students in many European and Asian nations. Even our students who have taken advanced courses in mathematics and physics perform poorly relative to their peers on international tests.
Last month, the National Academy of Sciences released a report warning that our nation's "strategic and economic security," as well as our leadership in the development of new technologies, is at risk unless we invest heavily in our human capital; that is, the education of our people. The academy report made clear that many young Americans do not know enough about science, technology or mathematics to understand or contribute to the evolving knowledge-based society. The best way to compete in the global economy, the report maintained, is to ensure that American workers are "the best educated, the hardest-working, best trained, and most productive in the world."
It is fair to say that we will not reach that goal if we accept mediocre performance and label it "proficient." Nor will we reach that goal if we pretend that mathematics taught in Alaska or Iowa is profoundly different from the mathematics taught in Maine or Florida, or for that matter, in Japan and Hungary.
Unfortunately, the political calculations that resulted in the No Child Left Behind law adopting a strategy of letting the states choose their own standards and tests remain the reality. In general, Republicans are wary of national standards and a national curriculum, while Democrats are wary of testing in general. Both parties must come to understand that the states are not competing with each other to ratchet up student achievement. Instead, they are maintaining standards that meet the public's comfort level.
America will not begin to meet the challenge of developing the potential of our students until we have accurate reporting about their educational progress. We will not have accurate reporting until that function is removed from the constraints of state and local politics. We will be stuck with piecemeal and ineffective reforms until we agree as a nation that education - not only in reading and mathematics, but also science, history, literature, foreign languages and the arts - must be our highest domestic priority.
Diane Ravitch, a research professor at New York University and fellow at the Brookings Institution, was on the governing board of the National Assessment of Educational Progress from 1997 to 2004.
New York Times