Orwell Award Announcement SusanOhanian.Org Home


Commentaries

Jedi Attackers Lose: Vouchers Don't Work

Posted: 2003-11-05

Gerald Bracey's got the goods on vouchers. Why does the Washington Post ignore him?



"Victory on Vouchers" read Washington Post the day after the House voted, 205-203 to use 13 million taxpayer dollars to let Washington, DC, public school children attend private schools (the Senate later declined to bring the bill to the floor). To the exultant editors vouchers--it was about the 10th pro-voucher piece to appear on either the editorial or op-ed page--made it inevitable that at least some children would receive a better education in private institutions than they would in the district's lousy public schools.



Alas, the Post editors and many other voucher advocates stand apart from the evidence. Privately- or publicly funded vouchers programs have existed in a number of cities now for a number of years. Various researchers have evaluated these programs. They lead to a conclusion starkly different from that reached by the Post editors: vouchers do not work. Students who use them do not have higher achievement than matched groups of students who remained in public neighborhood schools.



These results reduced even one of vouchers' most ardent advocates, Jay P. Greene of the Manhattan Institute to admitting as much last may in the Wall Street Journal. As might be expected, Greene attempted to cast his admission in a positive light. He referenced the voucher evaluation studies and claimed merely that "none of them finds students harmed by receiving a voucher." That is not true as we shall see below, but even if it were, it's a rather pusillanimous conclusion for the effect of an educational reform some considered a panacea, "none of them finds students harmed."



Indeed, Greene's is not the kind of frail rhetoric that voucher advocates have flourished over the years. Washington Post pundit, George F. Will, declared that voucher advocates successes had "pummeled" opponents and New York Times columnist, William Safire, climbed out on a limb to argue that vouchers would wipe out the black/white achievement gap. Paul E. Peterson of Harvard, perhaps the most avid voucher touter aside from Milton Friedman who has been at it since 1960, has also been the most eloquent. Peterson once described each of nine constraints on the Milwaukee voucher program as analogous to Dante's nine circles of Hell. Earlier, he had declared that he and other voucher advocates constituted "A small band of Jedi attackers, using their intellectual powers to fight the unified might of Death Star Forces led by Darth Vader whose intellectual capacity has been corrupted by the urge for complete hegemony."



But that was then, 1990 to be precise. Today, the battle cry is "No harm has been done." Well, yes, first do no harm, but that wasn't exactly Hippocrates' ultimate goal nor has it been until Greene's feeble essay the result either desired or forecast by voucher proponents.



In fact, the lone instance where voucher students appear to have outscored their public school peers is the case of mathematics in Milwaukee. While Peterson had claimed gains in both reading and mathematics, Cecilia Rouse of Princeton, using more reasonable assumptions about the data, could affirm it only for mathematics. Moreover, she suggested that the voucher students' small classes, not their vouchers, likely produced that effect. Rouse also found that Milwaukee public school students in small classes outperformed both the Milwaukee voucher kids and the matched sample in public schools. Other studies also find that small classes trump vouchers, but that hasn't caused the voucher advocates to cease and desist.



When we turn to disinterested studies elsewhere, we find no results to cheer voucher proponents. In 2002, the non-partisan General Accounting Office reviewed evaluations of privately funded vouchers in Washington, DC., Dayton, Ohio, and New York City. Of Washington it said, "The Washington, D.C. study demonstrated positive effects for African American students in the second year of the study, but these disappeared in the third and final year of the study." About Dayton, the GAO team wrote, "Voucher users in Dayton showed no significant improvements in reading or math scores." No harm was done.



The GAO did conclude that New York's voucher program had produced gains but only for African American students. Oddly, it failed to note that these gains had come in only one grade and that the gains in the one grade were so large they pushed to overall average of four grades to statistical significance. Asked about the wisdom of lumping all four grades together when only one showed an impact- most researchers would not-- Peterson said, "An average is an average."



While claiming gains in New York, the GAO report did observe that the New York study had come under challenge. This observation occurred in a footnote and one surmises the contradictory analysis arrived at the GAO too late for full consideration. Actually, the GAO team somehow missed an earlier challenge to Peterson's results by one of Peterson's co-investigators in New York, David Myers, of Mathematics Policy Research. Myers called Peterson's claim of voucher superiority "premature" and Myers' firm stated that "The report shows no overall differences in test scores between 3rd through 6th graders who were offered vouchers and those who were not." Note that Myers did not average scores across grades to get Peterson's effect for African Americans.



The challenge the GAO footnoted came from Alan Krueger and Pei Zhu at Princeton who obtained the raw data from Myers and discovered that Peterson had dropped over 40% of the cases from his analysis. When these data were -- rightly -- added in, the results ceased to be significant even for African American students.



Greene's Wall Street Journal op-ed challenged the Krueger-Zhu conclusions accusing the researchers of making "poor research choices" because the 40% were largely students for whom some background information or prior test scores were missing (some were also students whose categorization as "white" was most peculiar). This was a most curious accusation because Greene and Peterson had earlier defended such an approach: "analysis of randomized experimental data does not require controls for background characteristics or test scores. Such controls are necessary only when one doubts that the experimental data are truly random" (emphasis added). No one has expressed any doubts about the randomization of the New York study.



For his part, Myers again contradicted his co-researcher Peterson, calling the Krueger-Zhu take on the data "a fine interpretation of the results." Myers went on to say, "it is not a study I'd want to use to make public policy." Jedi attackers, however, showed no such reticence. Peterson mounted his steed, galloped to a press conference at the National Press Club in Washington, D. C., and accused Krueger and Zhu of "rummaging theoretically barefoot through the data in hopes of finding desired results."



So Dayton, New York, and Washington, D. C., show no achievement results that favor vouchers. Cleveland, on the other hand, refutes Greene's claim that no students have been harmed by receiving a voucher. Kim Metcalf and others at Indiana University have looked at the Cleveland program since its inception. In the beginning, public school students trailed their voucher counterparts significantly: by 14 points in reading, by 11 points in language arts and by 9 points in math. At the end of second grade, the researchers called the results inconclusive. By the end of third grade, though, a clear pattern had emerged: the public school students had closed the 14-point reading gap to 3 and the 11-point language arts gap to 5. In mathematics, the public school students had overtaken the voucher students' 9-point advantage and led by 2 points.





Metcalf and company reached the only conclusion available: "The most recent results do not reveal any significant impacts of participation in the voucher program on student achievement." Cleveland's vouchers might pass First Amendment muster, but they do not benefit the students who use them. The Jedi, again not deterred by facts, pushed a $10.5 million expansion of the voucher program through Ohio's legislature.



One need not conduct a formal meta-analysis of these data collectively to see that if one tallies the results of these five studies, one comes up empty at best, negative in the case of Cleveland. If we had five such studies showing some other reform to be ineffectual, we would be hearing demurrers and dissents and "Let's try something else" calls. Apparently when you're a Jedi, though, your beliefs endure free of facts. How else to explain the exhilaration of the Washington Post editors?



========

Gerald W. Bracey is an associate professor at George Mason University and an Associate with the High/Scope Educational Research Foundation. His most recent book is On the Death of Childhood and the Destruction of Public Schools (Heinemann, September 15, 2003).



More Commentaries


FAIR USE NOTICE
This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available in our efforts to advance understanding of education issues vital to a democracy. We believe this constitutes a 'fair use' of any such copyrighted material as provided for in section 107 of the US Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit to those who have expressed a prior interest in receiving the included information for research and educational purposes. For more information click here. If you wish to use copyrighted material from this site for purposes of your own that go beyond 'fair use', you must obtain permission from the copyright owner.