Presenting No Evidence: Advocates for 'Portfolio School Districts' Oversell their Product
Susan Notes: One thing to remember about New Orleans: Post-Katrina, 7,000 teachers were laid off and had to reapply for their jobs.
In Memphis, with a smaller number of schools undergoing 'reform,' more than 50 teachers and seven administrators were fired, with 5 teachers and 3 administrators rehired.
William J. Mathis, (802) 383-0058, firstname.lastname@example.org
Elizabeth DeBray, (706) 542-6249, email@example.com
BOULDER, CO (October 8, 2013) -- 'Portfolio district' reform is among the most dramatic and fast-growing changes to public schooling over the past decade. But its growth has not been matched by supporting research evidence. In fact, little research evidence is available at all to help policymakers consider the value of this highly promoted reform.
Into this vacuum have stepped advocates offering sales-pitches, most notably in the form of PowerPoint presentations. A new review of two such presentations examines their content and limitations, cautioning policymakers and others that the claims are weaker than they may initially seem.
Under the portfolio district model, a public school district contracts with outside providers to run the districtĂ˘€™s schools. The district administration is responsible for holding the outside providers accountable for their results--generally--the academic achievement of the schools' students, and typically measured by standardized tests.
Two recent presentations claiming success for the portfolio district approach focus on Memphis, Tennessee, and New Orleans, Louisiana. These presentations have now been reviewed for the Think Twice think tank review project by Elizabeth DeBray, an associate professor in the College of Education at the University of Georgia, and Huriya Jabbar, a doctoral candidate at the University of California, Berkeley. The review is published today by the National Education Policy Center, housed at the University of Colorado Boulder School of Education.
DeBray has conducted research on the politics of federal education policy, policy implementation, and the role of intermediary organizations in disseminating and producing research on KĂ˘€"12 education policies. Jabbar's doctoral dissertation research examines school choice, competition, and regulation in New Orleans; she has also investigated how state and district policymakers in Louisiana use research.
The review focuses on the following two PowerPoint presentation slide sets: Building the Possible: The Achievement School District's Presentation in Milwaukee, presented by Elliot Smalley, chief of staff for the Achievement School District in Memphis, and The Recovery School District's Presentation in Milwaukee, by Patrick Dobard, superintendent of the Recovery School District in New Orleans. The districts made their presentations in August to members of the Metropolitan Milwaukee Association of Commerce, the largest business lobby group in Milwaukee, which in the past has also supported legislation to enable changing the governance of the Milwaukee Public School District from the current elected board to the office of the Mayor of Milwaukee.
"Although no rigorous research has yet examined the effectiveness of portfolio governance structures, the presentations are aimed at encouraging their adoption," DeBray and Jabbar write in their review. Policymakers, they caution, should not attempt to implement such significant changes to the structure and governance of school systems "without a comprehensive and nuanced discussion of relevant evidence."
The presentationsĂ˘€™ "strong assertion of positive results"--namely, improved student achievementĂ˘€"omits any acknowledgement of "the thin evidence base on portfolio governance," the reviewers add.
They point out as well that the presentations "ignore alternative explanations for the results they do assert." The New Orleans data on improved achievement, for instance, are presented without accounting for factors such as the massive out-migration of New Orleans students, which by itself could plausibly have led to higher average achievement test scores for those who remained. Moreover, the Memphis findings are from too small a time period and reflect too small a scope of data to provide reliable conclusions, according to DeBray and Jabbar. Both presentations report "teacher and administrator human-capital improvements," but fail to provide any specifics about the kinds of improvements, their cost, or how widespread they were across the district.
Just as importantly, the cost of the portfolio model as it was carried out in each district is possibly underestimated, because both districts benefited from additional federal and philanthropic funds, which were not included in the PowerPoint analyses.
DeBray and Jabbar conclude that the presentations lack the research base policymakers should demand before making sound decisions about implementing the portfolio model. The presentations therefore do not serve the interests of taxpayers, communities, or schoolchildren.
Both PowerPoint presentations can be found at http://www.publicschoolshakedown.org/review-portfolio-districts.
INDEX OF RESEARCH THAT COUNTS