Response to "March of the Pessimists"
Susan Notes: Social Class, Schooling and Achievement...Again
On August 9th, 2006 New York Times reporter Diana Jean Schemo wrote a column, It Takes More Than Schools to Close Achievement Gap, which raised the issue of how much non-school factors determine student educational success. In her column she reflected on EPI Research Associate Richard Rothstein's work in Class and Schools: Using Social, Economic, and Educational Reform to Close the Black–White Achievement Gap. Schemo's article generated many responses, including an important one by Chester E. Finn (see March of the pessimists), president of the Thomas B. Fordham Institute, in its publication, Gadfly. Rothstein has in turn written a response to Finn (a shorter version been published in Gadfly, and this is his full response.
What we need is Rothstein's return to the On Education column in The New York Times.
Other research relevant to this debate includes the book, The Charter School Dust-up, co-written by Rothstein, and a re-analysis of the Education Trust’s data on 'high flyers' [PDF] by Florida State University professor Doug Harris.
by Richard Rothstein
Chester Finn, in his August 17 "Gadfly" posting ("March of the Pessimists"), responding to a New York Times article by Diana Jean Schemo (here) and a Wall Street Journal essay by Charles Murray, expresses puzzlement that "the likes of Schemo and Murray" can't see that good schools can overcome the disadvantages of poverty, racism, troubled families, crime-infested neighborhoods, and harmful peer influences.
These are complex issues, not elucidated by labeling these writers, as Mr. Finn does, 'liberal,' 'conservative,' 'pessimist,' or 'defeatist.' But I take Mr. Finn at his word that he genuinely does not understand why Schemo, Murray and others do not share his belief in the power of good schools to offset all other social and economic influences. I will attempt, as respectfully as I can, to explain why, for my part, I do not share his belief.
In short, given that, as Mr. Finn asserts, children's time influenced by families and communities exceeds the time they are influenced by schools "by a multiple of four or five," I am puzzled that he fails to agree that serious and successful efforts to substantially narrow the achievement gap must include social and economic policies to improve the circumstances of family and community life, as well as policies to improve the quality of schooling.
First, let's clarify some common imprecisions in the discussion. Mr. Finn asserts that good schools are "powerful enough instruments to boost poor kids' achievement to an appreciably higher academic plane." Nobody - not I, nor anyone with whom I am familiar - disagrees with this assertion. But what is commonly argued (and the notion that I dispute) is not that good schools can boost the achievement of disadvantaged children to
"an appreciably higher plane" but rather that such schools can "close the achievement gap;" i.e., produce achievement from lower class children that is approximately equal to the achievement of middle class children. More specifically, the claim is that if all disadvantaged children could attend such schools, their average achievement would not be appreciably different from the average achievement of middle class children – they would be as likely to attend good colleges, be no more likely to end up in prison or as teen parents, be as qualified for good-paying jobs, etc. Another way of thinking about the claim that good schools can "close the achievement gap" is that if all disadvantaged children attended good schools, and graduated, on average, with average middle class levels of achievement, the vast social inequalities that now pervade American society would disappear. Or, as New York's Mayor Michael Bloomberg put it, if his New York City school reform program succeeded, "a lot of what Dr. [Martin Luther] King wanted to accomplish in our society will take care of itself."
A puzzling aspect of Mr. Finn's confidence that good schools can overcome all or most of the negative influences of deprived social and economic environments is that he himself, in other contexts, wisely endorses "value-added" as a preferred way to evaluate school quality, and as the appropriate way to compare average school-type (charter/non-charter, private/public) performance. Examining value-added trends makes sense only if you understand that social class greatly influences the level of student achievement. Granting that, on average, disadvantaged children (for example, those living in poverty) cannot reasonably be expected to achieve at the same level as middle class children (also, on average), a school serving disadvantaged children can be considered successful if it raises their achievement to levels significantly higher than it was previously, even if these
higher levels remain, on average, considerably below those of typical middle class children. Advocacy of value-added comparisons as a preferred alternative to comparing raw achievement levels for accountability purposes makes sense because it recognizes that most children from poor families start their educations at a significant educational disadvantage to most middle class children, and that during their schooling, middle class children continue to enjoy extra-school educational benefits that children living in poverty do not possess. Advocacy of value-added comparisons makes no sense if you believe that good schools can fully overcome the social and economic influences that depress low-income children's achievement.
Thus, before we can shed light, rather than heat, on this discussion, we need more precision about what Mr. Finn means by an "appreciably higher plane." What does he claim that good schools can actually accomplish? At present, for example, the average achievement of black and white children in America differs by about a full standard deviation, or about 30 percentile points in a distribution, on most standardized tests - and we have no measures whatsoever of the black-white gap on the many school outcomes which such tests don't measure but which Mr. Finn, in his many writings, has also asserted that good schools produce, such as character, citizenship, and work ethic. Social scientists generally consider an intervention to be extraordinarily successful if it has an effect size of 0.5, or more than 15 percentile points. Such an impact of good schools would truly be extraordinary – my guess (without evidence) is that the best school reform, even including the extended school time that Mr. Finn advocates, might aspire to an effect size of 0.3, or about 10 percentile points. So let's assume that school reform, with an effect size of 0.5, might reduce the gap to half a standard deviation, and that this
is what Mr. Finn means by "an appreciably higher plane." This would still leave an enormous achievement gap, and typical black students would still not be able to compete fully successfully with typical whites in the world of academia, the professions, or other skilled work. But a school effect size on average black achievement of 0.5, or even of 0.3, would be a significant accomplishment: although there would still be a big gap in average performance, this shift in the distributions, resulting in greater overlap between black and white achievement, would allow many more black youths to compete successfully than can now do so.
This brings me to the second imprecision in the discussion. Mr. Finn speaks of the impact of good schools on "poor kids' achievement." Which "poor kids"? Mr. Finn's claim makes sense only if we focus on average "poor kids' achievement." Any particular school, whether it is a typical or a "good" school, may have a larger than usual share of children who are above, or below average for all poor children. As the previous paragraph suggested, the variation in poor children's achievement is wide, as is the variation in middle class children's achievement, as is the variation in most human characteristics. Indeed, once you have controlled for major demographic factors, like race and poverty, there is more variation in within-school achievement than in average achievement between schools. A broad range of children's achievement exists even under conditions of constant school quality, or identical income, identical family structure, identical neighborhood influences, or identical peer effects. Consequently, it is no simple task to compare the achievement of 'poor' children in one school setting to those in another: one must identify the prior achievement of particular students (not their school's average achievement) and a host of other characteristics of the students. It would be
useful and important to know whether such detailed comparisons of student performance can identify schools that substantially "beat the odds" and beat them to an extent that lifts students to middle class achievement levels. I know of no accounts of "beat the odds" schools that have attempted, or been able, to do this.
In almost every school, even poorly run regular urban public schools serving disadvantaged children, some children are "above average." Such disadvantaged children who are, even absent school reform, performing at or close to typical middle class levels have not "closed the achievement gap." They simply reflect the inevitable variation in performance that exists even after any social characteristic, like poverty, is controlled. And if such children can then be placed in a separate school, the average achievement of this new school will be high, even if it has no greater quality than the unreformed school from which these children came. Such a new school, notwithstanding its high average achievement for disadvantaged children, cannot be said to "beat the odds," as the term is commonly understood.
So the critical empirical question is this: Mr. Finn alludes to the "wealth of anecdote, example, and research attesting to the success of individual schools in '''beating the odds' and producing well-educated youngsters in spite of the hostile forces at work in many of those kids' lives." Do these individual schools enroll children who reflect the full range of ability of all disadvantaged children (those who ordinarily would perform below, close to, or above average), or do they enroll children who have, on average, greater readiness to learn than typical disadvantaged children – because the enrolled children are, on average, relatively more advantaged in home environment, motivation, health, natural ability, or other characteristics than the average for all disadvantaged children? In other
words, are these schools either explicitly selective, or implicitly selective (for example, because the more advantaged of disadvantaged children are more likely to choose to attend, or have parents who choose for them to attend)?
I have spent considerable effort in recent years examining claims like those of Mr. Finn, and have found that in every case, highly publicized "beat the odds" schools enroll children who are more likely to have higher achievement, often because they have some particular more favorable characteristic that influences achievement. (Examples from the Heritage report that Mr. Finn cites are a school where most children are poor but which is the location of a district-wide "gifted and talented" program whose test scores are included in the school's averages; and schools where most children are poor but where an unusually high proportion of parents have college degrees.) I do not suggest that this means these are not good schools. They may be. But even if so, only part of their higher achievement can be attributed to the quality of the schools. Some other part is attributable to the atypical potential of their students.
I will not engage in an extended discussion of the results of my examinations here. I detailed some of them in my book, Class and Schools (pp 61-83). With respect to the schools that Mr. Finn specifically cites in "March of the Pessimists," I showed that the Heritage Foundation's "No Excuses" schools were mostly selective, even though the students were mostly disadvantaged by low-income or minority status; utilizing analyses performed by Douglas Harris, I reported that the Ed Trust "high flying schools" most often had high percentages of students above proficiency because the schools were in states where the
proficiency standard had been diluted (an action that Mr. Finn himself has properly denounced), or had inconsistent high performance – in only one subject, only one grade, only one year; and I demonstrated that KIPP schools enroll students whose incoming capacity was better than the average capacity of the students in the schools from which they transferred. (The fact that KIPP attracts large numbers of such students, and must select its incoming class from these applicants by lottery, sheds no light on whether the pools in which the lotteries are conducted are representative. Nor does the fact that KIPP makes efforts to solicit applications from typical students shed such light – parents able or inclined, for whatever reason, to provide above-average support to their children are more likely to respond to such solicitations than other parents in the same communities and with the same income levels; to be used for the point Mr. Finn wishes, KIPP would have to enroll not only typical disadvantaged children, but a representative share of disadvantaged children with below-average potential, when compared to other disadvantaged children, as well.)
Following Diana Jean Schemo's article in the New York Times, Joel I. Klein, Chancellor of the New York City schools, published a letter to the Times (August 15) in which he claimed that in contrast to New York's typical regular public high schools with their high dropout rates, in "New York's new small schools, serving the same [disadvantaged] populations, graduation rates are projected to be 73 percent." I have not done a systematic study of these small schools. But I am familiar with enough of them to say that many of these schools, before admitting students, conduct recruiting fairs, examine applicants' test scores, and interview prospective students and their parents. Information gained is then used either explicitly for purposes of acceptance or rejection,
or less openly to encourage applications from students with greater potential and to discourage applications from students with greater difficulties. Further, in establishing the small schools, Mr. Klein exempted them from having to accept, at least initially, special education students in self-contained classes and students who were not fluent in English. New York's typical regular public high schools, of course, include such students whose presence brings down their graduation rates. Perhaps if these new small schools enrolled representative adolescents from the same populations, their projected graduation rates would still be higher than those of regular schools. But before evaluating the significance of Mr. Klein's claims, we need to know whether, and by how much, this may be the case.
Again, I have no quarrel with any of these schools. They may be better, even much better, than typical public schools. But although Mr. Finn cites three decades of research on this point, I am aware of none that distinguishes the extent to which the standardized test scores of "beat the odds" schools are attributable to school practices or to students with greater capacity to benefit from those school practices.
Nor do I have a quarrel with schools that select, either explicitly or implicitly, disadvantaged students who have a greater capacity to succeed. To enhance social mobility and equality in American society, we should do everything we can to give the most able disadvantaged and minority students a boost with the best possible education, so that they can more successfully compete for college and professional jobs with students from more privileged families. If doing this requires that the more advantaged and able of all disadvantaged students be concentrated in special schools, and isolated from the destructive influences of more troubled peers, such policies should be followed.
But social and educational policy is complex – most interventions are neither all good nor all bad, and the best of them may have some negative, if unintended consequences. When disadvantaged students with the greatest likelihood of success are selected out of typical public schools to concentrate and reinforce these students' potential, students with less likelihood of success who remain in typical schools are also concentrated, and their lower aspirations also reinforced. As I wrote in another book The Charter School Dust-Up, with co-authors Martin Carnoy, Rebecca Jacobsen, and Lawrence Mishel), there is too little public discussion of how to balance the costs and benefits of this policy. If concentrating the more able and advantaged of disadvantaged students in better schools not only improves the chances of these students, but also harms the chances of less able and more disadvantaged students left behind, how aggressively should such policies be pursued? There is a large literature on the costs and benefits of heterogeneous vs. homogeneous grouping of students within schools and classrooms. The difficult issues are the same when we consider between-schools selectivity, but a careful consideration of these issues has not made its way into our public debates about the merits of schools that "beat the odds."
In Ms. Schemo's article, I was quoted as saying that schools can't do "much better" without complementary reform in the social and economic conditions from which disadvantaged children come. This was an unfortunate phrase, as imprecise as Mr. Finn's notion that such schools can do "appreciably" better. What I have consistently written is that school reform alone can narrow the achievement gap, but cannot close it. Whether there is a significant difference between saying schools can't do "much better" and that-
they can do "appreciably better" cannot be determined unless we are more precise in measuring the extent to which schools can raise average achievement for the full range of disadvantaged children. Until we have done so, we can't know how much Mr. Finn and I truly disagree.
The reason I consider this imprecision such a serious issue is, as I wrote in Class and Schools, if we truly believe that school improvement alone can close (or even come reasonably close to closing) the achievement gap, then, as Mayor Bloomberg suggested, we need not worry terribly much about the serious social problems facing American society. All these problems – racial discrimination, economic inequality, inequitable access to health care, dysfunctional families and neighborhoods – will take care of themselves. But if school improvement alone cannot close (or come close to closing) the achievement gap, then assertions to the contrary have the effect of undermining public and political pressure to take action to reform other social and economic institutions, making a significant narrowing of the achievement gap less likely. In this sense, the rhetoric of school reform is counter-productive and dangerous.
Nearly 40 years ago, Daniel Patrick Moynihan and Frederick Mosteller rejected the notion that the Coleman Report was a "pessimistic" document. On the contrary, they wrote, it was a hopeful and radical document, because it showed that we could identify (and address) the causes of low average achievement for black children – these causes were mostly in the social and economic circumstances of these children's lives, and many (though not all) of these circumstances were remediable by well-designed social and economic policies. Today, many of those policies that we know will make a difference - providing better and more stable housing, improving the health of low-income children,
and boosting the incomes of these students' working parents – are economically and politically feasible. So in the Moynihan-Mosteller sense, I too am an optimist. I invite Mr. Finn to join me in my optimism by advocating a balanced set of reform policies, covering schools as well as the social and economic conditions that surround them.
He has begun. By advocating schools that are "starting young and running really long days, weeks and years," by acknowledging that such schools "cost more," Mr. Finn is proposing that we dramatically expand public responsibility for aspects of youth development that are not traditionally the province of public education. I don't know if he has ever tried to estimate how much such an expansion of public responsibility would cost. If he has, he might find that we are not that far apart in our views of what meaningful school improvement requires. A quick back-of-the-envelope calculation suggests that adding public responsibility for four or five years of early childhood care and education, after-school (doubling the in-school hours), and summer time (another three full-time months) for disadvantaged children could easily triple what we now spend for the public education of these children. Mr. Finn expects that some of this increase can be offset by "sweat equity from tireless teachers and relentless principals" who, unlike the rest of us, will not expect to be fully compensated for their efforts. I don't think that this is a reasonable source for a significant portion of the costs, but even if it contributed some, if we assume that 25 percent of all children would require such additional services, we'd increase our national average per pupil spending by something like 50 percent. If we add health clinics and a few other social services that we can all agree can counter the most important negative influences on children's development, we're close to increasing our national average spending by 60 or 70 percent. Perhaps, if we did all this for all
schools serving disadvantaged students, it would still be possible for the best of these schools to "beat the odds" by improving outcomes for disadvantaged children even more than typical public schools could do with their expanded resources. That would be a competition worth watching.
Economic Policy Institute
INDEX OF RESEARCH THAT COUNTS