Orwell Award Announcement SusanOhanian.Org Home


The Perverse Incentives of the No Child Left Behind Act

Susan Notes: This report deserves careful study. Although for the purposes of his argument here, the author doesn't wish to go into them, the author acknowledges "strong arguments for abandoning test-based accountability altogether."

Amazingly, Ryan also acknowledges, in passing, that some people feel that standards are at the root of the problem, even referencing Alfie Kohn's work and mine. This is amazing because so few people give the problem of standards even passing reference.

Ryan concludes that the way out of the NCLB fiasco may be a version of a value-added framework. What is missing from this analysis is any mention of the role the Business Roundtable and its minions have played in setting NCLB in place. Repeating the mantra that everyone supports the laudable goals of NCLB, he makes no mention of those whose hands aren't clean.

The activist will find parts to quarrel with and parts to cheer about. There are plenty of very quotable lines. But the author's argument deserves more than a soundbite. That said, here's one good line: The odds are quite good that the NCLBA is another fad.Rather amazingly, he also acknowledges, in passing, that some people feel that standards are at the root of the problem, even referencing Alfie Kohn's and mine. Again, he doesn't go into it. This is amazing because so few people give the problem of standards even passing reference. The author concludes that the way out of the NCLB fiasco may be a version of a value-added framework. What is missing from this analysis is any mention of the role the Business Roundtable and its minions have played in setting NCLB in place. Repeating the mantra that everyone supports the laudable goals of NCLB, he makes no mention of those whose hands aren't clean.

Here find the first part of the article. For the rest, go to the url below.


ABSTRACT
This Article examines the No Child Left Behind Act, which may be the most important federal education law in our nation's history. The Act is supposed to increase academic achievement in schools across the nation, raise the performance of disadvantaged students to the level of their more affluent counterparts, and attract qualified professionals to teach in every classroom. These goals are obviously laudable. As Professor Ryan explains, however, the Act creates incentives that actually work against their achievement. Specifically, the Act unintentionally encourages states to lower their academic standards, promotes school segregation and the pushing out of poor and minority students, and discourages good teachers from taking jobs in challenging classrooms. Should any or all of these effects occur, achieving the Act's goals will be more difficult, not less. Professor Ryan goes on to suggest a solution, albeit a partial one, to the problems created by the No Child Left Behind Act. Rather than focus on absolute achievement levels as the basis for school accountability, Ryan argues that the federal government and states should focus on rates of growth. Doing so would not only give a more accurate picture of school quality, and thus provide a fairer basis for school accountability; it would also diminish or eliminate the perverse incentives created by the No Child Left Behind Act. The Article concludes with a brief discussion of what the No Child Left Behind Act can teach us about the proper role of the federal government in education law and policy.

INTRODUCTION
The No Child Left Behind Act, perhaps the most important federal education law in our nation's history, is at war with itself. The chief goals of the Act are to boost academic achievement across the board and to eliminate the achievement gap among students from different backgrounds.(1) To accomplish these goals, the Act requires states to establish "challenging" academic standards for all schools and to test all students regularly to ensure that they are meeting those standards.(2) The Act also requires states and school districts to employ teachers who are "highly qualified," meaning that they have demonstrated some competence in the subjects they teach.(3)

Schools are expected to have all of their students scoring at the proficient level on state tests within twelve years of the Act's passage. In the meantime, states must establish intermediate goals that require an ever-increasing percentage of students to demonstrate proficiency.

The same intermediate achievement targets must be met both by schools as a whole and by various subgroups of students within each school, including those who are poor, racial and ethnic minorities, English-language learners, and those entitled to special education services.4 Schools that receive federal funding and fail to meet their targets face increasingly harsh sanctions for every year that they fail.(5)

The No Child Left Behind Act has been praised by some and condemned by others in the popular press and in education journals, although it has received surprisingly little attention in the legal literature.(6) Those who favor the Act emphasize its laudable goals and celebrate its tough accountability measures.(7) Those who criticize the Act lament the heavy emphasis on testing and the inevitable "teaching to the test" that will follow.(8) They also chastise the federal government for interfering with state and local control over education while failing to fund all of the costs associated with the Act.(9)

My assessment is somewhat different. I agree that the Act's goals are laudable. In my view, however, the Act's fatal flaw is that it creates incentives that work against the Act's goals. First, while the Act is supposed to raise achievement across all schools, it creates incentives for states to lower academic standards. Second, while the Act is supposed to close the achievement gap, it creates incentives to increase segregation by class and race and to push low-performing students out of school entirely, which will make it even more difficult for disadvantaged students to catch up to their more affluent peers. Finally, while the Act is supposed to bring talented teachers to every classroom, it may deter some from teaching altogether and divert others away from the most challenging classrooms, where they are needed the most. In short, although the Act is supposed to promote excellence and equity, it may work against both.

This Article explains how the Act creates these perverse incentives, and it offers a partial solution. The central problem is the Act's central feature-namely, the requirement that schools be sanctioned if their students fail to demonstrate an absolute level of achievement on tests within a relatively short time period. The requirement that an increasing percentage of students in every school achieve a certain test score each year is arbitrary and unrealistic, in that it establishes achievement goals without any reference to past achievement levels or rates of achievement growth. Many schools, including some that are considered effective, will be unable to meet these achievement targets. This will create pressure to make the targets easier to meet by dumbing down the tests or making scoring systems more generous. By this process, a law intended to raise academic standards may lower them.

Focusing on absolute achievement levels rather than achievement gains also will generate incentives for parents, teachers, and administrators to shun disadvantaged children and the schools that educate them. The reason is fairly simple: Disadvantaged students tend to do worse on standardized tests than do their more affluent counterparts. An accountability system that rewards and punishes schools based on absolute achievement levels will thus reward relatively affluent schools and punish relatively poor ones. Moreover, given that minorities are disproportionately poor, and that all schools are held responsible for the performance of their minority and poor students, this accountability system will tend to punish those schools that are racially and economically diverse. All of this will make racial and socioeconomic integration even more difficult to achieve than it is already, and it will provide even more incentives for good teachers to choose relatively affluent schools. These trends, in turn, make it possible that a law designed to narrow the achievement gap will help widen it.

Given that the main problem with the Act is its focus on absolute achievement levels, a partial solution is to focus instead on rates of growth. As explained below, this would provide a more accurate measure of the value that schools add to a student's knowledge and abilities, and thus a more accurate picture of school quality. Focusing on rates of growth would also level the playing field among schools with different student populations, making it less likely that judgments about school quality will simply track the socioeconomic status of the students who attend them. This, in turn, would reduce the incentives of good teachers to avoid disadvantaged schools.

Looking to growth rates is not an untested idea. On the contrary, it formed the basis of so-called "value-added" accountability systems, which were increasingly popular prior to the passage of the No Child Left Behind Act. While focusing on rates of growth is thus not a radical and untried proposal, it is also not a panacea. Rewarding schools for meeting growth targets that are realistic may not provide sufficient incentives for schools to do any more than improve marginally on the status quo. Focusing on growth, moreover, necessarily tolerates different levels of absolute achievement for different students. A school whose fifth graders begin the year reading at the third-grade level and end reading at the fourth-grade level certainly has added value and done a decent job; but those students would still be a year behind in reading.

Despite these shortcomings, a value-added approach seems clearly superior to the approach embodied in the No Child Left Behind Act. The question then becomes whether it is politically plausible. At first glance, one might be skeptical. Considerable tension exists between a value-added accountability system and the core rhetorical commitment of the No Child Left Behind Act, which is to bring all students to the same level of achievement in a relatively short period of time. Nevertheless, the Department of Education has given states some flexibility in implementing the Act, offering them the opportunity to incorporate some focus on growth rates in their accountability systems, at least in the short run. States should take advantage of this flexibility. All states, moreover, should recognize that it is highly unlikely that the No Child Left Behind Act will remain in force, unchanged, for the next ten years. States thus should concentrate on short-term attempts to combat the perverse incentives created by the Act.

This Article proceeds in three Parts. Part I places the No Child Left Behind Act in historical context and describes its central features. Part II describes how the Act may push states to lower their academic standards and goals, increase pressure on schools to segregate students by class and race and exclude low-performing students altogether, and deter talented teachers from entering the profession and taking jobs in challenging schools. Part III explores the benefits and limitations of using rates of growth rather than absolute achievement levels as a basis for school accountability. While recognizing the inherent problems of this alternative, I argue that it is preferable to the approach of the No Child Left Behind Act and that states should do what they can to incorporate elements of such an approach into their accountability systems. I conclude with some tentative observations as to what the No Child Left Behind Act reveals about the proper role of the federal government in education law and policy.

One final introductory note is in order. The topic of standards and testing is a very large one, which could be approached from a number of angles. This Article concentrates on the mechanics and incentives of test-based accountability systems, focusing primarily on the regime established by the No Child Left Behind Act. I leave to one side some basic questions about the wisdom of standards and testing generally. There are serious disagreements as to whether relying solely or primarily on tests to hold students and schools accountable is wise policy, especially in light of the generally poor quality of current tests and the pervasive inequality of school funding. Those arguments already have been rehearsed thoroughly by others.(10) What has received less attention is the issue that I address here- namely, the incentives created by the implementation of a test-based accountability system. Given that testing is ubiquitous and likely to continue for some time, even if the No Child Left Behind Act is modified or repealed, understanding the incentives created by test-based accountability is crucial.(11)

THE NO CHILD LEFT BEHIND ACT
A. Background
Supported by an overwhelming majority in Congress and signed into law by President Bush in 2002, the No Child Left Behind Act (NCLBA) is remarkably ambitious and unusually intrusive.(12) The NCLBA revises the Elementary and Secondary Education Act,13 which was first enacted in 1965 and has been reauthorized periodically ever since.14 The most important and well-known component of the Elementary and Secondary Education Act is Title I, which is the federal government's single largest educational aid program and ostensibly is designed to assist disadvantaged students. In exchange for federal funding, which all states receive, states and local school districts must comply with various federal directives. From its passage until fairly recently, Title I received more criticism than praise. Empirical studies generally concluded that Title I fell far short of its goal of closing the achievement gap between poorer and more affluent students.15 One problem was the way federal money was used. Title I funding mostly supported the hiring of teachers' aides and the creation of remedial classes for disadvantaged students, who typically were pulled out of regular classrooms and exposed to a watered-down curriculum.(16) Not surprisingly, this strategy did little to bridge the achievement gap.

By the time Title I was scheduled for reauthorization in 1994, many in and outside of the federal government agreed that the program needed alteration. Congress and President Clinton turned to standards-based reform for inspiration and direction.17 Standards-based reform centers on the simple idea that states should set ambitious academic standards and periodically assess students to gauge their progress toward meeting those standards.18 The reform traces back to the 1983 publication of A Nation At Risk, a highly critical and widely publicized report on public schools, which argued in dramatic terms that America's schools set their sights too low.19 Standards-based reform promised to raise the academic bar by requiring all schools within a state to meet uniform, challenging standards. In addition to promoting excellence, standards-based reform also promised to promote equity by requiring all students, not just those in privileged suburban schools, to meet the same rigorous standards.(20)

In reauthorizing Title I in 1994 through the passage of the Improving America's Schools Act (IASA),(21) Congress and President Clinton incorporated the core ideas of standards-based reform.(22) In doing so, they fundamentally changed the nature of Title I. Instead of providing funds to support remedial instruction for disadvantaged students, Title I funds now had to be used to create standards for all students.23 In order to receive Title I funds, states had to create "challenging" content and performance standards in at least reading and math, develop assessments that were aligned with those standards, and formulate plans to assist and ultimately sanction failing schools.24 Importantly, standards and assessments for Title I schools had to be the same as those established for all other schools within a state.25 In this way, the federal government hoped to ensure that states would hold all students to the same high expectations and hold all schools, regardless of their student population, accountable for failure.

B. Key Provisions
The No Child Left Behind Act follows the same basic approach as the IASA, but it establishes more ambitious goals and places greater constraints on the states. States must still develop "challenging" content and performance standards, now not only in reading and math, but also in science.26 States must still use assessments that are aligned with those standards, and must hold schools and school districts accountable for failing to meet ambitious achievement goals.(27)

The most significant changes have to do with teachers, testing, and accountability. As for teachers, the NCLBA requires that Title I schools hire only "highly qualified" teachers for all subjects28 and that veteran teachers in such schools demonstrate that they are "highly qualified" by 2005–6.29 The Act also reaches beyond Title I schools and requires that all teachers of "core academic subjects"(30) in non–Title I schools must be "highly qualified" by 2005–6.31 Pursuant to the Act and accompanying regulations, teachers are considered "highly qualified" if they are fully certified and have demonstrated competency in the subjects they teach. Competence is assumed if the teacher majored in the subject in college; alternatively, it can be demonstrated by passing a state test or, for existing teachers, by convincing state evaluators that they know their subject areas.(32)

As for testing and accountability, whereas the IASA required testing in math and reading at three points in a student's school career, the NCLBA requires annual testing in reading and math in grades three through eight. At least one more test in reading and math must be given in grades ten through twelve.(33) Beginning in 2007–8, students must also be tested in science at least three times between grades three through twelve.(34)

Test scores are the fuel that makes the NCLBA run. Scores are tabulated for schools in the aggregate and must be disaggregated for a number of subgroups, including migrant students, disabled students, English-language learners, and students from all major racial, ethnic, and income groups.35 All of these scores are then used to determine whether schools are making "adequate yearly progress." Adequate yearly progress (AYP), in turn, is the linchpin of the NCLBA.

Adequate yearly progress is tied to whether a sufficient percentage of students are performing proficiently on state tests.(36) The NCLBA requires states to bring all students to the proficient level within twelve years of the Act's passage (i.e., by 2014), and states must ensure that their definitions of adequate yearly progress will enable the ultimate twelve-year goal to be met.(37) To accomplish this, states must set a proficiency goal each year, and that percentage must rise periodically so that by 2014, it hits 100%. For a school to make adequate yearly progress, the student population as a whole, as well as each identified subgroup of students, must meet the same proficiency goal.(38) For example, if in the year 2004–5, the state determines that 65% of students must be "proficient" on the tests, 65% of all the students within a school and 65% of the students within each subgroup (e.g., disabled students, poor students, minority students) must be performing proficiently for a school to be making adequate yearly progress.

Adequate yearly progress is thus less about yearly achievement gains than it is about hitting uniform benchmarks. All states must set a uniform bar for achievement for all schools and all subgroups of students within a school. The first benchmarks were based on test scores from 2001–2. Using these test scores, states had to establish a starting point for AYP that was the higher of the following two values:

(1) the percentage of students in the lowest-achieving subgroup, statewide, who were performing proficiently; or (2) the threshold percentage of students performing proficiently in the lowest-performing quintile of schools statewide.(39) If 30% of a state's poor students, for example, scored at the proficient level in 2001–2, while 40% of all students in the school at the twentieth percentile of achievement scored at the proficient level, the initial AYP bar must be at least 40% for all schools and all subgroups of students. According to the language of the Act,(40) the percentage of students performing proficiently must rise every two or three years, like stair steps, until the 2013–14 school year, when all students must be scoring at the proficient level.(41)

Although the Act is quite strict in defining AYP, it is remarkably loose with regard to state standards and tests. (The basic reason for this structure is the continued resistance to national standards and tests.(42) States are free to determine their own standards, to create their own tests, and to determine for themselves the scores that individual students must receive in order to be deemed "proficient." The harder the tests or the higher the scores needed to be deemed proficient, the harder it will be for schools to meet the NCLBA's definition of adequate yearly progress. For the same reasons, some states have much farther to travel than others in order to meet the goal of 100% proficiency. The starting percentages in Massachusetts, for example, were roughly 40% proficiency in reading and 20% proficiency in math. In Colorado, the starting percentages ranged, depending on the grade level, from roughly 75%–90% in reading and 50%–80% in math.(43)

The Act requires all schools within a state, regardless of whether they receive Title I funding, to make adequate yearly progress. It also requires states and districts to disseminate information about each school's AYP status.(44) The stricter accountability mechanisms, however, are reserved for schools receiving Title I funding.(45) Although this is obviously a subset of all schools, it is a surprisingly large one. Over half of all schools in the nation receive Title I funds.(46) Not all of these schools, moreover, are in predominantly poor districts. Because Title I funds are widely distributed, it is not uncommon for relatively poor schools in predominantly middle class suburbs to receive Title I funding.(47)

Those schools that receive federal funding and fail to make adequate yearly progress are identified as in need of improvement.48 They are also subject to a range of progressively more serious actions. After two consecutive years of failure, schools must develop a plan for improvement and are supposed to receive "technical" assistance.(49) Students in those schools are also allowed to choose another public school, including a charter school, within the same district.(50) After three years, students who have not already departed for greener pastures must be provided with tutoring services from an outside provider, public or private.(51) Those schools that fail to make AYP for four consecutive years must take one of several measures, including replacing school staff or instituting a new curriculum, and those that fail for five years in a row must essentially surrender control to the state government, which can reopen the school as a charter school, turn over management to a private company, or take over the school itself.(52)

One final testing requirement should be mentioned, because it is important to the argument below. The NCLBA requires that the National Assessment of Educational Progress (NAEP) reading and math tests be administered every two years to fourth and eighth graders.(53) The NAEP is an extensive testing program that has been used for over thirty years to collect data about student achievement.(54) The test is essentially national, in that it is not aligned with any state standards. Instead, the NAEP attempts to measure content and skills thought common to all state educational systems. Prior to the NCLBA, participation in the NAEP was voluntary, but now all states must participate. Nonetheless, only a random sample of students within each state must take the test, and scores are not reported for individual students or individual schools.(55) The NCLBA does not indicate what is supposed to be done with the results of the NAEP, but supporters of the Act suggest that results on the NAEP will ensure the rigor of standards and tests used in each state.(56) Whether this use of the NAEP will be successful in keeping state standards and tests rigorous is subject to serious question, as the next Part explains.

— James E. Ryan
New York University Law Review, volume 79, number 3

http://www.law.nyu.edu/journals/lawreview/issues/vol79/no3/NYU303.pdf


INDEX OF RESEARCH THAT COUNTS


FAIR USE NOTICE
This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available in our efforts to advance understanding of education issues vital to a democracy. We believe this constitutes a 'fair use' of any such copyrighted material as provided for in section 107 of the US Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit to those who have expressed a prior interest in receiving the included information for research and educational purposes. For more information click here. If you wish to use copyrighted material from this site for purposes of your own that go beyond 'fair use', you must obtain permission from the copyright owner.