Inside the No Child Left Behind Law
Comments from Annie: Maybe it is a good time to revisit this 2003 debate between Gerald Bracey and Andrew Rotherham, recent co-founder and co-director of Education Sector.
Ignoring the distraction that this debate was initiated by Jay Mathews, who admits to his own confusion, there are some significant points, as well as accusations here which help reveal the debaters’ ideology.
Recently, as educational vocabulary has flourished in the area of rhetorical creativity, it can be difficult to translate words like “standards” or “reform” or “quality”. These words can now be used with simultaneous and hidden meanings.
It is often hard to judge a writer or speaker’s perspective when words are used persuasively; words that once had clear and genuine meaning have evolved to gain momentum and support for NCLB. Check your dictionaries and beware of hidden agendas.
Read it again and see if you can, in retrospect, understand the original and fundamental difference in perspectives on NCLB. (This is a copy of Part 1 only.)
by Jay Mathews
Like many people, I am having difficulty understanding the new federal No Child Left Behind Law. It helps me to have well-informed advocates debate issues that confuse me. So two experts have kindly accepted my invitation to argue about the law in this column.
I will start it off with a question for Andrew J. Rotherham, director of the 21st Century Schools Project at the Progressive Policy Institute in Washington and a former White House education adviser in the Clinton administration.
Gerald W. Bracey, a Fairfax, Va.-based educational psychologist and research columnist for the educational issues magazine Phi Delta Kappan, will respond and will eventually get the last word.
Please let me know if you think there are parts of the bill and this debate that need further examination. I hope to write several columns about the law this year, and I need to know what parts of it merit the closest attention.
JAY: There is much mystery and confusion surrounding the "adequate yearly progress" provisions of No Child Left Behind (NCLB). Some people say they are written in such a way that most schools in the country will have to be put on the "needs improvement" list, particularly if this assessment system survives long enough to approach the point where every student must test proficient. Won't that confuse teachers, parents and students and cause them to reject the program as a bloody meat cleaver, rather than a vigorous motivator of better schools?
ANDY: First, let's consider what Adequate Yearly Progress (AYP) is, and is not. It is a set of parameters that requires states to measure school performance, primarily teaching reading and mathematics. Under No Child Left Behind (NCLB), states must develop a system to ensure not only that overall student achievement is improving but that achievement of low-income and minority students is improving too.
The reason for this is that overall student achievement can often paint a misleading picture. For example, the average performance of American students on international comparisons masks high performance by affluent white students and at the same time obscures the, on average, lower performance of minority students. The same thing is true in many states and individual schools.
The AYP provisions will introduce much needed transparency and help ensure that schools are focusing on all students, not only those that are easy to educate. AYP specifically requires states to develop a system to measure the progress of students and schools and a corresponding strategy to address problems and close the achievement gap. The student proficiency targets are determined by the states, the states choose which tests to use, and states have considerable flexibility about how to use test scores to measure progress.
In addition, there are provisions and allowances to ensure that test scores are used in a statistically sound way. Test scores can be aggregated over several years and schools should not be penalized if there are too few students in a particular sub-group to yield reliable information or if a single subgroup fails to meet a particular target but makes progress nonetheless. At a minimum, schools must miss their AYP targets for at least two years before any action is taken and longer before more serious consequences come into play.
Remember, this is about adequate progress toward having all students meet state standards for proficiency. Schools are not required to meet some uniform bar for all students in the next few years.
If a state is thoughtful about how it implements AYP and does so with an eye toward measuring progress, intervening where necessary, and most importantly supporting struggling schools and students, then the AYP process can help. What AYP is not is a one-size fits all federal measurement scheme.
In fact, contrary to many media accounts, which schools are deemed as in need of improvement is a state-level issue based on a state's standards and AYP plan. The states report this information to Washington. Claims that federal officials will be going into states and declaring individual schools failures are simply not true. Prior to this law states were required to have an AYP plan although they could pretty much do as they pleased.
As a result there was considerable variance about how much attention states paid to low-performing demographic groups and schools.
So, requiring AYP is not new but the changes in NCLB are an effort to rectify existing shortcomings. Nowhere in the law is the term failure used and that is not the intent. The goal is to identify schools and subgroups of students who are not achieving and to help them.
Listening to some of the commentary about this law, one would be excused for believing that there are no low-performing schools. The fact is that while we have a public education system that Americans can take pride in, it is not uniformly strong.
Schools serving a lot of low-income and minority students too often are not getting the job done. That's why we have such a troubling achievement gap that manifests itself in a whole host of problems for low-income and minority Americans.
Many parents will not even notice the AYP system in their state because many public schools are doing a pretty good job. To the extent that high performing public schools get identified as needing improvement it will most likely be because one group of students in the school, for example minority students or English-language learners, are not making progress.
That doesn't necessarily mean that their school has an overall problem but it does signal a problem for a specific group of youngsters.
It's incumbent upon those criticizing AYP to show why identifying and remedying such problems is a bad idea. Unfortunately, if the leadership of a particular state wants to make a political point by identifying as many schools as possible for improvement, it is entirely possible to do that by designing an inflexible AYP plan.
That's the worst-case scenario, but in our decentralized education system, where we place a premium on localized governance decisions, it's a risk that can only be avoided by doing nothing at all.
The best-case scenario, sadly overlooked in most of the hysteria about this law, is that for the first time national education policy includes transparent accountability provisions for low-income and minority students and that their educational needs will finally get the attention they deserve. That's why proposals for changing AYP predated this new law.
Democratic representatives George Miller and Dale Kildee proposed similar changes several years ago and senators like Joe Lieberman, Evan Bayh, and Jeff Bingaman have been beating this drum for some time as well. In addition, civil rights groups like the Education Trust and Citizens Commission for Civil Rights are staunch supporters because of the benefits for minority students.
JERRY: Andy says that "claims that federal officials will be going into states and declaring individual schools failures are simply not true."
But they already have. In its first, preliminary, pass, the U.S. Department of Education compiled state-by-state lists of failing schools (the official phrase might be, "needs improvement," but the media and most people translated this to "failing schools").
Declaring an entire school a failure using only a single crude instrument--a test--is fundamentally unsound.
The lists revealed the procedure's irrationality. They labeled Michigan the national dunce with 1,500 failures and Arkansas the national genius with zero.
Does anyone believe that this result is meaningful? Silly-on-their-face results like these will lead to reliance on NAEP [a federal test of academic achievement]--which NCLB makes mandatory for all states--removing the pretense that this is a state-level issue.
Once states begin their NCLB testing, different states will have different failure rates, but most schools in most states will fail. A January 13, 2003, article in the Los Angeles Times put the figure for The Golden State at 98%.
The Congressional Research Service, hardly hysterical hand wringers, estimated the national figure at 90 percent.
Andy emphasizes using AYP to identify poorly served poor and minority children, but if 90% of all schools flunk, the measurement is at best meaningless. In fact, AYP is a mean-spirited device to make public schools look bad and, thus, to elicit more calls for vouchers and privatization.
Andy says the more serious consequences of AYP failure kick in only after two years. One can only wonder what he considers serious. Schools that fail to make AYP in any one year must offer students the option to transfer elsewhere.
The Department has awarded millions in grants to publicize this option. Regulations forbid a school to say, "We're full," and refuse transfers. It must increase class size, bring in portables, or build new classrooms--and hire new teachers (who must be, according to NCLB, 'highly qualified').
The schools can refuse transfer students only if they would so crowd the building as to violate fire or health codes. The district must pay for all this 'capacity building,' as the Department calls it, making NCLB, in a time of terrible budgetary crises, yet another unfunded mandate.
With a 90 percent failure rate, we could, theoretically, end up with 100 percent of our students attending schools currently attended by 10 percent of our students. (No one has yet dealt with how the transfers affect the schools. If they are low-scoring students, the sending school's scores automatically rise while the AYP of the receiving school is put at risk).
Although Andy asserts that safeguards prevent reporting of statistically unreliable subgroup data, even school-level scores exhibit great volatility from year to year, most of which is not associated with the quality of instruction. Subgroups will no doubt show much more volatility.
Rather than identifying good and bad schools, AYP will arbitrarily identify lucky and unlucky schools.
Reporting scores by ethnicity and socio-economic status is a good idea. Attaching sanctions to the results is not. The sanctions will be arbitrary, capricious and unfair and won't work.
ANDY: Jerry makes several false assertions. First, the lists of low-performing schools he refers to are based on information that states report to the U.S. Department of Education based on their standards, assessments and existing AYP provisions.
The states decide what schools need improvement, not the feds. There is an enormous difference between this and the idea that the feds are picking schools out for improvement.
These state lists pre-date the Bush administration, and during his second term President Clinton issued an executive order to get more help for these schools.
Concerning the disparities among states and the oft-cited Michigan-Arkansas example, that's precisely the reason for the changes to AYP. Arkansas is most likely underreporting and Michigan may well be overreporting. Neither scenario is good for students.
With regard to the Congressional Research Service report, Jerry omitted several relevant facts. The CRS report is heavily qualified and clearly states that the analysis is not generalizable, beyond the three states they looked at, to the nation as a whole. Jerry's ascribing that data nationwide is not supported by the study. Specifically, the researchers only had complete demographic information to base their estimates on for one of three states they examined, only had two years of test score data (states can average up to three years), and could not take into account all the variances in the ways that states can design these plans.
Most importantly, the CRS researchers are excellent analysts but at the end of the day they were modeling potential results rather than analyzing actual results. So while their work offers useful information for policymakers, what will be more valuable during the next few years is the actual experience of states as they implement the law.
Hardly anyone argues that some revision might not be necessary, but it's simply too soon to tell and wild speculation is irresponsible. Jerry also misrepresents the AYP provisions with regard to the number of years a school must fail to meet AYP before any consequences come to bear. He writes that, "Schools that fail to make AYP in any one year must offer students the option to transfer elsewhere." That is simply not what the law says. A school must fail to meet an AYP target for two consecutive years before any action is taken and the first year of identification entails minimal consequences, namely offering public school choice and developing a plan for improvement. Schools can also appeal these determinations if there is evidence that they are in error.
It's rhetorically catchy but untrue to say that schools must be held accountable for a single year's test scores. As with many components of this law, the public school choice provisions will require creative thinking and moving past tired orthodoxies.
Public charter schools, more intra- and inter-district public school choice options,and other strategies will be needed.
Let's not lose sight of what's important here, giving students, particularly disadvantaged students, access to high quality public education options. Some of the loudest champions for the public school choice provisions were civil rights groups like the Citizens Commission for Civil Rights. These are not anti-public school groups; they want what is best for kids.
JERRY: The "new" AYP might need two years of data, but transfers are happening now when the new law is only one year old. Reading Andy's description of the reform process, one finds a remote, abstract view of schools from afar.
Andy conveys the idea that a kid is a kid is a kid and a school is a school is a school. They are not. And requiring school choice is not a "minimal" consequence.
Schools are communities. They are organized around the needs of that community. People in the community become vested in their school. Hence the adage, it is easier to move a cemetery than to close a school. Indeed, the January 16, 2003, edition of The Washington Post carries a story that speaks to schools and communities and schools as communities. Some people want one Arlington school, now a magnet school for the entire county, restored to a neighborhood school. Apparently this is a nationwide trend.
When the transfer aspect of NCLB came up on a listserv, one commentator reported that the receiving school in her town does not have a free breakfast program, an in-school medical clinic, remedial reading, study halls, support for ESL students, etc. The teachers at the receiving school have been successful at teaching academically successful kids, but have no experience with low-scoring kids.
The receiving school is described as "scrambling."
One wonders if those transferring to the receiving school will be perceived as "intruders" into the community and how this will in turn affect the perceptions of those who transfer. The process is a recipe for chaos. And all this for less money than the law might cost the states to implement.
The U. S. Department of Education pays much lip service to "scientifically-based research" and the use of proven education techniques. There is no research on, much less any proven benefit of having kids transfer from school to school based on the test scores of one school. Former assistant secretary Susan B. Neuman declared "our children are not laboratory rats." NCLB makes them precisely that. At most, this should have been tried out on a small scale, not crammed down the nation's throat totally untested, higgledy-piggledy (and, despite Andy's claims that it's a state issue, it is a program imposed by the federal government).
The CRS study might lack generalizability, but failure estimates from elsewhere are all consonant with the CRS results. I already mentioned the 98 percent figure in California. The state department in Washington says 97 percent; editors at Phi Delta Kappan say in Indiana it's 80 percent, which is a relief since in neighboring Illinois it's 90 percent.
A legislator in Arizona was quoted saying "virtually all." In North Carolina, characterized by many as a state making much progress, 51 percent of the schools that the state had singled out as making exemplary progress were labeled failures under NCLB.
Andy so far has also ignored the technical problems I mentioned with using test scores to determine failure and the irresponsibility of using only test scores for this determination.
ANDY: The main thrust of Jerry's comments shed important light on the hidden nature of many education debates. Jerry is talking mainly about consequences for the system and for schools rather than about kids. It's precisely because kids are not all the same that we need more options and customization in the public sector, including more choice among public schools for parents.
As for the specifics of why students are being offered a right to transfer now, again Jerry is playing fast and loose with the facts. Many of the provisions of this law, including weaker AYP provisions, were part of the previous law.
Schools that were already identified as needing improvement (some that have been on their states low-performing list for years) are now required to offer public school choice under the new law.
That doesn't mean it's based on a single year test score as Jerry tries to suggest, but rather that parents are being offered public options based on multiple years of low-performance by some schools.
Jerry's account suggests that these school transfers are mandatory and that parents are being forced to switch schools. All the NCLB public school choice provisions stipulate is that if a state identifies a school as needing improvement, then parents must be offered the right to transfer to other public schools.
It's not required. If parents are satisfied with the school they likely won't go. But why is it not okay to offer other public options in this situation? Jerry provides one answer: that poor kids going into other schools can cause "chaos" or "scrambling."
That's an odd argument on the heels of jibes about an "abstract" view of kids and schools.
Like many I have some concerns about the Bush administration's interpretation of "scientifically valid research;" however, there is plenty of evidence to support the notion that we should do a better job economically integrating our public schools.
Public school choice is a key strategy to accomplish that. For the best summation of the evidence, Rick Kahlenberg wrote an excellent book about it a few years ago called "All Together Now."
Public school choice and charter schools are not a panacea but the transparency they bring to school finance, particularly when coupled with the data from state accountability systems, is one of the most promising ways to drive fiscal equity and improve educational quality for poor youngsters. In addition, those of us who argue against vouchers because private schools can pick and choose among students cannot simultaneously say that public schools should do the same when it comes to poor kids.
Returning briefly to AYP, these provisions are based in large part, but not solely, on test scores. While test scores are not an ideal measure, they're the best at hand right now.
Again, we simply don't know how the AYP numbers will play out because states are in the process of designing new systems. It's worth taking the wildest claims with a grain of salt and other states are coming up with more plausible estimates.
Nevertheless, it's pointless to sit here and toss out numbers, we'll be able to have a much more informed debate in a few years. Remember the horrific forecasts that greeted welfare reform and any other substantial social policy shift.
Right now it's best to heed the counsel of the wide range of analysts, researchers, and scholars who, while often disagreeing on the exact structure of state-level accountability systems, see their benefits for poor and minority students far outweighing their costs.
JERRY: Reading Andy's comments, three words come to mind: glib, unresponsive and disingenuous.
For instance, I have raised technical issues about tests (I could have raised many more) that compel the conclusion tests are too weak to bear the weight they're being asked to take on. Andy's glib, indifferent comment: "While test scores are not an ideal measure, they're the best at hand right now." The seriousness of the problems I raise and the unresponsiveness of his "reply" taken together suggest that he doesn't know enough about the technology of testing to actually understand what the problems are.
Andy somehow manages to tie NCLB to Rick Kahlenberg's ideas for integrating communities not by race but economically. That's amazing. [In a private communication, there would be an expletive between "That's" and "amazing."] I happen to think Rick's proposals are unworkable in the large, but they are at least laudable. But Rick's examples of communities that manage economic integration involve the communities doing it "All together now" as in the title of his book. It's a community effort (and they are quite atypical communities).
There is nothing communal or shared about NCLB. It's an imposed, underfunded federal mandate (an analysis in Vermont indicates that the Green Mountain State will get $51.6 million from NCLB and spend $158 million implementing it. Off hand, I can't see how a program that costs three dollars for every one it gives is going to do much for anyone).
Andy says, "We have systematically failed to educate minority students." Well, then, why do we need a huge testing program for virtually all students? NCLB is a punitive testing program, not an educational program. We already know where the bulk of the achievement problems are. Report test scores by subgroup to reveal the few other problematic areas. (Andy wondered if "Jerry had even one good thing to say about it" [NCLB].
Disaggregation by subgroup. That's the one good thing). Put resources where they're needed, and make them real resources, not the picayune drops-in-a-bucket available from NCLB.
Of course, if your real goal is to destroy the public education system, then NCLB in its present form serves you well.
ANDY: The only issue that Jerry has raised thus far with regard to test scores is the volatility of year-to-year scores because of random error. Here's a very basic primer on testing to hopefully help readers understand a little of what this debate is about.
When a test is given to students it yields information. There is a true score, the true performance on the particular test, which we can never exactly know. That's because what we really see is an observed score, made up of the true score along with systemic error and random error.
Systemic error is error endemic to the test instrument. For example imagine a car that is out of alignment and always pulls left, that's a systemic error. Random error is caused by non-systemic factors that nonetheless obscure the true score.
Imagine a car with a broken axle that pulls wildly in different directions. For students, random error is caused by a whole host of factors, distractions in and out of the classroom, occurrences at the school, or even a fire truck racing by during the test.
While (hopefully) addressing systemic error, quality tests cannot, obviously, eliminate random error each time a student takes the test. Among other reasons, this is why it is a bad idea to base decisions about individual students (or schools) on a single test score and why the law does not require this.
It's precisely to mitigate randomness associated with single-test scores and small subgroups of students that the bill gives states flexibility about designing their AYP system. States can average test scores over multiple years, across grades, and ensure that subgroups are large enough to yield sound information.
Schools can also appeal an AYP determination they believe is incorrect. Thus, rather than an arbitrary measure, in a well-designed AYP system, determinations about school performance that carry consequences will be based on a convergence of evidence.
This is worth explaining, not in response to Jerry's juvenile gambit, but rather because it is important to understand what AYP is and is not.
Standardized tests are far from a perfect way to measure school performance and a major concern about this bill is the amount of testing that it requires in the face of an uncertain capacity to deliver it in a high quality way.
However, let's not let the perfect be the enemy of the good.
Broadly speaking, the public seems to get this too because the sort of accountability plans states will develop enjoy consistent support as polls and surveys have shown for some time.
It's worth returning to the discussion the Kahlenberg book. As I read Jerry's earlier statement he argues that there is no evidence that allowing students to transfer to better schools will enhance educational opportunity and in fact seems to argue that it might not be good for them. This is ludicrous.
More often than not, the students in question will be low-income students transferring to more affluent schools, and there is evidence this will help their education.
I too question the viability of Kahlenberg's remedies but his book is one of the most exhaustive and accessible discussions of evidence about socioeconomic status and education. It's worth mentioning that allowing parents to choose schools within the public system can actually enhance rather than decrease the sense of community that Jerry and I agree is important for good schools.
There is more to NCLB than just the provisions we've mentioned thus far. The new law included provisions to better target funding to the poorest students, thanks to the tireless work of Sen. Mary Landrieu (D-La). Doing this might seem obvious, but politics make it very hard to get education funding to low-income communities.
It also includes more flexibility for local educators about spending federal education dollars.
Finally, as part of AYP it has more concrete provisions about helping low-performing schools improve. The bill does contain some nuisances and silly provisions about social issues and there are some items, for example the supplemental services provisions, which I think are quite problematic as currently written. However, legislation of this size requires a lot of compromises on Capitol Hill. That's the nature of policymaking. Overall, though, this bill is a step forward.
INDEX OF NCLB OUTRAGES