An Interview with Richard Rothstein, Rebecca Jacobsen, and Tamara Wilder about their new book, Grading Education: Getting Accountability Right
This interview provides a good overview of the book.
By Michael F. Shaughnessy
1) Guys, you have just written a book, Grading Education: Getting Accountability Right (published by Teachers College Press and the Economic Policy Institute). What brought this about?
As we looked at research on the effects of No Child Left Behind, it became clear that this form of accountability was not working. Our current accountability systems focus narrowly on standardized test scores in reading and mathematics. This test-based accountability system has only resulted in a corruption of the goals of public education. It has created incentives to focus only on reading and math instruction at the expense of other important goals, including not only other academic subjects such as science and social studies, but also other skills which are less easily tested through a paper and pencil exam, such as students' ability work cooperatively in groups, develop a commitment to civic and community responsibility, and develop an appreciation for the arts and literature.
NCLB has also created incentives for teachers to focus on some students at the expense of others. Because schools are only judged by how many students achieve an arbitrary passing score (called "proficiency) on state tests, there are no incentives to push already-proficient students even further in their academic growth, nor are there incentives to focus attention on those who are so far behind that even intense drilling would not help them pass the bar. Consequently, we see explicit school policies that require teachers to focus excessive attention only on students who are almost at the passing point, because inching them to passing is the only accomplishment which counts in calculating schools' "adequate yearly progress" Ă˘€“ the NCLB requirement.
These perverse consequences are so fundamental to how NCLB is designed, they cannot be avoided simply by tinkering with the law. But with nearly 15% of all tax dollars going to support public schools, it is entirely reasonable that citizens hold their schools accountable for all the goals of public education. Therefore, in our book Grading Education: Getting Accountability Right, we propose an accountability system that combines information from tests of basic academic skills, with a wide array of information from other sources, including tests of reasoning and critical thinking, and evaluations by highly qualified experts who observe schools to determine if they are performing satisfactorily.
2) How unrealistic were the goals in No Child Left Behind?
The main goal of NCLBĂ˘€”proficiency for allĂ˘€”is unattainable.NCLB's expectation of "proficiency for all" confused a minimum standard, one that all (even the least able) students should meet, with a challenging standard, one that requires typical students to reach well beyond their present level of performance.Inevitable variation in student performance makes it logically necessary that all students cannot be at or above a level that typical students find "challenging."
Although NCLB allows states to develop their own definitions of proficiency, the law does demand that all students in each state pass a single challenging standard. This guideline comes from the National Assessment of Educational Progress (NAEP) definition of proficiency as "demonstrated competency over challenging subject matter." However, there are no countries in the world, even the highest performing countries, in which all students are able to reach a challenging standard. In Grading Education we demonstrate this fact by predicting how students in Korea and SingaporeĂ˘€”the top scoring countries on international math assessmentsĂ˘€” would score on the United States' NAEP. We found that over 25 percent of students in Singapore and Korea would score below the 8th grade NAEP proficiency cut score in math.
It is impossible for all U.S. students to reach a challenging level of proficiency by 2014 or by any year.One standard cannot be challenging to most and achievable by all.
3) What kinds of recommendations have you made regarding schools in America?
In our book, Grading Education, we propose an accountability system that holds schools accountable for all of the goals of public education. To do so requires more than just standardized tests scores in math and reading.
Throughout history, American leaders have generally agreed that schools should assist students in developing knowledge and skills in a broad range of goal areas, including: 1) basic academic knowledge and skills; 2) critical thinking; 3) appreciation for the arts and literature; 4) preparation for skilled work; 5) social skills and work ethic; 6) citizenship and community responsibility; 7) physical health; 8) emotional health. When we conducted a survey that asked respondents to prioritize these goals, we found broad agreement both among representative members of the public and among educational leaders that, while teaching basic academic skills is important, it is not so much more important that we should be satisfied if other goals were sacrificed. Consequently, in Grading Education, we propose an accountability system in which schools would be held accountable for student achievement in all of these eight areas. Only with such an accountability system, can we avoid the goal distortion that results from accountability only for a few basic skills. Our proposals include an expanded federal data collection system, covering both standardized tests and performance assessments. Precedent for such a system exists in the early development of the National Assessment of Educational Progress (NAEP). Unlike today's NAEP exam which is almost exclusively a paper and pencil exam, early NAEP also collected data using survey and interview questions, and observations of student behaviors. With the exception of the arts and music exam in 1997, however, such performance assessments were eliminated by the federal government in the 1970s.
In addition to test data and other quantitative measures of student achievement, we propose in Grading Education that these data be supplemented by expert evaluations of schools and student work, conducted on a regular cycle. Even the most sophisticated test questions are not fully adequate to reveal students' abilities. Therefore, mandatory school evaluations should be conducted by professional inspectors to inform the public about how well schools are progressing towards the eight goals described above. Inspectors should observe lessons in every classroom, meet with members of the school community, shadow students selected at random and observe daily school and classroom practices. Observations would enable inspectors to make concrete recommendations based on their expert knowledge, and subsequent inspections could evaluate a school's progress in addressing areas identified as in need of improvement. Schools deemed in need of great improvement should be visited more frequently and schools meeting or exceeding standards could be visited less often. The reports of inspectors should be made available to the public, and if schools repeatedly fail to make the necessary changes to ensure student progress towards the multiple goals of public education, states should intervene and replace inadequate administrators and faculty.
By coupling an expanded federal data collection system with state-directed school inspections, we can be sure that all children are making progress in all the goal areas of public education.
4) Let's take an extreme example. Can we rightly, validly, reliably compare test scores of kids in Billings, Montana to kids in East Los Angeles?
We can compare these test scores, but we cannot expect them, on average, to be the same. Of course, because of the variation in human ability within any group, there will always be some disadvantaged children who achieve at higher levels than middle class children, but on average, there will be an achievement gap.
If you send two groups of students to equally high-quality schools, the group with greater socioeconomic disadvantage will necessarily have lower average achievement than the more fortunate group.
In a previous book of mine, Class and Schools (also published by Teachers College Press and the Economic Policy Institute), I tried to explain why this is so: Low-income children often have no health insurance and therefore no routine preventive medical and dental care, so have more school absences as a result of illness. Children in low-income families are more prone to asthma, resulting in more sleeplessness, irritability, and lack of exercise. They experience lower birth weight as well as more lead poisoning and iron-deficiency anemia, each of which leads to diminished cognitive ability and more behavior problems. Their families frequently fall behind in rent and move, so children switch schools more often, losing continuity of instruction.
Poor children are, in general, not read to aloud as often or exposed to complex language and large vocabularies. Their parents have low-wage jobs and are more frequently laid off, causing family stress and more arbitrary discipline.
The neighborhoods through which these children walk to school and in which they play have more crime and drugs and fewer adult role models with professional careers.
Such children are more often in single-parent families and so get less adult attention. They have fewer cross-country trips, visits to museums and zoos, music or dance lessons, and organized sports leagues to develop their ambition, cultural awareness, and self-confidence.
Each of these disadvantages makes only a small contribution to the achievement gap, but cumulatively, they explain a lot.
A broad and diverse group of distinguished Americans, including Republicans and Democrats, religious leaders, and policy experts from various fields, issued a public statement last year that rejected the expectation that schools alone can close the achievement gap. The statement, A Broader, Bolder Approach to Education, called for supplementing school improvement initiatives with provision of high-quality early childhood care, health clinics in schools, and high-quality after school and summer programs. The statement asserted that only if such policies were combined could we make serious inroads in the "achievement gap." We urge your readers to go to www.boldapproach.org to add their names and voices to this campaign.
5) Can we even compare education in the 1950's (before most of you were born) and education in the 1990's and now as we approach 2010?
The challenges we face today are different, but not as different as many people think. In the 1950s, Americans debated whether schools should narrowly focus on basic academic skills, or stress a broad curriculum Ă˘€“ much as we debate this today. In Grading Education, we describe a commission led by Nelson Rockefeller and Henry Kissinger that issued a 1958 report denouncing excessive stress on tests of math and reading. The report insisted that "Our conception of excellence must embrace many kinds of achievement at many levelsĂ˘€Â¦. There is excellence in abstract intellectual activity, in art, in music, in managerial activities, in craftsmanship, in human rela tions, in technical work." And the report urged that test scores not be the sole mechanism of school accountability. It said, "Decisions based on test scores must be made with the awareness of the imponderables in human behavior. We cannot measure the rare qualities of character that are a necessary ingredient of great performance. We cannot measure aspiration or purpose. We cannot measure courage, vitality or determination."
The work-related skills now required of school graduates are different from those of the 1950s, but also not as different as many people think. In an appendix of Grading Education, we reprint an article I co-authored with Lawrence Mishel, president of the Economic Policy Institute, in which we showed that, contrary to popular belief, American schools currently produce enough high school and college graduates to fill openings in the technologically advanced jobs of the modern era. Of course, we argue, our schools have an obligation to educate every child to his or her maximum potential, and we are not presently doing so, but we should not use flawed economic projections as a basis for denouncing the performance of our public schools.
6) You wrote that the failures of No Child Left Behind have parallels in other fields of social policy. What do you mean? Can you give some examples?
In one chapter of Grading Education, we show that the ways in which NCLB has corrupted American education (described above by Rebecca Jacobsen in her answer to your first question) should have been foreseen, because attempts in other fields to hold public services accountable, using narrow quantitative measures like test scores, have corrupted those fields as well. For example, police commanders have sometimes given police officers ticket quotas in an attempt to ensure that policemen work effectively. Policemen respond to such quotas by issuing more traffic tickets, but this has resulted in police ignoring other important responsibilities that are more difficult to measure, such as prevention of crime by community policing. When medical services have been held accountable, for example, for easily measured reductions in infant mortality, these services may achieve such reductions by shifting resources to hospital obstetric services and away from prenatal care for pregnant women. The services reduce mortality but then have higher proportions of low birthweight and seriously disabled births, because prenatal care was de-emphasized. The federal government has tried to hold job training agencies accountable by measuring how many unemployed workers were placed in jobs. Use of this simple quantitative indicator led agencies to de-emphasize education and re-training, because they got credit for placing workers in low-skilled jobs which might not last very long.
7) What can American school accountability policy learn from education accountability policies in other countries?
The American public has a right to hold schools and other institutions of youth development accountable for achieving adequate and equitable outcomes in the all the goals of American education. As Rebecca Jacobsen described at the beginning of this interview, NCLB's narrow means of holding schools accountable for only math and reading test scores creates incentives for schools to only concentrate on those two goals, at the expense of others.
Other countries have long known of the goal distortion that occurs when educational institutions are held accountable for only some of their many goals. Employing this knowledge in their accountability policies, many countries including England, the Netherlands, New Zealand, Scotland, the Czech Republic, Portugal, and France depend upon a system of school inspections that includes student test scores as only one indicator of performance for holding schools accountable. Although the specifics of school inspections differ from country to country, the principles are the same: employ trained professionals to visit schools to assess if their practicesĂ˘€”curriculum, instruction, facilities, leadership, materials, etc.Ă˘€”are likely to graduate students who will become successful, productive citizens. In cases where inspection demonstrates deficiencies in a school's practices, the responsible national or local agencies intervene to improve the school's methods.
The United States has some of the building blocks in place for such a comprehensive accountability system. We presently have six regional voluntary accreditation agencies for schools. In Grading Education, we recommend that the United States should look to the accountability systems of other countries to develop a mandatory expanded accreditation system, employing trained professional evaluators, to hold schools accountable for achieving all the goals of American education.
8) You say that we should assess more than just math and reading, but aren't other areas much more difficult to assess? Shouldn't we stick to what we know how to do?
Although it may seem that we only know how to assess math and reading achievement, these are merely the easiest and cheapest subjects to assess. We know, and have known since the 1960s, how to measure student achievement in all the goals of American educationĂ˘€”critical thinking, problem solving, science, civics, geography, social skills, health, community responsibility, citizenship, the arts and music, and preparation for skilled work.
The original design of the National Assessment of Educational Progress in 1963 included not only easily standardized paper-and-pencil tests, but observations of student performance and behavior. For example, to measure 17-year-olds' preparation for skilled work, NAEP gave them a job advertisement in a newspaper and asked them to write a letter of application. To assess 13- and 17-year-olds' development as citizens and sense of community responsibility, NAEP asked what they felt they should do if they walked past a park and observed an attendant barring minority children from entering. To measure social skills, trained NAEP evaluators observed whether 9-year-olds were learning to cooperate effectively in small groups. To determine if 17-year-olds were poised to make healthy decisions, NAEP presented them with food choices and asked them to recognize which had fewer calories.
NAEP today is only an assessment of academic skills, primarily basic academic skills, of students in the fourth, eighth and twelfth grades. Yet, we do know how to measure all the goals of American education. If we desire an accountability system that helps schools to provide all children with the skills and knowledge necessary become successful, productive citizens we must be willing to spend the time and money required to assess all the goals of American education.
We would be pleased to respond to further questions your readers have about these issues, and about our book. Your readers can reach Rebecca Jacobsen at email@example.com, Richard Rothstein at firstname.lastname@example.org, and Tamara Wilder at email@example.com.
For future reading some hyperlinks follow :
Grading Education is: http://www.epi.org/publications/entry/books_grading_education
Class and Schools is: http://www.epi.org/publications/entry/books_class_and_schools
A Broader, Bolder Approach to Education is: http://www.boldapproach.org
Rebecca Jacobsen is an Assistant Professor in the School of Education at Michigan State University, Richard Rothstein is a Research Associate of the Economic Policy Institute, and Tamara Wilder is a Post Doctoral Fellow at the Ford School of Public Policy at the University of Michigan.
Michael F. Shaughnessy
INDEX OF NCLB OUTRAGES