Orwell Award Announcement SusanOhanian.Org Home


Rational Responses to High-stakes Testing and the Special Case of Narrowing the Curriculum

Publication Date: 2009-09-15

Citation: Paper presented at the International Conference on Redesigning Pedagogy, National Institute of Education, Nanyang Technological University, Singapore, June 1, 2009.
An earlier and slightly different version of this paper was given as: Berliner, D. C. (2009, April 14). The Centrality of Curriculum in Contemporary Educational Psychology. Sylvia Scribner Award address, presented at the meetings of the American Educational Research Association, San Diego, California, April 14, 2009.

Note: References are cut short because of space limitation. Tables and charts and full references are available here.


My views are a critique of contemporary American education. I am hoping that some of what I document and opine about has relevance to the educational system here in Singapore, but you will have to be the judge of that.

I will start with a quote from one of America’s founding fathers, John Adams (1780), who was also the second president of the US and a school teacher, as well. In a May 12th, 1780 letter to his wife, Abigail, he said:


I must study politics and war that my sons may have liberty to study mathematics and philosophy. My sons ought to study mathematics and philosophy, geography, natural history, naval architecture, navigation, commerce and agriculture in order to give their children a right to study painting, poetry, music, architecture, statuary, tapestry, and porcelain.

This is the perfect quote to begin a paper about curriculum for American and Singaporan schools at the start of the 21st century. For Adams, the ultimate goal of education in a free society was to foster the arts. War was a necessity so that his children had freedom. Freedom was necessary to generate wealth by means of the commercial and agricultural skills that the new nation possessed and to do so independent of European powers. Adams knew, however, that the ultimate goal of education for a free person was the opportunity to appreciate and participate in the arts.

This view of the importance of what is more broadly called the liberal arts is derived from our Greek heritage. The root of the term “liberal” as used in the "liberal arts" actually has little to do with the term "liberal" in the political realm. Instead, the root of the term "liberal" when referring to the arts is in liberty, indicating that art, music, government, rhetoric, philosophy and the like, are the arts of free men and woman. Benjamin Barber (1994) contrasted the liberal arts against what were called the servile arts: business and law enforcement, food preparation and medicine, mechanics and technology. The servile arts encompass the knowledge and skills needed to run the country. This was exactly what the Greeks wanted their slaves and servants to do while they engaged in the kind of grand political and philosophical thinking that gave the Western world its intellectual heritage.

America being a practical country, a land for the promotion of Yankee ingenuity and dominated by the interests of business, never was in awe of the liberal arts or the humanities. There never was a golden age in America in which the liberal arts and humanities flourished while the servile arts were looked down upon. In fact, for at least the first hundred years of widespread public schooling in the US, the practical was deemed appropriate to teach most middle-class white students and was considered even more desirable as the curriculum for the poor and minorities, were they afforded any education at all. The exception to this focus on the practical was for a small and select group of wealthy, white, male students who did get to study the liberal arts in upper high school grades and college.

Today may actually be worse for poor children in the US than at any time in the last half century. This is because the lower classes are being kept from the liberal arts and humanities curricula by design. Using the argument that we must get their test scores up, we in the US are designing curriculum for poor children, often poor children of color but certainly, numerically, for poor white children, that will keep them ignorant and provide them with vocational training, at best. Their chances of entrance to college and middle class lives are being diminished, and this is all being done under the banner of "closing the gap," a laudable goal, but one that has produced educational policies with severe and negative side effects.

What we find today is that America's students are primarily enrolled in courses Adams understood were needed for his children. Our youth are not enrolled in great numbers in the courses Adams wanted for his grandchildren. The theme of this paper is that the curriculum balance, long in favor of the practical and the servile arts, has tipped even further in that direction, particularly in the schools that serve poor and minority children. America apparently has developed an apartheid-like system of education.

The newest difficulty in promoting the arts and humanities in the curriculum is due to the use of high-stakes testing. Such testing is found in all 50 states as a function of the No Child Left Behind (NCLB) act that went into effect in 2002. I will make clear, however, that the decrease in exposure to certain curricula is a rational response to high-stakes testing. But this decrease in exposure to a varied curriculum is of great concern as we contemplate what the 21st century might have in store for our youth. Compared to the past, the future is likely to be more Volatile, Uncertain, Complex, and Ambiguousâ€"A VUCA world (Johansen, 2007) for our children to face. I think adaptation to such a world requires a citizenry with the broadest possible curriculum, not a narrow one that constricts the skills of the youth because of a need to demonstrate accomplishments on a small set of assessments.

Curriculum time and curriculum content are changing and narrowing in the USA as function of over reliance on a small set of assessments. My data come from the Center on Education Policy (2008a), a non-partisan monitor of the effects of NCLB. Table 1 presents their data on curriculum changes from a nationally representative sample of school districts.

Table 1. Changes Since 2001-2002 in Instructional Time for Elementary School English Language Arts and Mathematics, in Districts Reporting Increases (Center for Education Policy, 2008a).

These data show that changes in the time allocated for teaching reading and mathematics in elementary schools were quite dramatic between 2002 and 2007. These are the years of the NCLB act and mandated high-stakes testing. The time allocated to reading has been increased, on average, over two and a third hours a week, while mathematics time has been increased, on average, about an hour and a half a week. What needs to be kept in mind when interpreting this table is that the “average” masks relevant information. It is likely that many school districts increased time in these subjects a great deal more than the average, because the average includes districts serving high-income children, who typically score well on the tests used to satisfy NCLB requirements. Those districts probably changed their time allocations very little. On the other hand those serving low-income students probably changed their time allocations a lot. In a previous study of the curriculum by the Center for Education Policy (2006), 97 percent of the school districts not making Adequate Yearly Progress (AYP) had changed their curriculum times and content. These are, of course, the districts where the pressures to score higher on the tests are greatest and where you expect rational responses to that pressure.

Clearly, the changes documented above may not be all bad. We can all agree that it is highly desirable that students read and do mathematics well. Thus we might normally welcome such changes unless there is reason to believe that the increased time is used poorly and is having either no effects, or detrimental effects, on students. This is where things get interesting: Evidence exists that this may indeed be the case.

If reading and English language arts consists of too much phonics practice; too much drill and test preparation; too many worksheets for practicing reading skills; not enough writing to express complex thoughts; not enough reading for enjoyment; and not enough reading of academic material to increase vocabulary in order to aid comprehension; then the reading is more to foster the goal of basic literacy and not literacy for its pleasure, or for its value in exploring the arts, the sciences and the humanities.

We are learning that reading that is more critical and emancipatory is not stressed as frequently when tests are highly consequential for schools. Instead, drill on test formats and items suspiciously like those on the test constitutes too much of the reading curriculum (Nichols and Berliner, 2007). Sadly, evidence exists to support the hypothesis that the increased time spent on reading and mathematics is not helping us make better readers and mathematicians. Table 2 presents gains made in reading at the 4th and 8th grade on the National Assessment of Educational Progress, tests we call the NAEP tests. These are considered by many to be the best tests the USA has, and they show quite clearly that the gains made before NCLB were greater than the gains made after NCLB. Despite large increases in time allocations for reading, greater growth in reading is not observed.

Table 2. 4th and 8th grade reading gains and rate of gain before and after NCLB became law.

A second look at reading achievement and the effects of greatly increased reading instruction on the performance of the various US states comes from the Educational Testing Service (Barton & Coley, 2008) and is presented as Table 3. What is obvious is that average scores are not increasing, and many states have actually done worse since the enactment of NCLB.

Grade 8 reading change, 2002-2007* in: States Improving States Unchanged States Doing Worse
Average Score 0 32 12
Percent Proficient 0 40 3
Average score for students in the top quartile 5 27 11
Average score for students in the bottom quartile 5 24 14
*Includes data from states participating in both assessments, as well as the District of Columbia and Department of Defense schools.

Table 3. Changes in average reading score and percent proficient in reading for grade 8 US students on the NAEP tests, 2002-2007.

The evidence is that the schools with the poorest children, and therefore the schools with the greatest likelihood of being sanctioned under NCLB, are those where the reading curriculum in now often of the most basic type. While such a level of literacy might have been good enough at the beginning of the 20th century, it is hard at the beginning of the 21st century to defend the forms of instruction used and the kinds of literacy attained by the children in many of our poorest schools.

Some of our psychological theories are appropriate to interpreting these data. For example, although we have argued in our journals over the power of the undermining hypothesis, it does appear to be a replicable experimental finding. We now know, therefore, that for many children the motive to engage in activities found pleasurable for their own sake is diminished when those same tasks are rewarded. This suggests that a significant number of poor and minority children who really do enjoy reading for pleasure and edification are much more likely to be turned off of reading because reading has become a task governed by extrinsic rewards. In many schools with the poorest students stars are awarded for rather trivial multiple-choice questions answered correctly about books just completed. There are other rewards for competitive reading, which appears to me to be akin to a new school sport, where rewards are given for the number of books read per unit of time. Other schools have class parties for high numbers of books read collectively per unit of time. None of these approaches is wrong from a behaviorist theory, yet all of these short-term motivational strategies are likely to have a negative influence on continuing motivation to read. We don't know this, of course, because we usually do not study the long-term effects of these programs. But there is good reason to believe that continuing motivation to read will suffer under some of these instructional programs.

There is another theory in our field that comes to mind when looking at these data. It is related to time and learning. I did some of that research myself. From all the research, and from the common sense that is found in the humblest of homes, we have been able to derive a sound educational law, namely, that the more time students spend studying in some area of the curriculum, the more likely they will have learned more in that area. Time and learning are believed to be, and are empirically found to be, causally related. But this principle of learning is directly challenged by the reading data we have. Significantly more time spent in reading is leading to less improvement on the high quality assessments of reading that are used, the NAEP tests. This suggests that students may be studying the wrong things, or that their motivation is being undermined, or both. This is not good.

What do we know about mathematics? As noted in Table 4, we see almost the same things we noted when looking at reading. The exception is eighth grade gains for Hispanics. All other cells show a pattern of higher gains before NCLB and the additional time that was allocated for mathematics instruction.

Table 4. 4th and 8th grade mathematics gains and rate of gain before and after NCLB became law.

Actually, we learn from the 41 states for which there were complete data at the 8th grade that mathematics scores overall did go up quite a bit over time. Its just that in 24 of those states the gains were larger in the three years before NCLB than in the 4 years after NCLB. So NCLB seems to improve the gains in achievement in fewer than 50% of the states. And in about 10% of the states for which we have data NCLB has no discernable effects at all. So in mathematics, as was true in reading, NCLB does not seem a sensible social policy.

While rarely taught as well as the experts would like it to be taught (c.f. Ball, Lubienski & Mewborn, 2001; Lampert, 2001), mathematics can be even more boring and inadequately taught than ever before under the threat of sanctions. Mathematics, can [be] a subject that is a rich source of discourse and debate, of conjecture and the testing of ideas, and even an important contributor to democratic practices (Ball & Bass, 2008) when taught correctly. But like reading it can be turned into a drill oriented, teacher dominated subject in which the increased time results in increased boredom and dislike of the subject.

Moreover, in a recent analysis by Jaekyung Lee, in RER, he pointed out that the achievement gap between wealthier and poorer students has not been closing at all on the National Assessment of Educational Progress, the best audit test America has (Lee, 2008). This all suggests that something is quite wrong. Increases in allocated time ought to result in increased learning. But if the increased time for learning reading and learning mathematics results in a less interesting curriculum for teachers to teach, and for students to learn, then the results we are getting are actually quite sensible, though certainly quite disheartening. Questions need to be asked about motivation to teach and motivation to learn, and about the attitudes formed toward the subject matter, through the instruction that accompanies the high-stakes testing imposed by NCLB. Such research is specially needed in schools that serve the poor.

I am struck, as always, by Dewey’s prescience and our failure to take him seriously: he said:

Perhaps the greatest of all pedagogical fallacies is the notion that a person learns only the particular thing he is studying at the time. Collateral learning in the way of formation of enduring attitudes, of likes and dislikes, may be and often is much more important than the spelling lesson or lesson in geography or history that is learned. For these attitudes are fundamentally what count in the future. The most important attitude that can be formed is that of desire to go on learning. (1938, p. 49)

Other odd data about contemporary curriculum exist. For example, England has also tried to reform its schools through accountability plans that are heavily test oriented. And it seems to have problems similar to the US. In fact, for the first time since IQ tracking was started, in which an upward trend has been the norm for decades, IQs appear to be declining quite dramatically in UK middle grades. And this has occurred in a relatively short amount of time, according to James Flynn, who gave us the Flynn effect and monitors these trends worldwide (Flynn, 2009). In truth, no one currently has a good explanation for this downward trend. But at least one reason given is that the UK has become a test-oriented culture and this has stunted the intellectual growth and motivation to learn among British students.

Following up his own research of the 1970s, the British classroom researcher Maurice Galton found that teacher-centered pedagogy, characterized by interactions of a very low cognitive level, managerial in their intent, had increased dramatically between 1976 and 1996. Pupils had fewer opportunities to question or to explore new ideas after the tests became the primary instrument that the government used to change the schools (Galton, 2007). Assessment pressures have resulted in 42% of teachers’ time being taken up with whole class teaching, compared to 18% in 1976. In primary schools in England there are now few opportunities for expressing anything that resembles creative reasoning (Galton, 2007).

Galton and McBeath (2002) surveyed primary teachers in England. Teachers in their study regret that time pressures no longer allow them to engage in informal conversations with individual children during lessons, or to allow pupils, at certain times, to pursue their own ideas and interests as part of topic and project work. The British data tell us that since the seventies this time has decreased by nearly 50%. Yet, teachers regarded exchanges of that kind as highly rewarding and motivating because they greatly enhanced the teacher-child relationship and provided what some classroom practitioners described as 'magic moments.'

Galton and McBeath (2002) quote teachers. For example, a female with 23 years experience says:

Too often the subjects like art, and history and geography and the subjects that children really enjoy, and P.E., are squeezed out and those children that are not academic are not getting a chance to shine. We are actually turning them off education rather than actually encouraging them to want to improve the things that they are good at because we’re not actually finding out what they're good at any more.


Another female teacher with 9 years experience says:

Everything else has suffered hasn't it? From my point of view I think probably the saddest thing is the arts being elbowed out. I just find that to be a subject that lots of children who don’t achieve particularly academically achieve wonderfully well at the arty crafty sorts of activities. And they’re not always able to express it, because we don’t have the time to dedicate.

Galton and McBeath (2002) report that many teachers noted the creative subjects were being squeezed out, with the consequence that there were fewer opportunities for children to be good at something, to succeed or to excel, and the teachers knew that this was not good for the children and it made classroom management all that more difficult. The emphasis on the core subjects, with increased focus on content, simply meant that there was less space in the school day for less structured activities, though it was in those kinds of activities that some non-academic children excelled.

The British and US experience is exactly what Hong and Youngs (2008) report happened to curriculum in their study of Chicago and Texas, as that district and that state responded to high-stakes testing. In Chicago the researchers found that high-stakes testing seemed to narrow the curriculum and make it harder for students to acquire higher-order thinking, writing, and problem-solving skills. In Texas, it was found that schooling changed in ways that emphasized rote learning, not broad intellectual skills (Hong & Youngs, 2008; McNeil, 2000). Lipman (2004) also studied the Chicago schools and reports that the accountability program insured that the more affluent students in Chicago received a much richer and more intellectually challenging curriculum than did the poor children in Chicago. Poor minority children, in particular, were required to memorize fragmented facts and information, and they were constantly taught simple test-taking techniques. Lipman is probably quite right when she says that this differential access to high-quality curriculum will have significant consequences in terms of the social inequalities we will observe in the future. White students who possess a great deal of the cultural capital valued by schools are going to be much more likely to get to college and thus more likely to attain higher status through higher paying jobs. But low SES and minority students in Chicago's schools are much more likely to end up in lower-skilled and lower-paying jobs. The decisions about curriculum and instruction in Chicago and other urban districts results in access to rigorous curriculum for some, but not for others, thus allowing for the continuation of the current unequal social structure. Apartheid is not a word I like to use, but it is descriptive of what has happened in many schools across the nation.

Let me give you an example of how this plays out in a high school language arts curriculum for low income Latinos. This is from our book, Collateral Damage (Nichols and Berliner, 2007). This is a transcript of a lesson recorded by my student Sandra Foster (2006), in a Texas high school as the students prepare for the TAKS testing-- the Texas Assessment of Knowledge and Skills. The setting is an English class with 16 Latino juniors. A young Anglo teacher has handed out an essay similar to those that are to be written on the test.

T: Okay, this is last minute work for TAKS. You can pass the test. You don’t want to take it again, right?

S: No response
T: You don’t want to have to all practice again, right?
S: No response
T: Please say yes.
S: No response.
T: You are brilliantâ€Â¦.The test is not hard. Take your time; in fact take all the time you need.
S: No response.
T: Remember, be here for the test and give it all you gotâ€"take you’re time and do your best.
S: No response
T: You’re awesome. If someone tells you differently, don’t listen to them. Don’t be afraid to use those dictionaries. If you don’t know a word, look it up. If you want to use another word, look it up
S: No response.
T: Okay, there will be three types of open-ended questions and three types of literary selections. What does literary mean?
S: No response
T: Is it fiction, non-fiction or biography?
S: No response
T: Are you going to talk to me or you don’t know?
S: No response
T: Its fiction you all (in an angry voice). (Pause) First thing you do is answer the question. It must be insightful and thoughtful. Do not restate the question. You have five lines to fill in. Then you have to support a response. If you summarize in an open-ended question you get a zero. But if you use support for the passage, you get points. Look at this essay. Do you see how this student used textual support?
S: No response
T: Come on! (in an angry voice and shaking her head signaling "no"). Oh, before I forget, it is important that you must stay inside the box and you must use all five lines.
S: No response.
T: See how this student answered the question with insightful evidence?
S: No response.

And so it goes. Another exciting day at school marked only by passive resistance to what is accurately perceived to be an inferior education by these students. Performance on high-stakes tests determines the curriculum choices being made and those choices, paradoxically, may actually result in lower gains on the tests.

In many schools, particularly schools like this one that serve America’s poor, an impoverished system of instruction is now the norm. Galton reports that things in England are no different. In year 6, when national high-stakes tests are given in May, at a school with two classes, the teachers broke up the two classes into three classes, the children who they thought would do well, those who need help, and the "no-hopers," those who teachers believed would not do well. One teacher took the high performers, another took those that needed help, and they worked a lot on test preparation materials while subjects like the arts were cancelled for them. The "no-hopers" were assigned a classroom aide for the months leading up to the test. Social class distinctions in the three different school classes were as you might expect.

Figures 1 and 2 present data related to the British, Chicago and Texas studies, from another report by the Center for Educational Policy (2008b). These data come from a case study of NCLB implementation in Illinois. What is obvious from the two graphs is that the nature of school work differs quite a bit in schools that are under scrutiny for failing to make Adequate Yearly Progress (schools needing to make improvements), and those that are not working under such pressure.

Sadly, in assessment oriented cultures like the US, it is convergent thought and mimetic processes that are valued and it is these styles of teaching and learning that we see a lot of in schools that are not scoring at the appropriate levels. But as we think deeper about this, the narrow, test-compatible curriculum extant in many classes, particularly the classes for the poor, may actually be least likely to change the constructs underlying the tests of achievement (or intelligence) that we use to assess the outcomes of schooling. While you in Singapore are aware that you may have to "teach less to learn more," we in the USA are demonstrably teaching more to learn less! It appears to be self-destructive.

The rest of the curriculum.
Time added for reading and math, in the typical 6.5 hour day, and the
typical180 day year, needs to come from somewhere else in the curriculum.
* not enough data
Table 5. Decreases in instructional time for various curriculum areas to accommodate increases in time for English language arts and mathematics. (Center for Education Policy, 2008a)

Table 5 makes clear where that time has come from. Again it is important for interpreting this table to remember that the average masks the bigger cuts that some districts have made in these subjects. With this caveat in mind we see that the teaching of social studies, intended always to be part of youth development for responsible citizenship, is down, on average, over an hour a week.

Yet Americans of all political persuasions ask that the schools help to develop citizenship. So this trend in curriculum is in opposition to the aspiration that all Americans have for their school curriculum. Furthermore, school programs that might foster citizenship have been cut because of the need for more time in reading and mathematics, so cleaning up neighborhoods and parks, visiting nursing homes, going on field trips to the legislature, projects that examine pollution at a local level, and so forth, all have been jettisoned. Although this nation has never been great at teaching community involvement as a responsibility of citizenship, I expect we will do worse in the future.

Science, a field that probably will be even more important in the 21st century than in the 19th and 20th centuries, is down, on average, over an hour a week as well. Although science is now one of the areas tested under the NCLB law, and is a privileged curriculum area, scores on the science tests do not count toward AYP. Thus, lack of progress in science, and/or low performance on science tests, can safely be ignored by schools and districts since no sanctions attach to the test. Science, like social studies has been robbed of minutes to expand time for reading and mathematics. Thus curriculum that might help insure American economic competitiveness in the future, and surely will contribute to intelligent citizenship in our science- and technology-rich future, has been sacrificed.

Table 5 also documents that time for physical education is down, despite the fact that our youth are more sedentary than they should be, are quite overweight, and Type 2 diabetes is becoming more common. It is easy to argue that physical education is more important today than ever before, and it is acknowledged as one of the most important ways to keep medical costs down as we slowly move to universal medical coverage. Yet physical education is sacrificed for the possibility of a few more points on state tests that have to rise continuously to satisfy the requirements of NCLB.

Not enough data were obtained in this study about the time allocation for lunch. But lunch is obviously wasted time for those who feel the pressure of testing under NCLB. Anecdotally, therefore, it was not surprising that a teacher at a Massachusetts district reported her concern that lunch at her elementary school was less than 15 minutes on many days "so that more time could be put in on the rigorous curriculum areas." "Rigorous curriculum areas" is code, meaning the areas that are tested. Anything else (social studies, history, government, art, music, physical education) has been defined in her school as inherently a non-rigorous subject. The school she reported on had actually abandoned traditional luncheon meals and started serving finger foodâ€"wraps and chicken nuggetsâ€"to get the students in and out of the cafeteria faster! (Nichols and Berliner, 2007).

Nationally, as seen in Table 5 recess was found to be down, on average, about an hour a week. We even discovered, in Maryland, that naps for preschoolers and kindergartners were forbidden by one county school superintendent.

Art and music, nationally, are down an average of an hour a week. This is particularly troublesome because the nation never spent a lot of time in these subjects. You might think that California would be a place that countered this trend, because the arts provide a large source of employment for the state and a great deal of wealth for the nation through the huge overseas earnings from films and videos that greatly reduce our trade imbalance. Yet 89% of California K-12 schools fail to offer a standards-based course of study in all four disciplinesâ€"music, visual arts, theatre, and danceâ€"and thus fall short of that states own goals for arts education. At the elementary level, arts instruction is often left to regular classroom teachers, who rarely have adequate training. In fact, 61% of California schools do not have even one full-time-equivalent arts specialist. Secondary schools are much more likely than elementary schools to employ such specialists, but even when they do, participation rates in secondary arts subjects are low (Woodworth, Gallagher & Guha, 2007).

The defense of the arts can be made on many grounds, but one stands out in terms of the needs of the 21st century, namely, that the arts are alternative ways to represent reality. Ideas expressed through the visual arts, dance, and music are not presented in the verbal or mathematical symbol systems that are in everyday use. So by cutting the arts we limit the ways our students can represent the world in which our students live and about which they may choose to comment. A diminution in curriculum for learning the arts, therefore, restricts our students’ ways of thinking. It limits the possibility for creativity. Yet, according to many pundits, it is creativity that will be the key to economic survival in the 21st century. The arts, by providing ways to think differently about the world, are a way to promote such creativity. And so their defense in the curriculum can be based on economic and cognitive psychological reasons, which add to other reasons for defending the arts as both a natural expressions of our humanity, and for their occasional attainment of indescribable beauty.

The curriculum for the poor
The California study also makes clear that the arts are rationed: They are taught primarily to the wealthy and not the poor.

Figure 3. Percent of California students receiving instruction in various areas of the arts, by poverty level of the school they attend (Woodworth, Gallagher & Guha, 2007)

Figure 3 presents these data. Close to twice as many students in schools that serve the wealthy (low poverty schools) receive instruction in the arts as do the students in schools that serve the poor. This an example of the apartheid system of schooling to which Kozol (2005) refers.

Wealthier students, if they are lucky, will be exposed to a wider range of the arts and humanities in their high schools because the breadth of the curriculum offerings in the high achieving schools has not needed to be cut back. Students in these schools are usually passing their state tests, their schools usually make adequate yearly progress, and their parents have the political power and resources to maintain a broader curriculum. These wealthier students, even were they to miss some exposure to the arts and humanities in the public schools, have parents who pay to provide them with extra curriculum activities (music lessons, drama club, sports), and they are much more likely to encounter the arts and humanities in their colleges. But poorer public school students may not be exposed to the ways of thinking embedded in the arts and humanities at all, and since their college attendance rates are low and getting lower at the most prestigious institutions of higher education (Gerald and Haycock, 2006), poorer students may never get adequate education in the arts and humanities.

Although teachers' voices are often dismissed, surveys of teachers reveal how the NCLB high-stakes testing culture affects the content of their courses. In Colorado teachers say (Taylor, Sheppard, Kinner & Rosenthal, 2003):

"We don't take as many field trips. We don't do community outreach like we used to like visiting the nursing home or cleaning up the park because we had adopted a part and that was our job was to keep it clean. Well, we don’t have time for that any more" (p. 30).

Another Colorado teacher says:

"We only teach to the test even at 2nd grade, and have stopped teaching science and social studies. We don't have assemblies, take few field trips, or have musical productions at grade levels. We even hesitate to ever show a video. Our 2nd graders have no recess except for 20 minutes at lunch." (p. 31)

A Florida teacher says (Jones & Egley, 2004):
"Our total curriculum is focused on reading, writing, and math. There is no extra time for students to study the arts, have physical education, science, or social studies. Our curriculum is very unbalanced."

In Arizona, a teacher of English language learners talks about the test- oriented curriculum and why she might leave the field (Wright, 2007).

"I'm going to get hired by another district to see if it's like this everywhere, because I haven't been teaching really all that long, and if it is like this everywhere, with just nothing but teach-to-the-test type stuff, and to heck with what the kids want to know, then,....I'm not going to stay in the classroom because it just breaks my heart. There are things the kids just want to learn about. You teach them a little bit in these programs, but it's so structured that you don't have time to deviate from the program. I mean, we aren't allowed to have parties, they don't have recess. There is no time during the day where I am allowed to just have fun with my kids and just learn something that is just for fun. And it's really depressing."

The point about learning for fun is perhaps the most important part of this teachers' lament. The ability for students to learn in areas that are of interest to them seems almost unlimited, as seen in their commitment to their hobbies and to acquiring skills in video games. But in this era of high-stakes testing students cannot be allowed time in school to follow their interests. The standards define what students should know at different grade levels, and deviation from that plan is considered dangerous because it might result in missing some items on the states high-stakes accountability test. Of course schools never allowed much time for individualized work, but now even the teachers that made some use of problem-based or project-based leaning, forms of instruction that could ignite students’ interests through a curriculum more personally tailored for an individual, are not allowed to do so. One size of the curriculum is supposed to fit all students. Yet we are reasonably sure that the 21st century economy will require from our work force a broad set of skills, not a narrow one. Thus diversity in the outcomes of the educational system ought to be a goal of American education, not sameness.

Education for a Volatile, Uncertain, Complex, and Ambiguous future world, a VUCA world, would seem to demand breadth of talent in society so at least some of the talents that exist in society would be appropriate to whatever the world brings our way. It is like evolution. If characteristics of the niche that one inhabits change, only organisms that are adaptive will survive. This means that in changeable times variation in talents, like variation in genes, is needed. Identical skills, like identical genes may prove of no value for survival. The emerging VUCA world of the 21st century, in both Singapore and the USA, more than ever before in our histories as nations, requires breadth of talent so that we might posses an adaptable work force. The behaviors associated with high-stakes testing work against that goal.

What Curricula do Americans Want?
Rothstein, Jacobson and Wider (2008) surveyed school administrators, school board members and the general public to see if they prized similar goals for our schools. Remarkably, all three groups performed almost identically, indicating great consistency in contemporary American beliefs about the goals of schooling. Not surprisingly the highest ratings were to the basic skills. But not far behind "basic skills" in the ratings of importance of particular curriculum goals, was the goal of critical thinking. Yet given the high-stakes testing environment, with predominantly decontextualized multiple-choice measures used to asses what has been learned, there appears to be no place in the curriculum to teach, and no way to assess, critical thinking. Thus this major curricula goal for American education, perhaps never taught well, is made even less likely to be included in the school curriculum. This is even more of a problem since the pundits all say critical thinking is a necessary 21st century skill. While we may not know how to teach critical thinking well, we probably do know how not to teach critical thinking well, and apparently have designed just such a system.

Nickerson (1987) offered a set of ideas for thinking about the behavior and characteristics that a critical thinker would display. He would say that the woman or man displaying critical thinking skills would:
  • use evidence skillfully and impartially;

  • organize thoughts and articulate them concisely and coherently;

  • distinguish between logically valid and invalid inferences;

  • suspend judgment in the absence of sufficient evidence to support a decision;

  • understand the difference between reasoning and rationalizing;

  • attempt to anticipate the probable consequences of alternative actions;
  • understand the idea of degrees of belief;

  • see similarities and analogies that are not superficially apparent;

  • learn independently and have an abiding interest in doing so;

  • apply problem-solving techniques in domains other than those in which they were learned;

  • structure informally represented problems in such a way that formal techniques, such as mathematics, can be used to solve them;

  • strip a verbal argument of irrelevancies and phrase it in its essential terms;

  • habitually question one's own views and attempt to understand both the assumptions that are critical to those views and the implications of the views;

  • be sensitive to the difference between the validity of a belief and the intensity with which it is held;

  • be aware of the fact that one's understanding is always limited, often much more so than would be apparent to one with a non-inquiring attitude;

  • recognize the fallibility of one's own opinions, the probability of bias in those opinions, and the danger of weighting evidence according to personal preferences.


  • The list is certainly daunting, but some versions of this is what the public wants from our schools, and so we let the public down by going along with NCLB and not working on these issues.

    There never were many teachers or relatives who modeled these behaviors for me with any regularity until graduate school, and I recall no classes in which these skills were systematically taught. But today I think the situation might be worse, because a curriculum driven by scoring well on high-stakes multiple-choice tests is unfriendly to a pedagogy that uses up time to develop critical thinking. So although a high priority goal for our schools, and considered a “must” for success in the 21st century economy, we have developed a system of assessment less conducive to development of these skills than ever before. Even more sad to me is that I think we have done this just as cognitive psychologists like Deanna Kuhn have demonstrated that we may actually know how to teach one aspect of critical thinking, scientific reasoning. Given the work she and others have conducted, she states "Skilled thinking is fundamental to adults' effective participation in a democratic society, it is measurable, and most important given the proper attention, the evidence indicates that it can be developed (Kuhn, 2009, p. 5). But such an unnatural cognitive skill as scientific reasoning often develops out of projects and debate, for which there is no time in a high-stakes testing environment."

    Supporting that line of research is a recent meta-analysis of planned for, quite deliberate use of classroom time to promote controversy. The work is by brothers David and Roger Johnson (2009). They provide evidence that allocating time in the curriculum for constructive controversy, so that conflicts can be worked on and worked out, promotes many of the skills that Nickerson has identified as characteristics of a person who can engage in critical thinking. As just one example, they find that creativity, as measured by numbers of ideas, originality of ideas, novel solutions to problems, and the like, is a replicable outcome of constructive controversy. But time for those experiences cannot be found easily in the classrooms of teachers whose classes must make yet another few points to show Adequate Yearly Progress on examinations. Controversy, meaning well controlled argumentation in class, teaches students a great deal that is good for their future schooling, employment and participation in democracy. But we rarely see argumentation deliberately used in American classrooms, and I will bet that it is not a prized teaching technique here in Singapore, despite the research base that supports this teaching practice.
    What to do?

    Are there ways to give the public and the pundits what they want? One of the simplest ways to do that is to change the tests used for school accountability under NCLB. Currently almost all the tests used to comply with NCLB make heavy use of multiple-choice items and thus are designed to reward memory of decontextualized bits of knowledge. But we know that tests with high-stakes attached to them drive curriculum and instruction (Borko & Elliott, 1999; Lane, Parke, & Stone, 2002; Wolf & McIver, 1999). So the construction of tests that measure things like creativity and critical thinking need to be designed so teachers have tests worth teaching to. We learned in Vermont, for example, that mathematics teachers increased their emphasis on problem solving and representations of mathematical ideas when the state instituted, for a short time, a portfolio-based assessment system and virtually abandoned multiple-choice testing (Koretz, Stecher, Klein, & McCaffrey, 1994). In Maryland, where they once had a performance oriented, relatively rigorous mathematics test, Suzanne Lane and colleagues (2002) found that teachers changed instruction to genuinely try to increase understanding of the mathematics that was assessed. As measured by the state test they seemed successful, but more important, it even looked like there was transfer to the NAEP tests.

    Simply using tests with open-ended items has also been found to change teacher’s instructional behavior. Under those conditions teachers more frequently required their students to explain their answers in the classroom, and the teachers used more open-ended tests in their own classrooms as they tried to give students experience that would help them on the end-of-year tests (Taylor et al., 2003; Hamilton et al., 2007).

    A substantial set of studies informs us about how we can change things using precisely the same social psychological mechanisms that have messed up the system in the first place. If we could develop assessment items that were worth teaching to we might use the rational responses of teachers to high-stakes testing to affect instruction in more desirable ways. The distinguished psychologist Robert Sternberg (2008) has tried to do just that, so we already have models. Sternberg wanted to assess aspects of cognition that were non-traditional and offers the examples presented in Table 6, 7, 8 and 9.

    If we cared to, in social studies or history, we might assess understanding of the Civil War by asking questions taping:

    ANALYTIC SKILLS: Compare and contrast the Civil War and the American Revolution.

    CREATIVITY: What might the United States be like today if the Civil War had not taken place?

    PRACTICAL INTELLIGENCE: How has the civil war affected, even indirectly, the kinds of rights that people have today?

    WISDOM: Are wars ever justified?
    Table 6. Example of test items in history or social studies.


    If we cared to, in English, we might assess understanding of a novel like Tom Sawyer by asking questions taping:

    ANALYTIC SKILLS: How was the childhood of Tom Sawyer similar to and different from your own childhood?

    CREATIVITY: Write an alternative ending to the story.

    PRACTICAL INTELLIGENCE: What techniques did Tom Sawyer use to persuade his friends to whitewash Aunt Polly’s fence?

    WISDOM: Is it ever justified to use such techniques of persuasion to make people do things they do not really want to do?
    Table 7. Example of test items in English or language arts.

    If we cared to, in science we might ask the following questions to tap:

    ANALYTIC SKILLS: What is the evidence that global warming is taking place?

    CREATIVITY: What do you think the world will be like in 100 years if global warming continues at its present pace?

    PRACTICAL INTELLIGENCE: What can you, personally, do to slow the effects of global warming?

    WISDOM: What responsibility do we have to future generations to act on global warming now, before it gets worse?
    Table 8. Example of test items in Science.


    If we cared to, in mathematics we might ask the following questions to tap:

    ANALYTIC SKILLS: What is the interest after 6 months on a loan of $4,000.00 at 4% interest?

    CREATIVITY: Design a mathematical problem for a 10 year old involving interest on a loan.

    PRACTICAL INTELLIGENCE: How would you invest $4,000.00 to maximize your rate of return without risking more than 10% of the principal?

    WISDOM: Why do states set maximum rates of interest that lenders can charge, and should they do so?
    Table 9. Example of test items in mathematics.

    Sternberg and others provide us ways to make use of teachers' rational (but unfortunate) responses to high-stakes testing in constructive waysâ€"all we need is a solution to the monetary issues associated with such assessments, and even that might exist. We could have teachers score these tests as part of their own professional development, and pay them for that work so that they end up with a higher yearly salary and more experience with the kinds of learning that we want to promote in our students.

    Rothstein, Jacobson, and Wilder (2007) identified six other areas that were all close to being tied for third place among desirable curriculum goalsâ€"after basic skills and critical thinking. Let me briefly describe these:

    Developing social skills. Large percentages of the respondents to the survey wanted to be sure that our schools teach social skills. That has always been a goal of schooling but it takes on even more importance because such skills are predicted to be in greater demand due to changes in the nature of work in the 21st century. But with the current tests and the pressure they engender, project-based, inquiry-based, problem-based cooperative work groups are less frequently seen in our public schools. Instead classes have become more like workplaces where individual efforts are expected and it is individual effort that is rewarded, despite the expectations of the public and the predictions about the modern workplace.

    A 21st century workplace is likely to value such social skills as active and tolerant listening, helping each another to define problems and suggesting courses of action, giving and receiving constructive criticism, and managing disagreements. But in today's high-stakes school environments, collaborative work where such skills can be developed is seen less frequently than ever because such work always means a loss of time that could be used for preparation to take high-stakes reading and mathematics tests.
    The public also wants the schools to develop a work ethic in youth. This is harder to do with some students than others, especially poor and minority students who are often being punished by the high-stakes testing programs because they don’t have the social and intellectual capital of their middle class peers. Poor children come to school on day one, academically behind (Lee and Burkham, 2002; Hart and Risley, 1985). On top of that they lose academic ground in the summer while their middle-class age-mates gain academically (Alexander, Entwisle, & Olson, 2007). In a high-stakes testing environment these poor and minority students quickly learn they are a liability to the schools that they attend, that they "haven't got the right stuff," and that schools are about winners and losers. In such a setting, instead of developing a healthy work ethic many of these students develop despair. They know that they are the "no-hopers" and disengage themselves from schoolwork, often leaving school before they graduate. My own work with Audrey Amrein (Amrein and Berliner, 2002), and the terrific newer study by McNeil, Coppola, Rodriquez, & Vazquezâ€"Heilig in Texas (2008) has shown that an emphasis on test scores in mathematics and reading only, so that school success can be demonstrated, has been found to increase drop out rates substantially.

    Large numbers of students seem to need alternatives to the ordinary curriculum assessed by means of high-stakes testing to show the world that they have valuable talents. If a nations' success in the 21st century depends on a racially and ethnically diverse, highly motivated and highly educated work force, with a wide range of talents, than the emphasis on high-stakes testing may only serve to restrict the economic future of that nation.

    Other curriculum goals show the breadth of what Americans desire. They want schools to develop in students' personal responsibility, an ability to get along well with others, especially others from different backgrounds. Youth were thought to need knowledge of how government works and of how to participate in civic activities like voting, volunteering, and becoming active in communities. The respondents believed our students should receive vocational, career, and technical education that could qualify youth for skilled employment that does not require a college degree. Not surprisingly, the respondents also wanted the schools to provide a foundation for lifelong physical health, including good habits of exercise and nutrition. They wanted our schools to develop in our students a love of literature and the capacity to participate in and appreciate the musical, visual, and performing arts. Finally, in the area of emotional well-being, our students were thought to need tools to develop self-confidence, respect for others, and the ability to resist peer pressure to engage in irresponsible personal behavior.

    Many of these curriculum goals call for both teachers to have time to develop personal relationships with children, and having counselors in sufficient numbers to work with children who need help. The constraints on implementing the very sensible set of curriculum goals expressed by the American people are, therefore, the usual ones: time and money. But instead of saying that's just the way it is, my countrymen and woman should all be asking ourselves how each of us can deal with our complicity in schooling that is so far from the perfectly sensible ideals expressed by the American people?

    Conclusion
    As is obvious, I believe our accountability policies have completely rational but quite serious side-effects. The worst of these may be the unintentional development of an apartheid-like educational system through the withholding of certain curricula and certain instructional methods from the poorest students served by our public schools.

    The same politicians and business persons that want high-stakes testing to be the cornerstone of a school accountability system also want 21st century skills developed. They do not yet understand that they cannot have both at the same time. These are incompatible goals.

    I have shared my concerns about curriculum narrowing, but I also shared with you some data about the aspirations of the American people for the curriculum of the public schools. I don't quite know how to do it but I think more American educators need to join the sensible general public in getting some semblance of balance back into the US school curriculum. Over the next few years those who care about these issues need to work to help stop this pendulum swing toward an irrelevant and an inadequate curriculum for the 21st century economy. Pendulums do swing, and I’d like to help push it back the other way before I am finally too old to push back.

    I have tried to present evidence that high-stakes testing may actually limit the kinds of achievement our youth attain, and thus high-stakes testing may limit each of our nations' ability to compete well in the 21st century. It seems to me that all but the most privileged students come into public schools where the pedagogy may actually be closer to that of the 19th rather than the 21st century. In schools for the poor, Dickens's (1854/1868) wonderfully written caricature of a teacher, Mr. Gradgrind, still lives. Gradgrind said:

    Now, what I want is, Facts. Teach these boys and girls nothing but Facts. Facts alone are wanted in life. Plant nothing else, and root out everything else. You can only form the minds of reasoning animals upon Facts: nothing else will ever be of any service to them. This is the principle on which I bring up my own children, and this is the principle on which I bring up these children. Stick to Facts, sir!
    .........In this life, we want nothing but Facts, sir; nothing but Facts!" [said Mr. Gradgrind as the other adults present] all backed a little, and swept with their eyes the inclined plane of little vessels then and there arranged in order, ready to have imperial gallons of facts poured into them until they were full to the brim.

    But it is not just pedagogy that needs improvement. Many of our students receive too limited a curriculum for dealing with what the eminent psychologist Howard Gardner (1999) reminds us are always the most important questions facing humankind: what is true, rather than false; what is beautiful, rather than ugly or Kitschy, and what is good rather than compromised, or evil. A broad liberal arts curriculum is needed to deal with these eternal questions. But we in the US are far from providing that now, and moving further away from that model as high-stakes testing changes what and how we teach.
    No one really knows what 21st century skills are needed to foster success for individuals and nations. But developing critical thinking, engaging in activities that require problem solving and creativity, and doing individual and collaborative projects of complexity and duration, are all good candidates for helping each child and both of our nations to thrive. We need to keep in mind what has been found from studies of the NELS data, and from longitudinal studies of the effects of high quality pre-school. It is the soft skills that determine success every bit as much as literacy and numeracy when looking at college completion, earnings, and a host of other outcome variables later in life (Deke and Haimson, 2006; Lleras, 2008).

    Each of our nations needs to be reminded that math and reading skills are only necessary, they are not sufficient for predicting success either for individuals or for nations. Thus we need to cultivate our children's talents, whatever they may be, so that they learn how work and success go together in sports, drama, student government, music, geography, computer illustration, computer gaming, fashion design, cooking and more. We need to remember that when administrators and teachers concentrate their efforts on raising only a few skills, they detract from the talent pool for individual and national success in an economy that will demand adaptability and will suffer if we have more uniformity of outcomes.

    I opened this talk with a quote from John Adams, who was actually quite a difficult person, but I will close with a quote from him as well. In 1775, as my wonderful country was about to be born, he sketched out his thoughts on government and said (Adams and Adams, 1841):

    Laws for the liberal education of youth, especially for the lower classes of people, are so extremely wise and useful that to a humane and generous mind, no expense for this purpose would be thought extravagant.

    There is persuasive evidence that, as a function of high stakes testing, a liberal curriculum for our poor, and our not-so-poor, is often denied. Those of us with a humane and generous nature should work to change this. At this time my country and yours are each attempting to change, through redesign, their pedagogical system. But I believe that the US is going in the wrong direction, while Singapore is at least thinking about going in the right direction. But I caution you, that too heavily a reliance on test scores based on multiple-choice items provokes a rational and inappropriate response by educators that is the antithesis of the pedagogy thoughtful educators hope to promote. I do urge you in Singapore to be wary of trying to change your system to go in one direction, while you keep in place assessment programs that provide impediments to achieving your new goals.

    References
    Adams, J. (1780). Retrieved May 30, 2009 from:
    http://www.quotationspage.com/quote/33545.html
    Adams, J. & Adams, C. F. (1841). Thoughts on governement, in a letter from a gentleman to his friend. Page 276. In Letters of John Adams addressed to his wife. Boston, MA: C.C. Little and J. Brown, 1841.
    Alexander, K. L., Entwisle, D. R., & Olson, L. S. (2007). Lasting consequences of the summer learning gap. American Sociological Review, 72, 167â€"180.
    Amrein, A.L. & Berliner, D.C. (2002, March 28). High-stakes testing, uncertainty, and student learning Education Policy Analysis Archives, 10(18). Retrieved January 13, 2009 from: http://epaa.asu.edu/epaa/v10n18/
    Ball, D. L., & Bass, H. (2008). The role of mathematics in education for democracy. In G. Fenstermacher , D. Coulter, & J. Wiens (Eds.), Yearbook of the National Society for the Study of Education: Vol. 107 (1). Why do we educate in a democratic society? (pp. 171-184). Malden, MA: Blackwell Publishing.
    Ball, D. L., Lubienski, S., and Mewborn, D. (2001). Research on teaching mathematics: The unsolved problem of teachers' mathematical knowledge. In V. Richardson (Ed.), Handbook of research on teaching (4th ed.). New York: Macmillan.
    Barber, B. (1994). An Aristocracy of Everyone: The Politics of Education and the Future of America. New York: Oxford University Press.
    Barton, P. E. & Coley, R. J. ( 2008, April). Windows on achievement and inequality. Princeton, NJ: Policy Information Center, Edu


    FAIR USE NOTICE
    This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available in our efforts to advance understanding of education issues vital to a democracy. We believe this constitutes a 'fair use' of any such copyrighted material as provided for in section 107 of the US Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit to those who have expressed a prior interest in receiving the included information for research and educational purposes. For more information click here. If you wish to use copyrighted material from this site for purposes of your own that go beyond 'fair use', you must obtain permission from the copyright owner.