Orwell Award Announcement SusanOhanian.Org Home


The Mismeasure of Education: A Review

Susan Notes:

This essay review of The Mismeasure of Education by Jim Horn and Denise Wilburn
does what a good review should do: Offer insights to the reader about an important book. As Garrison points out: Few educational professionals are aware of the force of assessment history and how the current reform efforts echo so much of the driving force of this history. Certainly if teachers read this book they'd better understand the why and how of the current assault on them and the schools in which they teach.

Take a look at other opinions. These blurbs appear on the book:

"When the Obama Administration decided to spend the billions it got for schools as part of the stimulus package to launch the Race to the Top program and the NCLB waivers, forcing many states to adopt teacher evaluation based on changes in student test scores, leading experts warned that this "value added" system did not have a reliable scientific basis and would often lead to false conclusions. This sobering and important study of the long experience with this system in Tennessee (where it was invented) shows that it did not work, was unfair, and took attention away from other more fundamental issues." Gary Orfield Distinguished Research Professor, UCLA, Co-Director, Civil Rights Project/Proyecto Derechos Civiles, UCLA â¨

"If The Mismeasure of Education offered only its penetrating new look at Conant and Coleman, it would be worth the price. But that's just the beginning. Horn and Wilburn uncover the obsessive instrumentalist quantification and apocalyptic rhetoric soapboxed by both liberal and conservative political elites. Their autopsy of value-added accountability reveals the pathology of ed reform's claim about teachers not being good enough for the global economy." Susan Ohanian Educator, Author, Activist â¨â¨

"A well-researched (and frightening) look at examples of shameful pseudoscience in America, the latest manifestation of which is value-added assessment for determining teacher competency... A well-documented and thorough analysis, inescapably leading to the conclusion that student test data cannot be used to determine teacher effectiveness. A must read for policy makers enamored of the idea that value added assessments will do what is claimed for them. They do not!....An excellent and scholarly history of how we got to an educational-testing/industrial complex, now promoting invalid assessment strategies that are transforming education, but not for the better. A scary book that should be thoughtfully read by those who value America's greatest invention, the public schools." David Berliner Regents' Professor Emeritus, Arizona State University â¨â¨

"The Mismeasure of Education is a magnificent work, an elegantly written, brilliantly argued and erudite exposition on why the "what," "how" and "why" of effective teaching cannot be adequately demonstrated by sets of algorithms spawned in the ideological laboratories of scientific management at the behest of billionaire investors... This book will serve as a sword of Damocles, hanging over the head of the nation's educational tribunals and their adsentatores, ingratiators and sycophants in the business community... The Mismeasure of Education will have a profound resonance with those who are fed up with the hijacking of our nation's education system. This is a book that must be read by everyone interested in the future of our schools. It is a book that advocates real educational justice, for student, teachers, administrators and the public; it is informed by impressive scholarship and compelling argument. It is surely to become a classic work." Peter McLaren Professor, GSEIS, University of California, Los Angeles, Distinguished Fellow in Critical Studies, Chapman University


by Mark Garrison

Recently I was asked to talk to a large group of area educators and parents about the relationship between the Common Core Standards Initiative (CCSI) and the use of student test scores on high stakes standardized tests to evaluate teachers and principals-- so-called valued-added models (VAMs). Public criticism of this disfiguring of teacher evaluation and the Common Core testing regime continues to grow across New York State and elsewhere, giving rise to many public forums such as the one described below. More than 40,000 parents reportedly opted their children out of New York's Common Core tests this year.â  Many parents are "refusing the tests" on the grounds that consistent and vocal public concerns about the Regents Reform Agenda have been ignored (e.g., Strauss, 2013).

During the discussion following my talk linking the rise of the test-delivery Common Core regime and VAMs to cuts to education funding and privatization, one teacher likened her experience to being flushed down a toilet, "day after day," struggling but never being able to escape that dark vortex known as "education reform," which, she said, "sucks the life out of education" and renders any authentic work of students and teachers "wasteful." After the crowd had left, the event organizer told me that a teacher sitting next to him during the talk, "cried quietly for the first half hour." When I asked why, he said: "The talk put everything together for her, helped her understand the pain she had been experiencing over the last decade." In no small measure was my ability to "bring everything together" based on having just finished reading Horn and Wilburn's volume, The Mismeasure of Education (MME). Readers should know that MME is imbued with an activist spirit and so it seems imminently fitting to introduce the book in the light of its role in my own work as a public intellectual.

I've been studying and writing about standardized testing, and VAMs in particular, for some time, so I asked myself, "what was it about MME that proved so valuable to me?" The value of MME for me -- and I believe this will be the case for many readers -- is the manner in which it links the rise of test-based accountability policy to elite ideologies and efforts to block public demands for equality of educational opportunity, and demands for policies that foster social equality, more generally. Through its case study of the rise of
VAMs in Tennessee, MME leaves the reader with a keen sense of the dynamic relationship between the increasing reliance on standardized tests, school finance litigations, and privatization efforts, which include increased expenditures on for-profit prisons and a simultaneous reduction in funding for public schools. This analysis and the authors' mode of presentation helped me put all that knowledge together such that I was more able than before to effectively communicate an analysis to a public audience, one that was both partisan -- unabashedly in favor of defending and renewing the democratic potential of public education -- and eager for objective analyses of the actual conditions, developments and facts related to the "mismeasure" of the work of teachers and students.

In short, the value of MME is how it makes the case that testing is political. It makes the case that standards used to judge schools, teachers and students represent the interests and values of those who establish them. For me, it is further proof of the thesis outlined in my own work (Garrison, 2009) regarding the inherently political and value-laden nature of academic standards and assessments and their role in struggles between political factions and social classes. The Mismeasure of Education will help readers understand that fights over testing practices can be understood as means for sorting out larger political and economic contradictions. Horn and Wilburn's work, then, can be summarized as presenting a case history and political analysis of the mismeasure of education. This is an analysis of who does and does not benefit from the systematic distortion of social reality and alteration of the goals and control of public education that emerges from VAMs and high-stakes testing more generally.

An Objective Yet Partisan Analysis

Horn and Wilburn present their argument about mismeasurement in four sections. While each stands alone -- expanding the pedagogical uses of the book -- each section contributes to an overall argument, guided by a definite method of analysis. With its title paying homage to Stephen J. Gould's famous treatise on intelligence testing (1981),â  The Mismeasure of Education is, in the author's words, "a more modest volume," that is focused on "how the vestiges of this dangerous ideology [of eugenics] and this obsessive practice of quantification among education policy people has molded the dominant dogma of testing that has held sway from the very beginnings of educational today" (p. xi).

To guide their study, Horn and Wilburn draw on Flyvbjergâs (2001) notion of an activist oriented social science. His framework highlights the manner in which goals, values, and political and economic interests necessarily shape social life. Thus the study of goals, values and interests must be at the center of disciplined social inquiry. The
authors summarize the approach in this manner:


Flyvbjerg throws down the gauntlet
to all social policy researchers to aim
for an understanding of social
organizations and systems as
products of human actions that occur
within multiple contexts of expressed
power and values, which make the
traditional natural science goals of
"explanation and prediction"
impossible to attain. (Horn &
Wilburn, 2013, p. xiii)

While Flyvbjergâs ideas frame the major questions social science should strive to answer -- where are we going, is this desirable, and what should be done, questions that constitute the basis for the book's organization -- Horn and Wilburn's volume does not suffer, in my view, from what some critics have identified as weaknesses of Flyvbjerg's approach. These include observations that even Flyvbjerg engages in prediction and a form of grand social theory, while possibly assuming a one-sided emphasis of phronesis at the expense of episteme (see Falk & Rocha, 2009).

But Horn and Wilburn make regular use of the findings of "normal" social science that Flyvbjerg challenges; everything from the Coleman Report to psychometric reviews of the validity and reliability of VAMs inform their analysis. The point here is not that these modes of science do not warrant scrutiny, but rather that Horn and Wilburn are able to use them to craft a deeper understanding of the origin of mismeasurement and thus construct a basis for forming alternatives. In fact, Horn and Wilburn repeatedly and masterfully use the findings of positivist social science to expose the political machinations behind the seemingly neutral mantras for data-driven accountability. This accountability campaign is exposed as ideological and political in nature, as it repeatedly ignores or selectively uses the published findings of key major education research efforts.

In short, MME is premised on the idea that one can be both objective and partisan. The authors' urgent analysis is clearly informed by their adherence to principles of equality, democracy and against oppression, inequality and degradation of the natural environment, and these convictions are neither veiled nor something for which they apologize. Just as the values of those who organized VAMs are exposed, the authors are upfront regarding their own commitments. But the aim of such disclosure is to objectively answer questions regarding mismeasurement and its beneficiaries --without an admission that partisanship is itself an objective feature of social life, the analysis would suffer. And while Horn and Wilburn, following Flyvbjerg, reject a narrow model of social inquiry premised on the methods of the natural sciences, it becomes clear to the reader that MME nonetheless offers a grand prediction: if education policy continues on its current path, further harm to the quality and public purpose of education will result.

Readers of this review will benefit from an overview of each of MMEâs four sections. In each, I highlight what I believe to be particularly important points, including how each section might serve as a resource in education-related coursework. I generously quote from the volume to shine a light on the narrative style that I find engaging, honest and direct.

I should emphasizes here that a strength of the book is the wide range of topics and literature that it presents as necessary for understanding standardized tests and their policy use. As a result, the book may serve a variety of purposes in a variety of educational contexts. One caveat is that for the full benefit of the book to be felt in a college classroom, the instructor must have prior knowledge of the varied and multiple strands of literature that the authors draw from. While not dense in its prose, the volume covers a great deal of ground and makes a variety of important connections across a fairly wide body of literature, making the book far more complex and deep than first appears. Most of my students failed to grasp the book's significance, for example, until I reassigned several portions toward the end of the semester following additional lectures and discussions.

Mismeasure, Past and Present

Part 1 is the broadest or most varied section of the book, and requires the most space to properly summarize its features and offer highlights. It stands as a useful introduction to a variety of issues surrounding current policy debates and may serve as an introduction to the politics of testing for some, or a refreshing means for knowledge integration, for
others.

Part 1 is guided by the question, "where are we going with educational assessment in the United States?" To answer this question, the authors argue, "we make the case that the succession of education reforms during the past century represent the institutionalization of an historically repetitive motion that serves to reproduce dominant power relations and values" (xiii). Testing and quantification, not surprisingly, are the indicated key "repetitive motions". Importantly, however, Horn and Wilburn begin this history from the present with an overview of the Obama administration's Race to the Top (RTTT) grant programs. Starting from the present helps orient readers to see the significance of the past for understanding present practice.

Significantly, for it is unlike other contemporary analyses (e.g., Manna, 2011) the
overview of RTTT highlights the role of the Bill and Melinda Gates foundation in setting the stage for Tennessee -- the "case" that is the subject of much of the book -- as an unlikely winner of RTTT grants, given its consistently low performance on accepted measures of achievement. A fairly lengthy quote here will give readers a sense of the book's tone as well as a sense of how the authors integrate literature and information. They write:

In Tennessee, the Gates influence
was no joke. Tennessee's education
reform history since 1980, along
with its use of value-added testing
(Tennessee Value Added
Assessment System [TVAAS]) since
1992, had not escaped the attention
of the BMGF [Bill and Melinda
Gates Foundation]. Having satisfied
all of the BMGF reform criteria for
favorable consideration for grants,
including the demand for "no
firewall barring the use of student
achievement data in teacher
evaluations" (McNeil, 2009, para. 4),
Tennessee, indeed, was one of 15
states hand-picked in August 2009 to
receive a $250,000 Gates Foundation
grant to help prepare the lengthy
RTTT applications, which BMGF
personnel had helped to construct.
Then, in addition to receiving
assistance from the BMGF, the
organization that had helped
establish RTTT goals, the criteria for
selection, and the application
assistance grants, Tennessee recently
received another plum advantage
from the BMGF during the frenetic
weeks leading up the announcement
of winners in the RTTT contest. In
November 2009, Gates had
established a beachhead in Memphis
City Schools, with $90 million in
grant money "to improve teaching in
tie district" (Roberts, 2009). The new
program was focused on attempting
to isolate teacher influence on raising
test scores, to the exclusion of other
influences that affect student
achievement. (p. 4)

Horn and Wilburn outline the role of RTTT and the rise of VAMs in Tennessee and across the country, and after this introduction, set the stage for the rest of the chapter, as follows: "A number of things have not changed in the repeated revival of the testing drama for which we offer our present interpretation and critique." Horn and Wilburn prepare readers for "the recurring theme of science of some quantifiable variety applied for personal and public betterment," where the "impacts of abstracting socioeconomic, cultural, and political experience into a quantifiable format that can be efficiently conveyed in the name of scientific-based education have had lasting effects on the way schooling is conducted and the purpose we attach to education in general." And they emphasize: "The fact that testing, standardized for the masses, became the primary tool to sort the privileged from those under the privileged is no mistake or fluke of nature" (p.7).

The book excels in how it highlights the role eugenicist and meritocratic ideologies played in the development of standardized testing technology and related practices, those more or less famous instances of mismeasurement. These examples, the authors argue, stand as the foundation of current policy; while traceable to the writings of Thomas Jefferson, these impulses are not, unfortunately, left only to the past.

Few are likely conscious of the degree to which eugenicist ideology informed the architects of standardized testing technology, an ideology that, for example, justified sterilization laws. Horn and Wilburn, after summarizing the roles of E. L. Thorndike, Henry Goddard, Lewis Terman, and others, note:

By 1920, the social efficiency social control ideology based on differentiation through scientific management provided a central
rationale for the "progressive" use of intelligence tests and achievement tests to measure, sort, and segregate school children in ways that upheld social structures based on class and race prejudice. (p. 16)

Later, emphasizing racist ideology as a basis for prominent social and educational policy, Horn and Wilburn observe:

Since the eugenics chapter of
American social history is rarely
taught in schools where even the
story of our slave-holding history
remains controversial, most
Americans do not know that
eugenics was taught as a regular part
of science curriculums in junior high
and high schools, as well as at
Princeton, Harvard, Yale, University
of Chicago, Stanford, and dozens of
other American colleges and
universities. (p. 23)


Few educational professionals are aware of the force of this history, and thus they are not in a position to discern similar patterns in present reform efforts. One of the strengths of the book is the manner in which it leads readers to look to the past to understand the present. Throughout Part 1, I found myself making marginal notes comparing past practices to our present "Race to the Top" era. For these reasons alone, MME offers any reader a policy-relevant critical history of testing and its supporting ideologies, and this past semester, it served as an exceptional tool for classroom
discussion and debate.

Horn and Wilburn close out Part 1 with a targeted history of the SAT, the Elementary and Secondary Education Act, and the significance of the Coleman Report in the transition from a policy focus on "inputs" to "outputs." I say targeted because MME does not present a narrative history, but rather pulls together facts and trends that will help readers quickly draw connections between past and present conditions.

A key example of this is found in their prescient reading of James Coleman, emphasizing not only the well-known findings relative to social class, but also his less cited finding with respect to forms of segregation. "Coleman found hope," Horn and Wilburn underscore, "strongly correlated with the presence of a sense of autonomy, which is more easily demonstrated, measured, and retained where racial and economic mixing prevails, rather than in racially and economically segregated environments" (p. 38). This point is especially relevant in our present era, where elite planners and ideologues alike have abandoned the goal of integration, abandoned efforts to remedy extreme school funding inequities, favoring instead market-based policies which serve to further segregate schools (Glass, 2008; Saltman, 2012).

In a March 3, 1970 speech by Richard Nixon, Horn and Wilburn find a public genesis for a focus on accountability driven reform, or "outputs," aligned with Nixon's so-called silent majority.

The President's speech called for a new
focus on school outputs rather than inputs,
along with the creation of a new National
Institute of Education to "lead in the
development of educational output." In
initiating "a new concept: accountability,"
Nixon called for new "dependable
measures" even at the local level: "School
administrators and school teachers alike are
responsible for their performance, and it is
in their interest as well as in the interest of
their pupils that they be held accountable."
(p. 43)

With Nixon, Horn and Wilburn point to the origin of test-based "accountability" in efforts to counter democratic and egalitarian movements associated with the Civil Rights movement and War on Poverty.

Origins of Test-based Accountability

One of the key contributions Horn and Wilburn make is with respect to understanding the rise of "accountability". While there are a variety of interpretations of the rise of test-based accountability and value-added models (Dorn, 2007; Manna, 2006; McGuinn, 2006; Mehta, 2013; Pierce, 2013; Proefriedt, 2008) none given central place to the role funding litigation played in establishing the context for what made VAMs a viable political alterative for policy makers. Dorn (2007), for example, concludes that an important condition for test-based accountability,

came from popular discontent with
the arrogance of administrators.
Resistance to desegregation was
certainly one face of this arrogance.
The civil rights movement had
targeted schools as one public
institution that was treating poor and
minority children unequally, and the
responses by both school boards and
administrators reshaped educational
politics. (p. 4)

For Horn and Wilburn, testing grew out of a crucible of racism and reductionist forms of reasoning regarding how educators and the public were to think about schooling. Their point is to draw a line between scientific racism and the eugenicist project of the "administrative progressives" of the early 20th century and present high stakes testing in general and current valueadded models in particular. It is not, as Dorn suggests, that the tools adopted by administrators during the progressive era have been turned against them in the fight for equality. Horn and Wilburn write:

What resulted from that first
generation of testing and sorting was
a system that continues today to
provide "scientific" rationalization
for the creation and maintenance of
measures whereby children of the
privileged display test results, on
average, consistently higher than
those children under the privileged
on tests that were devised to show as
much. By using measures stamped
with the seal of science, then, high
test scorers are guaranteed seemingly
legitimized access to the legacy of
privilege that accompanies higher
performance, thus reproducing social
and economic dominance by
descendants of the middle class elites
who first established their dominion
in the Colonies during the
seventeenth and eighteenth centuries.
Some who read this will surely doubt
such a claim, but we hope that by the
time readers finish the book, this
contention will be an indisputable,
though unacceptable, fact. For those
less skeptical now, we hope this
book will provide a deeper
understanding as to how the
mismeasure of children became
standard pedagogical practice. (p.
14)

Unlike Dorn (2007), and unlike (Manna, 2006) and McGuinn (2006), Horn and Wilburn emphasize the role of corporate interests and government service to those interests in institutionalizing mismeasurement and establishing an ever increasing centralized and privatized form of educational governance. This theme parallels some of Mehta's (2013) analysis, which highlights the role of the Rand Corporation and Department of Defense in popularizing the technocratic framework that built upon the "scientific management" of previous decades. While Dorn (2007) argues "One
can thus view statistical accountability systems as one way to resolve the dilemma between granting autonomy and authority to educators and keeping them under some political control" (p. 13), Mehta traces these "statistical accountability systems" to technocratic practices developed to manage the Cold War. He highlights the connection between the Rand Corporation, Robert McNamara and the
Department of Defense in building a technocratic model that would later be adopted to manage education policy at federal and state education department levels. Mehta described this approach as, "defining objectives, measuring goals, and aligning available resource" (2013, p. 64). What is key about Mehta's analysis is that it links a test reliant management technology to a specific outlook and form of governance: testing, nor its use, is politically neutral, benefiting some, and not others. What Horn and Wilburn add here is the key role funding litigation played in the rise of test-based accountability from the 1980s onward. Thus, while almost all scholars who have explored the rise of test-based accountably emphasize the shift in policy from one that regulated on the basis of inputs, to one that focused on outputs, Horn and Wilburn link this shift to resolving contradictions arising from struggles over school funding. And so now I turn to review MME's Part 2.

Accountability Against Funding Equity and the
Rise of Value Added Reform


Following the general line of questioning introduced by Flyvbjerg (2001), Part 2 of the book asks "who gains, and who loses, and by what mechanism of power" (p. xiv)? Using Tennessee as a case study, testing is explored as a key mechanism of power. Part 2 explores how and why assessment came to dominate curriculum and instruction and who benefited from this arrangement. The most important aspects of this section is how it shows test- based accountability to be a political response to funding litigation, struggles over who should receive state funds, and the struggle for equality more generally.

The authors begin by tracing how business elites and accountability for output reformers assumed control over education policy at the state level. Importantly, Horn and Wilburn make clear in this section a point often lost in discussions of education policy and school finance: the idea of focusing on outcomes as a means for being efficient and saving money had little to do with actual budget problems. The actual problem was a diverting of funds elsewhere and so, Horn and Wilburn argue, test-based accountability was in fact a means to justify the transfer of funds to the private sector.

As a result of this transfer, public spending for public schools decreased, creating a "political and economic context for corporate influence" over education reform (p. 99). The authors explain: "Revenue lost in tax breaks to business and industry, [increased] state spending for highway systems and other infrastructure, and vocational education programs tailored to meet business and industry workforce needs," reduced the funds available to provide "adequate and equitable education for all students in the state" (ibid).

These changes were taking place in the context of school funding litigation at the state level, following a failed attempt to convince the U.S. Supreme Court that education is a right in the United States. This context is presented as key to understanding how the value-added teacher evaluation model of William Sanders -- a politically connected biostatistician -- became written into Tennessee education law in 1992. Thus, Horn and Wilburn observe: "Running parallel to ... the crafting of new accountability management and funding systems was the ongoing lawsuit filed by the Tennessee Small School Systems in July 1988 (p. 87). "By the end of the 1980s," the authors explain, "lawsuits had been filed in 16 states by local school systems seeking relief from inequitable funding structures, and by 2007 almost 100 'education reform cases had been brought before state supreme courts' ... It was from within the context of these state lawsuits over education funding that this part of our story unfolds, and it takes us back once more to Tennessee." They continue:

If economic deprivation could be
statistically adjusted for and the
influence of teachers on "progress
rates" could be determined and
closely monitored, and if some
teachers were getting better
"progress rates" than others,
independent of where the school was
located, then it stood to reason that
holding teachers accountable for
"progress rates" made sense. Clearly,
then, the education answer for
Tennessee policy elites lay in teacher
improvement and more
accountability measures, rather than
in expensive interventions to
equalize funding that may or may not
work; the Sanders Model, framed as
it was in 1990, offered the remedy
for a serious problem that remained
unspoken, even if it was on every
legislator's radar screen. (p. 75)

Thus, "Sanders' claims offered policy makers a surefire rationale for turning attention further from financial inputs for education and toward student test score outputs" (p. 65). Horn and Wilburn argue that the Sanders Model offered the "missing link to a coherent legislative package for education reform based on accountability and measurable results that minimized political and economic risk for policymakers and the business community" (ibid). And thus the

attention that may have accrued in
earlier years to the continued
shrinkage in education funding in
Tennessee was effectively shifted
toward outcomes that allowed
disparities in learning opportunities
to continue largely unchallenged by
a public bedazzled by charts, graphs,
and numbers that plotted winners
and losers in an undeclared race with
increasingly high stakes and no
finish line in sight. (p. 65)

As the Small Cities case continued, Horn and Wilburn chart how value-added policy logics became integral to winning the courts approval without resolving funding inequities. "When Chancery Court held its hearing to begin drafting an order in July 1993, the State offered the 1992 passage of the EIA [Education Improvement Act] with a specific revised funding formula of the Basic Education Program (BEP) as new evidence that the State was making good on its responsibility to phase in an equitable funding formula. Chancellor High was impressed and ruled that he would allow for the BEP to take effect before any further orders would be rendered (p. 89) even the BEP did not account for disparities in teacher salaries. Value-added offered thus became central to the new mismeasurement.

Value-Added Models: The New Mismeasure

Part 3 asks, "is it desirable," answering that question by examining the validity and reliability of the value-added system that emerged in Tennessee. While researchers and policy analysts have increasingly raised serious concerns about the validity and reliability of VAMs for personnel decisions (e.g., Baker, Oluwole, & Green, 2013), Horn and Wilburn provide a unique presentation of otherwise now well-known facts regarding VAMs as applied under Race to the Top. They articulate three reasons the Sanders Model in particular falls short:

Sanders assumes (a) that tests and
test scores are a reliable measure of
student learning; (b) that
characteristics of students,
classrooms, schools, school systems,
and neighborhoods can be made
irrelevant by comparing a student's
test scores from year to year, (c) that
value-added modeling can capture
the expertise of teachers fairly. (p.
157)

The authors argue out each point in Part 3, providing one of the most robust and accessible critiques of VAMs available. But here I want to only briefly highlight three insights Horn and Wilburn offer that do not commonly appear in most current VAM critiques: (1) the validity and reliability problems with VAMs were well established and known to researchers and policy makers in Tennessee in the 1990s; (2) VAMs intensify the harmful, long-standing practice of education reformers viewing schools in terms of "production-function measurements" --that is, as factories; and (3) that VAMs as developed in Tennessee serve to mask educational inequality. While the first two points are, I believe, readily understood without elaboration, the third point does require explanation.

Horn and Wilburn emphasize that the "focus of education reform in Tennessee for the past quarter century has not been on the adequacy of resources to provide equal opportunity for all children." Instead, focus has been on a narrow test based accountability. And despite decades of testbased accountability, "Tennessee's average 2012 ACT composite score for general knowledge and reasoning skills was 19.7, ranking Tennessee 48th among 50 states and the District of Columbia;" Tennessee ranked 50th in ACT math, with a score of 19.1. "Predictably," the authors note, ACT scores and graduation rates for districts with greater percentages of economically disadvantaged students "are significantly lower." "However," they continue,

most poor districts' value- added
scores are above the state average,
sometimes exceeding the growth of
wealthier districts. For state
legislators looking to convince
constituents that a third decade of the
same accountability plan is what is
needed, it is easy to see that Dr.
Sanders' growth numbers offer the
only evidence that could be used for
such a purpose. (p. 152)

A few pages later, they ask:

Do TVAAS growth scores mask
students' proficiency level for
parents, the media, and politicians
alike? If Humboldt (a district with
much poverty) is making more
growth than more advantaged
districts like Williamson County,
then can parents and politicians
agree that the BEP is adequate for
providing an equal educational
opportunity for all students across
the state? (p. 154).

Thus Horn and Wilburn assert the "TVAAS's masking effect." While showing growth, TVASS (and all growth models adopted under Race to the Top), may mask the limited academic progress of poor and minority students, letting the state "off the hook for maintaining and supporting an adequate system of public education for all students" (p. 155). While VAMs appear to be policy mechanism for weakening teachers unions and thus a basis for further reducing expenditures on public education, moving states like New York from court established standards of adequacy, VAM- based systems of accountability may also create the conditions for more education inequality in terms of both funding and academic outcomes. In the words of Horn and Wilburn: What "value- added modeling has contributed in the testing fairness formula by acknowledging different starting points in the testing race, it takes away by helping to conceal the chasms that constitute the inequalities that mark the very different starting points of the disadvantaged and the privileged" (p. 214).

The Alternatives that are Urgently Needed

Horn and Wilburn conclude their work with a simple question: what is to be done? In this last section, the authors begin by comparing the past to present, analyzing the themes that remain constant -- the economic justification for reform for example -- and those that are new, or different. They write:

The biggest difference between 2013
and 1913 is that, instead of requiring
the waling of defectives for the
protection of society, as Terman
cheerfully admitted, we now call
constant surveillance and total
compliance "freedom" and "choice"
and pretend that children and their
parents prefer penal-style school
settings and test prep to the more
humane schools and real content that
we find where middle class parents
demand it. In the name of not leaving
children behind, reform schoolers
have applied more and more
draconian compliance measures that
allow a two-tiered school caste
system, whereby the testing
"defectives" are segregated,
contained, and provided behavioral
interventions that no middle class
parent would allow. (p. 210)

Here the authors highlight the present as having a stronger link between proposed reforms and business opportunities for a variety of education related companies and especially emphasize the big education foundations, including their role in applying policies once reserved for the K12 system to higher education. Significantly, Horn and Wilburn highlight the undemocratic nature of this corporate-philanthropic command and control of education, and direct the reader to focus on the political implications of "instrumentalist" reform initiatives. As readers have already come to appreciate, Horn and Wilburn have a way of getting to the point, and so they are worth quoting again:

For those who view the schools' and
the universities' core mission as
foundational to creating and evolving
knowledge and understanding to
advance the autonomy and improved
living for all people and cultures, the
fixation by unelected plutocrats on a
singular vision of what is fair for
everyone except themselves
expresses a level of arrogance for
which no parallel exists in the
national history. Public education
policy steering by billionaires offers
a real and present danger to the
purpose and functioning of
democratic institutions. (p. 199)

A page later they assert:

By the 1980s the accountability
movement that had originated in
attempts to evaluate the effectiveness
of ESEA Title I programs became
solidified and deeply entrenched by a
standards and testing movement
driven largely by an ideological
agenda aimed at establishing
alternatives to public schools at
public expense. (p. 200)

In short, then, they offer that what is needed is a renewed commitment to public education and a recognition that social inequality and environmental degradation must be dealt with head on. Predictably, they call for an end to high-stakes testing and reductionist instrumentalism in general. And they defend the wisdom of educators and theorists that has existed for some time. A choice example is this:

The most productive way to assess
student understanding beyond strict
adherence to rules is for teachers to
observe and evaluate, using refereed
criteria by educators and disciplinary
professionals, student performance
of real life tasks in either simulated
or actual disciplinary contexts. (p.
212)

So that the reader is not confused, these authors are not dogmatic in their rendering of value-added modeling, and offer that it might be useful for diagnostic purposes for large systems. They advocate a model of assessment based on studies of the transformation of novices to experts. Thus, policymakers must address the roots of social inequality, to be driven by a renewal of the democratic purpose and control of public education, and public support for education professionals and their central role in evaluating students and schools
at the site of practice.

I think Many readers will find, as I have, that several passes through the Mismeasure of Education are required to grasp its contribution to understanding the origins of our current situation and to fully consider the orientation it provides as a counter to what more and more practitioners, scholars and even policy analysts are seeing as the wrong direction for education. The profound "mismeasurement" that plagues schools is premised on a distorted and narrow understanding of the purposes of education, the nature of teaching and learning, and a refusal to admit that school governance cannot be separate from either its purpose or vision of the educated person.

References
Baker, B. D., Oluwole, J. O., & Green, P. C. I.
(2013). The legal consequences of
mandating high stakes decisions based on
low quality information: teacher evaluation
in the race-to-the-top era. Education Policy
Analysis Archives, 21(0), 5.
http://dx.doi.org/10.14507/epaa.v21n5.2013
Dorn, S. (2007). Accountability Frankenstein:
Understanding and taming the monster.
Charlotte, N.C.: Information Age
Publishing.
Falk, T. M., & Rocha, S. (2009). Social science and
its discontents: An essay review of Bent
Flyvjergâs Making Social Science Matter.
Education Review, 12(4). Retrieved from
http://www.edrev.info/essays/v12n4.pdf
Flyvbjerg, B. (2001). Making social science matter:
why social inquiry fails and how it can
succeed again. New York: Cambridge
University Press.
Garrison, M. J. (2009). A measure of failure: the
political origins of standardized testing.
Albany, NY: SUNY Press.
Glass, G. V (2008). Fertilizers, pills, and magnetic
strips: the fate of public education in
America. Charlotte, N.C.: Information Age
Pub. Retrieved from
http://www.loc.gov/catdir/toc/fy0804/20080
03133.html
Gould, S. J. (1981). The mismeasure of man. New
York: W. W. Norton & Company.
Manna, P. (2006). Schoolâs in: federalism and the
national education agenda. Washington,
D.C.: Georgetown University Press.
Retrieved from
http://www.loc.gov/catdir/toc/ecip0519/200
5027244.html
Manna, P. (2011). Collision course: federal
education policy meets state and local
realities. Washington, DC: CQ Press.
McGuinn, P. J. (2006). No Child Left Behind and
the transformation of federal education
policy, 1965-2005. Lawrence, Kan.:
University Press of Kansas. Retrieved from
http://www.loc.gov/catdir/toc/fy0613/20060
06836.html http://www.hnet.
org/review/hrev-a0f1v9-aa
Mehta, J. (2013). The allure of order: high hopes,
dashed expectations, and the troubled quest
to remake American schooling. New York:
Oxford University Press.
Pierce, C. (2013). Education in the age of
biocapitalism: optimizing educational life
for a flat world. New York, NY: Palgrave
Macmillan.
Proefriedt, W. A. (2008). High expectations: the
cultural roots of standards reform in
American education. New York: Teachers
College Press.
Saltman, K. J. (2012). The failure of corporate
school reform. Boulder, CO: Paradigm
Publishers.
Strauss, V. (2013, October 13). How New Yorkâs
education commissioner blew it big time â"
principal. The Washington Post. Retrieved
http://www.washingtonpost.com/blogs/answ
er-sheet/wp/2013/10/13/how-new-yorkseducation-
commissioner-blew-it-principal/

About the Reviewer
Mark Garrison
Professor, Education Policy & Research D'Youville College
garrisom@dyc.edu
Dr. Mark Garrison is Professor of Education Policy & Research at D'Youville College. He is a noted sociologist of education focusing on applying the insights of social research to help educators better understand and thereby improve the quality of education. His research and publications have won him acclaim, including the 2010 CHOICE Outstanding Academic Title from the American Library Association for his book, A Measure of Failure (SUNY Press, 2009). He is also the recipient of the 2004 DYC AAUP Scholar of the Year award. He has become a noted critic of current education reform efforts, with a growing array of presentations, papers and journal articles examining the political aspects of education reform.

— Mark Garrison
Education Review
2015-04-
http://edrev.asu.edu/index.php/ER/article/download/1848/476


INDEX OF RESEARCH THAT COUNTS


FAIR USE NOTICE
This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available in our efforts to advance understanding of education issues vital to a democracy. We believe this constitutes a 'fair use' of any such copyrighted material as provided for in section 107 of the US Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit to those who have expressed a prior interest in receiving the included information for research and educational purposes. For more information click here. If you wish to use copyrighted material from this site for purposes of your own that go beyond 'fair use', you must obtain permission from the copyright owner.