Assessment as Inquiry
Susan Notes: The author's point about the danger of treating theories as truths rather than as frames for temporary understanding is important.
For more than 10 years now, arguments have been constructed regarding the need for new forms of educational assessment and for a paradigm shift with a focus on supporting learning rather than on sorting and selecting students. The call for change in assessment follows an almost unanimous recognition of the limitations of current measurement theory and practice. The conceptions of learning represented by theories of learning and cognition appear strikingly different from those implied in current educational assessment and measurement practices. Indeed, most educational measurement specialists are still working from century-old understandings and behaviorist perspectives. Although the call for change is clear, the proposals and recommendations being put forward have limitations of their own and are unlikely to yield the kinds of fundamental changes envisioned by researchers. In this article I explore the possibility of using inquiry as a way to understand, and hence to assess, learning. After reviewing the assessment literature in which the need for change has been asserted, I analyze the theoretical and epistemological foundations that seem to undergird these writings. I focus primarily on the meaning of learning, knowing, and teaching implied in this literature and consider the limitations of its recommendations. I then consider notions of learning that seem to be excluded from current assessment practices and begin to uncover similarities between learning, knowing, and inquiring that could make inquiry an appropriate metaphor for what we currently know as educational assessment. Finally, I discuss important issues that would need to be considered in an inquiry framework for assessment.
Recognizing that educational measurement is facing a crisis, researchers have attempted to address the conceptual vacuum reflected in the assessment literature, calling for the integration of new understandings of human cognition, for theory- and construct-driven assessment, for changing the nature of assessment practices, and for integrating assessment with instructional practice. Included in their writing is a serious criticism of current testing practices and of the right-wrong scoring schemes they typically employ and a recognition that even the most promising applications of contemporary test theories are inadequate. The need to change assessment practices has been asserted and seems to converge on the integration of theoretical considerations about learning and cognition and on making assessment an integral part of learning and instruction.
The limitations of the proposals for change that have been put forward lie either in the focus of the work, in the lack of a clear articulation of the theories and concepts, in the nature of the assumptions made about learning (many of which remain implicit and unchanged), in the exclusion of certain conceptions of learning, or in some combination of these problems. The divide between researchers focusing their work on classroom assessment and those focusing on large-scale assessment used for accountability purposes is problematic because it implies that assessment can mean something different in these two contexts. As long as different people with different orientations address classroom assessment and external assessment separately, there is only a remote chance for change in educational practices. Furthermore, researchers calling for changes in assessment are often not clear about or fail to specify the theoretical perspectives and the definitions of learning and knowing from which they work. When the literature raises theoretical concerns, it seems bound by the same categories and ideas that emerged from a quantitative and acquisitive view of knowing and learning and the predictive, deterministic psychometric framework. There is a sense that the field is reacting to its initial inductive empiricism and attempting to move toward a more deductive theoretical process. The danger here lies in treating theories as truths rather than as frames for temporary understanding. In so doing, we run the risk that we will simply fix learning in different categories without really furthering our understanding. The theories of learning and cognition now being considered are more complex and the statistical models more sophisticated, but are these new developments really working from different assumptions with different implications for assessment?
The call for change in assessment is in part a call to move away from simple, mechanistic behaviorist notions of learning toward cognitive, constructivist, and situative representations. Researchers tend to collapse these last three theoretical orientations while working primarily within a cognitivist perspective. It seems, however, that behaviorist and cognitive theories make similar assumptions about human behaviors compared with those made by social constructivist or situative learning theories. Cognitive theory is often thought to be fundamentally different from behaviorism, but both perspectives are grounded in the same epistemology and work from the same ideas rooted in empiricism. The various constructivist schools of thought, which educators have overwhelmingly rallied around, have not been duly recognized in the assessment literature. Moreover, how their respective assumptions differ from each other and from cognitive theory has not been considered, nor have their implications for assessment. The historical, sociocultural, and activity theories of learning reflect a shift from knowledge as generalized propositional and symbolic representations internalized by individuals and transferable from context to context-a conception that shaped the development of most current forms of assessment-to knowledge as action, participation, and transformation of individuals within specific social and cultural contexts. In these perspectives, learning cannot be predetermined; this has serious implications for educational and assessment practices and for the learning-teaching connection that these typically presume. In the move from to know, to knowing-that is, from a state of having (knowledge) to an action involving participation, transaction, and transformation-the traditional notion of assessment becomes elusive and the phenomenon under consideration much less tangible.
If our purpose is to understand and support learning and knowing and to make inferences about these phenomena, then it seems that the idea of inquiry-open, critical, and dialogic-rather than that of assessment (as currently understood) would be more helpful because it would encourage consideration of the epistemological and theoretical assumptions from which we work. There is considerable overlap between the literature focusing on learning and that focusing on inquiry. Although these writings reflect different perspectives, audiences, and purposes, the questions they address and the terminology they use are quite similar. We need to question the distinctions often implied in practice between learning, understanding, knowing, inquiring, and so on. How are these different? Learning has traditionally been conceptualized in the context of student learning, whereas inquiry has been more closely associated with social science or scientific inquiry. Might this reflect the hierarchical nature of the activities in which researchers and students are engaged? In other words, experts inquire to generate knowledge (generative activity), and students have to learn or acquire that knowledge (mimetic activity) to become knowledgeable or expert. Or is it that researchers have traditionally engaged in a reflective and detached understanding of the world, making their understandings different from those of students engaged in learning activities, which have traditionally been based on acquiring information produced by others? Are these distinctions still appropriate or relevant given our changing understanding of learning, understanding, and knowing?
Given the absence of theoretical clarity or consensus evident in the literature on learning, knowing, understanding, and so on, it seems that we should remain cautious about the assumptions we make when we engage in the study of their representations, as is being done in current assessment practices. Resisting simplification and stating and questioning the assumptions and perspectives from which we work might be the only way we have to advance our understanding of the phenomena we now claim to assess. In fact, current assessment practices and methods may prevent us from studying and further understanding the very phenomena we are claiming to measure. The concept of assessment, as we understand it today, may only exist within conceptions of knowledge as product of learning, process of learning, or both, definable a priori for categories of individuals and regardless of the broader sociocultural context in which knowledge develops. Thinking of assessment as inquiry opens this inferential process to the broader field of inquiry and its various traditions, which could move current educational and assessment practices beyond the status quo. As in any inquiry practice, the questions posed, the theories and assumptions involved, the determination of what constitutes data, and the strategies used to generate and interpret data have to be articulated much more purposefully than they typically have been in the context of educational assessment.
For more than 10 years now, arguments have been constructed regarding the need for new forms of educational assessment (e.g., Shepard, 1989, 2000), for a paradigm shift with a focus on supporting learning rather than on sorting and selecting students (Gipps, 1994, 1999; Resnick & Resnick, 1992; Shepard, 2000). Some researchers have pointed to the dangers of trends, such as the minimum competency movement in the 1970s and to the negative consequences of state assessment practices for classroom assessment and for the curriculum (e.g., Delandshere & Jones, 1999; Madaus, 1994; Madaus & McDonagh, 1979; Shepard, 1989, 2000). Some have questioned measurement assumptions in the context of complex forms of assessment (e.g., Delandshere, 2001; Delandshere & Petrosky, 1998), and others (e.g., Embretson, 1993, 1998; Frederiksen, Mislevy, & Bejar, 1993) have considered the development of a new test theory grounded specifically in more recent developments in cognitive theory. Although all of these efforts address the problems and limitations of today's educational measurement and assessment practices, working with existing concepts, methods, and technology toward improving the current system, they seem to be buried under the renewed call for accountability and for increased measurement of learning outcomes as a way to evaluate the efficiency of the educational system. The sense of urgency and the pressure to use current forms of assessment for political aims are compromising the search for new ways to study and assess learning and to make claims about the state of education in this country.
Currently most state assessment programs and many classroom assessment practices resemble poorly designed research projects for which the most fundamental questions have not been asked. "What do students know?" seems to have been one of the main assessment questions. But the question "What does it mean to know?" is rarely asked, let alone discussed or answered, other than implicitly through the content of the questions included on the test. Such a question is critical, and yet as Gill (1993) notes, "Among the many and various articles and books on the quality and direction of American education, one searches in vain for an in-depth discussion of how knowing takes place, of who knowers are, and of what can be known" (p. 1). Until we come to grips with, or at least frame the issue of, knowledge and knowing in ways that can guide education practices (including assessment), the enterprise of education runs the risk of being fruitless and counterproductive. In its current state, assessment appears to be a process of collecting data about phenomena or constructs that we have not adequately defined, to answer questions that we have not articulated, and on the basis of which we draw inferences about the quality of the education system. Such practice has the allure of na´ve inductive empiricism, which has been seriously criticized in most areas of social science research for quite some time (e.g., Cronbach & Meehl, 1955; Douglas, 1971; Feyerabend, 1978; Lakatos, 1970; Polanyi, 1958; Wallace, 1971). Before we can generate valid inferences about the important project of education, we need to reconnect our educational practices to theoretical and philosophical considerations to clarify the assumptions we make about learning and teaching.
In this article I explore the possibility of using inquiry as a way to understand, and hence to assess, learning. I first review the assessment literature in which the need for change has been asserted and analyze the theoretical and epistemological foundations that seem to undergird these writings. I focus primarily on the meaning of learning, knowing, and teaching implied in this literature and consider the limitations of its recommendations. I then consider notions of learning that seem to be excluded from current assessment practices and begin to uncover similarities between learning, knowing, and inquiring that could make inquiry an appropriate metaphor for what we currently know as educational assessment. Finally, I discuss important issues that would need to be considered in an inquiry framework for assessment.
Teachers College Record
INDEX OF RESEARCH THAT COUNTS