Popham Pitching a Plan to Improve NCLB Testing
Ohanian Comment: So what's W. James Popham doing in Pennsylvania? Apparently the same thing he was doing in Hawaii: Pitching his plan. See conference notes below this article.
Today, thousands of U.S. parents will be notified that their children are attending a "failing" school.
That's a pretty unsettling message. In Pennsylvania as well, a substantial number of parents will be on the receiving end of such school-failure notices. But there is reason for optimism in the commonwealth. Let's see why.
The upcoming spate of school failures stems directly from the No Child Left Behind Act (NCLB), a law signed by President Bush in early 2002. If that law is implemented unwisely, it will cause an enormous number of America's public schools to be inaccurately labeled as failing. Pennsylvania's citizens, especially parents of school-age children, need to understand why.
NCLB requires that annual standardized tests in reading and math be given to all of the nation's public school students in grades three through eight, and once again during high school, no later than the 2005-2006 school year. These state-determined tests must be based on a state's approved curriculum. Students' scores on the tests determine whether a school fails. Clearly, then, the nature of these tests is crucial.
Public schools must annually demonstrate that they have made adequate yearly progress based on their students' NCLB test scores. To avoid failure, each school must substantially improve its test scores not only for students as a whole, but also for specified student subgroups.
Schools receiving federal NCLB funds, if they don't make adequate progress for two consecutive years, will be placed on a penalty-laden reform track. But the law also requires that every public school, whether it gets federal funds or not, must distribute to the school's parents an annual report card indicating how well the school's students have performed on that state's NCLB tests. The prospect of a school's test-based failure will obviously increase the pressure on teachers to raise their students' test scores.
For this new federal law to work properly, however, a state's NCLB tests must be "instructionally sensitive" - that is, they must be able to detect improved learning. A state that adopts instructionally insensitive NCLB tests will almost certainly mislabel many of its public schools.
If a state's NCLB tests are not instructionally sensitive, many understandably frustrated teachers are likely to employ classroom procedures that, although intended to boost students' scores, actually undermine their education. For example, some teachers will engage in almost constant drill-and-kill preparation for upcoming NCLB tests, thereby eroding the joy students ought to experience while learning.
But what is an instructionally insensitive test? Currently, all of the nationally standardized achievement tests, such as the California Achievement Tests, are instructionally insensitive. The measurement mission of these tests is to compare students with one another. To produce the needed comparisons, however, many of those tests' items turn out to be strongly linked to students' socioeconomic status. Such tests, therefore, tend to measure the composition of a school's student body, not how well those students have been taught.
Most of today's so-called standards-based tests are also instructionally insensitive. That's because of their "curricular aims" - that is, they attempt to measure students' mastery of far too many state-approved content standards. As a result, teachers never really know which content standards are going to be measured in a given year. Nor is any standard-by-standard reporting provided so that teachers or parents can find out which particular content standards a student has actually mastered.
So states should design instructionally sensitive tests so that NCLB rules won't mislabel healthy schools as failing. Such tests:
Assess only a modest number of extraordinarily significant skills, such as a student's ability to compose various kinds of original essays;
Clearly describe the significant skills they measure, and
Supply skill-by-skill results to teachers, students, and parents.
Instructionally sensitive tests will provide accurate evidence of a school's quality and help teachers do a better job.
Happily, Pennsylvania Secretary of Education Vicki Phillips and colleagues have directed that any newly developed NCLB tests are to be instructionally sensitive. With such tests, Pennsylvania's policymakers can count on getting accurate evidence regarding school effectiveness. And, more important, Pennsylvania's parents can count on getting a better education for their children.
W. James Popham, a former president of the American Educational Research Association, is an emeritus professor in the UCLA Graduate School of Education and Information Studies.
The Education Policy and Leadership Center
The following presentation in Harrisburg, PA, is based on a presentation to Hawaii State Superintendent Patricia Hamamoto and the Department of Education staff
April 17, 2003
INSTRUCTIONALLY SUPPORTIVE NCLB TESTS:
WHY THEY'RE NEEDED, HOW TO GET THEM
W. James Popham
University of California, Los Angeles
1. Instructionally supportive accountability tests are those that (1) measure only a modest number of super-significant curricular aims, (2) supply lucid teacher-palatable descriptions of what's to be assessed, and (3) provide instructionally informative results so that a student's mastery of each assessed curricular aim can be determined.
2. If a state's NCLB tests are not instructionally supportive, then many of that state's public schools will soon be identified as ineffectual because of students' failure to make adequate yearly progress on tests whose very nature makes such progress essentially unattainable.
3. If a state's NCLB tests are not instructionally supportive, then the state's public school students will be educationally harmed because of (1) curricular reductionism, (2) excessive test-preparation, (3) unethical test-preparation/test-administration, and (4) the resultant widespread perception that the state's public schools are unsuccessful.
4. If a state's NCLB tests are not instructionally supportive, then an increased number of NCLB-induced school failures will surely cause the state's public school educators to be held in lower esteem by the state's citizens and policymakers than is currently the case.
5. A major impediment to the creation of instructionally supportive NCLB tests in most states is a sprawling array of curricular aims---content standards crafted at a different time and for a different purpose.
6. Another equally important obstacle to the creation of instructionally supportive NCLB tests is often the absence of a commitment to do so on the part of a state's educational leadership.
1. Unless the chief state school officer (CSSO) insists that a state's NCLB tests must be instructionally supportive, and devotes sufficient energy/resources to ensure that instructionally supportive NCLB tests become a reality, there is no sense in moving forward.
2. A small CSSO-appointed committee of 3-5 hard-thinking individuals should be appointed with the sole purpose of monitoring the state's movement toward the creation of instructionally supportive NCLB tests so that the CSSO can be apprised, possibly on a monthly basis, of progress toward the creation of those tests.
3. A somewhat larger committee of curriculum and assessment specialists, persons chosen for their intellectual rigor and creativity, should derive---from the state's existing content standards---a modest number of re-conceptualized, super-significant content standards that will serve as the state's standards-based NCLB assessment framework.
4. Descriptions of the skills and knowledge to be measured according to this assessment framework should be created by the same committee, so that the state's educators will readily understand what curricular aims are being assessed at every grade level by the state's instructionally supportive NCLB tests.
5. Suggested instructional approaches to help teachers promote students' mastery of the modest number of high-import content standards being assessed by the state's NCLB tests should be identified by the same committee. Because there will be only a few assessment-based curricular foci per grade and subject field, the state's teachers need to become truly proficient in promoting students' mastery of those curricular aims. The committee's instructional suggestions would be the initial step in a concerted state-sponsored effort to enhance the instructional capabilities of the state's educators to achieve the truly significant curricular aims assessed by the new NCLB tests.
W. James Popham
No Child Behind: Sensitive Tests Will Identify the Good Ones
August 11, 2003
INDEX OF NCLB OUTRAGES