Orwell Award Announcement SusanOhanian.Org Home

Compstat for Teachers: Public schools can join the same data revolution that transformed urban policing

Ohanian Comment: Note to all those progressives who still hold out hope for the Obama/Duncan education policy: Shouldn't the fact that the right-wing Manhattan Institute loves Duncan be a clue?

I will summarize the premise of this screed by pointing out that depending on McGraw-Hill, Pearson, et al student standardized test results is the most expensive, least effective, and most damaging way to evaluate teacher performance. Period.

NOTE: Marcus A. Winters City Journal is a quarterly magazine of urban affairs, published by the Manhattan Institute.

An example of the Manhattan Institute activity: On Thursday, November 13, 2008, the Manhattan Institute proudly hosted President of the United States George W. Bush at New York's historic Federal Hall National Memorial. The President spoke in defense of free market principles and warned of the dangers of excessive financial regulation.

"Free market capitalism is far more than economic theory. It is the engine of social mobility--the highway to the American Dream."
Meanwhile, at City Journal website they have revolving words of praise for their efforts from: William Bennett, Mona Charen, Rudolph Giuliani, Peggy Noonan, Clarence Thomas, and George Will.

These folk shout "Hands off!" when it comes to managing money, but they're all for tightly managing teachers. The piece below is an example of just how corporate/right wing the Obama education policy is: They point out that Arne Duncan is pushing the notion that The use of administrative data sets has the potential to improve public schools as dramatically as it cleaned up New York City streets--if only we'll let it happen.

My answer is the same as it always is: These people aren't advocating the close examination of poverty data: health, tooth decay, presence of iron, family income. No, they declare test data is king. All you have to do is look at the really ineffective, misleading, inappropriate, and just plain stupid test questions on which they are basing all this data collection to know the data emperor has no clothes.

As Rich Gibson of the Rouge Forum observes, "The education policy is a war policy."

Marcus A. Winters

As City Journal's Heather MacDonald has argued, New York's most important policing reform in the 1990s was Compstat. This revolutionary data system tracked crime precisely, allowing the New York Police Department to focus its efforts on the most troubled neighborhoods--and to hold precinct captains accountable when things went wrong. Good information is a powerful tool not just in policing but in many policy areas. In fact, a similar data revolution has been brewing in education for about a decade, and President Barack Obama and his secretary of education, Arne Duncan, are now pushing it. The use of administrative data sets has the potential to improve public schools as dramatically as it cleaned up Gotham's streets--if only we'll let it happen.

Large administrative data systems were a natural by-product of the movement toward increased student testing in the United States. As more and more states began administering their own tests to students, the scores needed to be collected and the data maintained. Some states and districts went the extra mile and created data systems capable of tracking the performance of individual students over time. And the most sophisticated systems also match students' data to their teachers, enabling researchers--with the aid of powerful statistical tools--to identify the influence that each teacher has on student academic performance. To a layperson, there may be nothing less interesting than volumes of test-score data inside mainframe computers, but these systems have enormous potential to improve the way we evaluate the quality of our teachers.[emphasis added]

The current system of teacher evaluations, moreover, is clearly flawed. Teachers are usually observed for a single class period once or twice a year, and in some states, senior teachers undergo evaluation just once every three to five years. In theory, these limited observations are supposed to reveal whether teachers' performance has been satisfactory throughout the year. In practice, they are rubber stamps. The nonprofit New Teacher Project analyzed teacher evaluations in 12 large school districts across four states and found that in districts using a binary evaluation system (the only ratings being "satisfactory" and "not satisfactory"), over 99 percent of teachers received the thumbs-up rating. Even districts that used broader evaluation distinctions ranked 94 percent of teachers in one of the top two tiers and deemed just 1 percent "unsatisfactory." NOTE: Kati Haycock of Education Trust is chair of board of directors of the New Teacher Project. I was in the audience and heard first-hand the way she describes public school teachers to reporters.]

No honest person can argue that all teachers are performing up to par. In fact, the homogeneous results of teacher evaluations are completely at odds with a wide body of teacher-quality research conducted using administrative data sets. This research consistently finds vast differences in teacher effectiveness both across and within schools. We all know that there are good teachers and bad teachers out there, but we don't distinguish between them in any meaningful way. As Secretary Duncan recently lamented to a group of education researchers: "In California, they have 300,000 teachers. If you took the top 10 percent, they have 30,000 of the best teachers in the world. If you took the bottom 10 percent, they have 30,000 teachers that should probably find another profession, yet no one in California can tell you which teacher is in which category. Something is wrong with that picture." He's right. [What nonsense! Anyone with extended direct contact with schools can distinguish between the two. Ask other people working in the building, including principals, teachers, and custodians. Ask parents. Ohmygod, ask students. Depending on McGraw-Hill, Pearson, et al student standardized test results is the most expensive, least effective, and most damaging way to evaluate teacher performance. Period.]

Currently, 21 states have data systems capable of matching teachers to students. Duncan has pledged to use his discretionary funds under the federal stimulus package to get more states to do the same. It seems like a no-brainer. After all, who's against having more information? Yes, indeed, it is a no-brainer. We need to ask just whose brain is missing.

The teachers' unions, that's who. They're fighting hard against the adoption of these systems precisely because the information they reveal is so useful. The unions insist, against all evidence and logic, that no meaningful variation exists in teacher quality. Further, in a clear case of making the perfect the enemy of the good, they argue that because test scores are a limited measure of student proficiency and statistical models for evaluating teacher quality are imperfect, the information that data-system analyses produce for individual teachers are not ready for prime time.

As always, the unions have gotten far by coupling their dubious arguments with their overwhelming political influence. Not long after advocates for a new data system suggested that it be used to inform teacher-tenure decisions in New York City, for example, the state legislature explicitly banned its use for that purpose. (The city can still use its data system to develop unofficial teacher evaluations, which are distributed to principals.) Not to be outdone in legislative insanity, California has made it illegal to link electronically the state's student test-score data set with a data set identifying individual teachers.

The first step to improving teacher quality in the United States is to measure it accurately. Expanding administrative data systems to all states would improve our ability to identify which teachers are effective, which need assistance, and which should be shown the door. To be sure, these measures aren't perfect and shouldn't be used in isolation to make employment or compensation decisions--but they can certainly be used to inform them. Those interested in education reform should join the push to increase the use of teacher and student data.

— Marcus A. Winters
City Journal





This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available in our efforts to advance understanding of education issues vital to a democracy. We believe this constitutes a 'fair use' of any such copyrighted material as provided for in section 107 of the US Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit to those who have expressed a prior interest in receiving the included information for research and educational purposes. For more information click here. If you wish to use copyrighted material from this site for purposes of your own that go beyond 'fair use', you must obtain permission from the copyright owner.