Orwell Award Announcement SusanOhanian.Org Home

In the Courtroom: Call for Directed Verdict of Public Insanity

Publication Date: 2012-03-03

Michael T. Martin, Research Analyst, Arizona School Boards Association, adapted this courtroom scene from his online report Misconceptions of Achievement Testing, which is well worth a careful read.

I call to the stand Professor James Popham, an expert on testing who noted that test construction requires that for a question to be valid it must be answered by only about half of the students.

Me: Professor Popham, would you please tell the court what you stated in a March 1999 Educational Leadership article Why Standardized Tests Don't Measure Educational Quality.

Professor Popham: "Thus, the better the job that teachers do in teaching important knowledge, and/or skills, the less likely it is that there will be items on a standardized achievement test measuring such knowledge and/or skills."

Me: So you are saying that if something is extremely important for students to know, it will never appear on a standardized test. Is that correct?

Professor Popham: "It may seem strange that these tests are designed not to measure the most important things that teachers teach. But these tests were not designed to judge the quality of schooling.â (from an April 25, 2001, Public Broadcasting Service Frontline interview)

Me: So, professor, donât you find it a little strange that the effectiveness of teachers is supposedly measured by these achievement tests?

Professor Popham: "Employing standardized achievement tests to ascertain educational quality is like measuring temperature with a tablespoon. Tablespoons have a different measurement mission than indicating how hot or cold something is."

Me: Thank you Professor Popham, you may step down. If it pleases the court, I would suggest this is a general professional understanding of test inadequacy. I call the National Research Council to the stand. They were asked by the Washington, D.C., school district to evaluate their schools. Would you please read for the court this paragraph in your official evaluation report.

NRC: "For this discussion, it is perhaps most important to underscore that most tests are not designed to support inferences about related questions, such as how well students were taught, what effects their teachers had on their learning, why students in some schools or classrooms succeed while those in similar schools and classrooms do not, whether conditions in the schools have improved as a result of a policy change, or what policy makers should do to solidify gains or reverse declines. Answering those sorts of questions requires more and different kinds of evidence than test scores."

Me: Thank you, you may step down. Your honor, I move for a directed verdict of public insanity.

This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available in our efforts to advance understanding of education issues vital to a democracy. We believe this constitutes a 'fair use' of any such copyrighted material as provided for in section 107 of the US Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit to those who have expressed a prior interest in receiving the included information for research and educational purposes. For more information click here. If you wish to use copyrighted material from this site for purposes of your own that go beyond 'fair use', you must obtain permission from the copyright owner.