Orwell Award Announcement SusanOhanian.Org Home

NCLB Outrages

The Rational Thing to Do

1. W. Edwards Deming, the Total Quality Management guru for whom Japan named its national quality prize, focused his enmity against Management By Objectives and Merit Pay. Deming said that merit pay was destructive for many reasons but the primary reason was that it forced people to do destructive things to the organization that put their individual efforts in a better light. He cited, for example, a woman who came to him for training in a bedraggled state. On inquiry he discovered that in order to attend his training she had to get up at three in the morning to catch an airplane because the travel department in her company was boosting its performance by not having her fly in before hand and stay the night so she would be fresh for the training. Deming also preached against end-of-line quality testing such as high-stakes testing, he preached against merit pay, and he preached against setting quantitative goals. He said they were destructive to corporations and they are just as destructive to educational organizations.

Thus high-stakes testing should be considered destructive to an educational organization, not for some esoteric pedagogic reason, but for fundamental management theory reasons.

2. I have to marvel at the use of the word "cheating" in this context. In the current context of merit pay and test scores, teachers are being paid to raise test scores. In any logical sense, teachers should be the judges of student progress, even if one chooses to use standardized tests. The point being that the logical role of the teacher is that of an independent referee in determining student progress. BUT when you institutionalize test outcomes to determine the teacher's performance then you have essentially corrupted the referee. It is no different than putting traffic ticket quotas on police officers, or bribing judges. You cannot institutionalize corruption and then claim cheating. If you pay people to become corrupt, and they become corrupt, then you ipso facto have to conclude this is intended.

In the America legal system there is a concept called "subornation" which is defined as "the crime of inducing another to commit perjury." Even if high-stakes testing was based on a valid test, which is usually incorrect, and even if high-stakes testing validly measured student progress, which is demonstrably false, and even if high-stakes testing had some empirical relevance to student success, which has never been demonstrated, and even if high-stakes testing could mathematically measure minimum knowledge, which it cannot, it would still be idiotic to bribe the referees.

Thus high-stakes testing is a fraud to begin with, imposed in a corrupt regime, that creates a circumstance where it is only rational for cheating to occur, indeed it is suborned by the institutionalization.

3. When you use an invalid measure to make valid decisions, rational people are going to make valid decisions, only irrational people are going to follow the measure. I have stated repeatedly that there is a simple mathematical measure of test validity for subject matter tests: the coefficient of determination based on an item-test correlation is mathematically the same thing as measuring the degree to which answering the question correctly is explained by subject matter knowledge. There is no escaping this mathematical fact, this is not opinion, it is mathematics.

Therefore I have said, repeatedly, that the minimum coefficient of determination for any test question on a subject matter test should be 51% because knowledge of the subject matter should, at a minimum, be the majority reason for getting a question right or wrong. To my knowledge NONE of the questions on any high-stakes tests meet this criterion. I have discussed this with several testing experts and they all whine about how it is impossible as a practical matter to have questions meet this criterion. I don't care that you are incapable of making valid decisions. The choice is either to use a valid measure to make valid decisions or to use an invalid measure to make valid decisions. Otherwise it is not cheating to change an outcome to match what is valid, it is the only rational decision one can make.

— Michael Martin
Arizona School Boards Association


This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available in our efforts to advance understanding of education issues vital to a democracy. We believe this constitutes a 'fair use' of any such copyrighted material as provided for in section 107 of the US Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit to those who have expressed a prior interest in receiving the included information for research and educational purposes. For more information click here. If you wish to use copyrighted material from this site for purposes of your own that go beyond 'fair use', you must obtain permission from the copyright owner.