Orwell Award Announcement SusanOhanian.Org Home

[Susan notes: A whole lot rides on whether the PISA scores are worth a hill of beans. In 2013 The Guardian offered this summary.

What criticisms have been made against it?

From the outset, Pisa has been met with scepticism, criticism and even outrage, most of which has stemmed from the claim that the study's findings are arbitrary. One such voice was Dr Svein Sjøberg of the University of Oslo, who claimed that a small change in question choice or weightings could result in a big change in a country's overall rankings.

Though the methodology for collecting the results might be clear, the way they are interpreted and analysed to become final results is less so. This lack of statistical transparency has also been a focal point of criticism levelled at Pisa and indeed the OECD. The last time they published a technical report to make their methods more open, they werenât able to make it any more concise than 419 pages.

The sheer breadth of Pisa has also raised questions about the comparability of results within it. Even those who accept it is fair to assess countries and economies alongside one another question some of the cultural factors that divide them. Joachim Wuttke, a teacher of computer science in Germany, is one such sceptic. Wuttke looked at which students failed to complete the test and which preferred to increase their speed -- even at the risk of increasing their errors. He found significant differences in student behaviour that affected student performance. . . .

Meanwhile, countries such as Finland, a consistent top-performer in the Pisa league tables, has been accused of failing miserably on other international academic tests such as TIMSS. Why? Gabriel H Sahlgren, the author of a book on the dubiousness of the Finnish miracle claims it is "because its centralised curriculum has ignored certain concepts that are not tested in Pisa". . . .

Don't you wish we had media that dug deeply on such topics as PISA--instead of mouthing hot air?]

Submitted to Newsweek but not published

To the editor

Lessons from the world's best public school (May 9) considers Shanghai to be the "world's best education system" (p. 30), because Shanghai's scores on international tests such as the PISA are at the top of the world. As Newsweek notes, however, Shanghai is "one of China's richest cities" (p. 31) and the children of migrant workers are excluded from their public schools.

In contrast, American students' PISA scores are unspectacular. But children are not excluded from school in the US, and American schools must deal with a very high rate of child poverty, 24%, the second highest among 34 advanced economic countries.

Any comparison of educational programs must consider the effect of poverty. Every study ever done has shown that poverty has a devastating effect on school performance. Poverty means, among other things, food deprivation, lack of health care, and lack of access to books.

When researchers control for the effect of poverty, American scores are among the best in the world. When we examine middle class American students in well-funded schools, their scores are close to Shanghai's and ahead of all other countries tested.

The current move in the US to become more like China in education, with more competition and more rigorous examinations, will do nothing to improve student achievement in the US. The problem is poverty.

Stephen Krashen

This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available in our efforts to advance understanding of education issues vital to a democracy. We believe this constitutes a 'fair use' of any such copyrighted material as provided for in section 107 of the US Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit to those who have expressed a prior interest in receiving the included information for research and educational purposes. For more information click here. If you wish to use copyrighted material from this site for purposes of your own that go beyond 'fair use', you must obtain permission from the copyright owner.