[Susan notes: The writer responds to Jay Mathews' unsupported enthusiasm for his own quality index.]
Submitted to Washington Post but not published
The Mathews column to which this letter refers was sent to the ombudsman as well as to editorial:
(Please take a look at the following excerpt and the available summary, if not the whole of this study.) I believe that the Post has a responsibility to carefully distinguish between educational opinion and researched, journalistic reporting. In the case of Jay Mathews' orientation in support of his own publications, clearly, balanced reporting should include the fact that his opinions are NOT those of the
scientists, educators and researchers who studied the effects of the AP curriculum in Math and Science for two years to create this paper.
The paper, Learning and Understanding: Improving Advanced Study of Mathematics and Science in U.S. High Schools (2002) Center for Education (CFE), states: "The practice of ranking schools by the number of AP or IB tests administered, or using students' scores on AP or IB exams for evaluating teachers or comparing the quality of teachers and schools, as some parents, school administrators, or policymakers may do, is making these programs high stakes in ways the program developers never intended."
It goes on to say, in relation to Mathew's "quality index" orientation, in
particular: "Such misuses of AP and IB assessments are both educationally inappropriate and counter to the Standards for Educational and Psychological Testing (American Educational Research Association [AERA]/American Psychological Association [APA]/National Council on Measurement in Education [NCME], 1999)."
In Mathew's Nov. 23, 2004 column, "A Chart Exposes High School Malpractice," although he admits to his limited intellectual and reasoning abilities: I do not use many charts and graphs in this column. They can be confusing, and I often don't understand them anyway, Mathews continues to maintain his unsupported enthusiasm for the merits of his opinion and uses strong rhetoric to persuade: Show them the numbers, and tell them if they don't understand them, they might want to try another
line of work. They don't really qualify as educators any more.
Responsible, supported, and balanced reporting on such critical subjects as Education, I believe, is truly what the media and particularly the Post must exemplify. I am dissapointed and discouraged that this important piece of research has not been discussed nor reported on in the context of recent discussions on educational reform.
(excerpt and reference identification below)
LEARNING AND UNDERSTANDING
IMPROVING ADVANCED STUDY OF MATHEMATICS AND SCIENCE IN U.S. HIGH SCHOOLS
Committee on Programs for Advanced Study of Mathematics and Science in American High
Jerry P. Gollub, Meryl W. Bertenthal, Jay B. Labov, and Philip C. Curtis, Editors
Center for Education Division of Behavioral and Social Sciences and Education
National Research Council
NATIONAL ACADEMY PRESS
Learning and Understanding: Improving Advanced Study of Mathematics and Science in
U.S. High Schools (2002)
Center for Education (CFE)
Uses, Misuses, and Unintended Consequences of AP and IB
This chapter examines a number of ways in which the Advanced Placement (AP) and International Baccalaureate (IB) programs and their assessment results are used. Some of these uses are appropriate and intended by the program sponsors; others are not. Unintended uses are making these programs high stakes in terms of their consequences for students, for teaching and learning, and for schools. Using the number of AP and IB examinations or number of AP and IB courses offered in a school as a measure of school quality also penalizes in the arena of public opinion schools that have chosen, because of different educational values or priorities, to offer other rigorous options to their students. Ignoring the value of these alternative approaches can stifle creativity and innovation in the development of new programs. Such alternative approaches could be as rigorous as AP or IB and also be more suitable to the student body served by a particular school.
Additionally, ranking schools by the proportion of students who take AP or IB examinations ignores the central issue of how many of those students are adequately prepared through high-quality courses to succeed on the exams. The committee notes that among states and school districts where all AP students are required to take the examinations, a notable number of students may not complete the examination or
answer test questions conscientiously.
Evaluating School Quality by the Numbers
The fundamental objective of education is to promote the academic growth of each student. Evaluations of school quality should help identify those schools that are accomplishing this goal and those that are not. When assessment scores are used to evaluate schools, efforts must be made to determine whether the scores are indicative of growth (or lack of growth) that is due to the quality of the
instructional program, or merely reflect students' home environments, external
learning experiences, and available resources. There are many schools that provide
high-quality instruction in school districts where resources are scarce whose students continue to grow academically. There are other schools in which test scores are high, but the quality of instruction may not be as good as in schools with lower test scores. The higher assessment scores may be more reflective of the students' other opportunities than of the quality of the school itself. In sum, relying
solely, or even primarily, on AP or IB test scores to evaluate school quality reflects a failure to recognize that there are substantive differences in educational institutions across the nation (see Chapter 2, this volume), while also ignoring the varying characteristics of the students who attend these schools.
With increasing calls for educational quality and accountability, educators and policymakers alike have turned to AP or IB to improve the qualityof their schools and curricula. The pressure to introduce or expand AP or IB offerings in high schools can have a number of unintended consequences. For example, schools that claim to offer advanced study programs may or may not be able to support such programs adequately, and the resulting courses may be far from what is intended by the program sponsors. Ensuring the quality and integrity of the AP and IB programs is a complex endeavor that it is frequently affected by factors beyond the control
of either the College Board or the IBO.
Standards and Regulation of Courses
The IBO carefully regulates which schools can offer IB courses and how those courses must be structured. As a result, schools are unable to offer IB courses or the IB Diploma without the imprimatur of the IBO.
In contrast, the College Board has no clear standards for what constitutes an AP course or the schools that offer them. This lack of consistency invites misuse of the AP name. For example, schools have been known to label non-AP courses as AP in an effort to make their students more competitive for admission to college. Others may have instituted AP courses without ensuring that they have the facilities and personnel needed to offer a college-level program. Consequently, all four of the panels call on the College Board to certify or regulate AP programs and teachers.
The committee found that schools’ efforts to prepare students for AP and IB science and mathematics courses often help stimulate improvements in the prerequisite courses, and that this is an important and positive effect of these programs. However, there also may be some unintended consequences. For example, some students may be adversely affected when schools compress preparatory courses in order to
prepare as many students as possible for AP or IB courses. Compression can occur by reducing coverage of specific subjects in a curriculum to provide adequate time for advanced study or by beginning the high school sequence of courses earlier in a student’s career.
To illustrate the point, strong preparation is important in mathematics because of the hierarchical and cumulative nature of the subject. However, the mathematics panel notes that there is considerable anecdotal evidence that some students intending to take calculus are rushed through prerequisite courses without
thoroughly learning the preparatory material. Mathematical sophistication takes time to develop, and knowledge of a catalog of mathematical facts and techniques is not sufficient. The mathematics panel notes that most of the students who do poorly in AP or IB calculus did not learn their algebra well enough. These students probably would have been better served by spending more time learning algebra and saving calculus for college. At the same time, there are students who are ready for a
rigorous calculus course in high school. Schools need to maintain a delicate balance between meeting the needs of this latter group and pushing too many students into advanced mathematics before they are ready. Achieving this balance is not easily accomplished.
Compression of the curriculum also can occur when students are allowed to skip prerequisite courses and take an AP course as a first course. Among the sciences, AP physics is the course students most frequently select as a first course in the discipline. Data obtained from the College Board (College Entrance Examination Board
[CEEB], 2000d) indicate that almost half of all AP physics test takers had had no prior experience with physics before enrolling in the AP course. Thus, the AP course had to cover both a year of high school physics and a year of college physics, making in-depth examination of any topic nearly impossible.
Participation in Examinations
The IBO expects that all students who take IB courses will take the associated examinations. The IBO uses student performance on its examinations to monitor and improve the overall quality of IB programs in different countries and schools. In contrast, the four panels note that the College Board has not developed examination policies or expectations for students’ participation in the AP examinations. Because the examinations provide the only external evidence that schools are preparing
students in a manner consistent with the College Board’s expectations, the panels suggest that a clearly articulated policy is necessary if the College Board is to maintain quality control of the AP name.
A. E. Levin Garrison