The Scourge of the Challenge Index Returns
Ohanian Comment: This system ranks schools to three decimal places. Don't all those people in the suburbs already know that they are rich and powerful? Do they need to chart their real estate values to three decimal places?
Most of the educators I know hate to compare one school to another. It is unfair and unscientific, they say, to try to determine which has the better teachers or the better principal or the better courses. What works for this student may not work for that one, they say, and there is no trustworthy way of measuring such things anyway.
Most of the parents I know, including me, have a slightly different view. We admit that it is difficult to make such comparisons, but we would like a chance to do so, since we know our children pretty well and have a good idea of what kind of school we are looking for.
It is for people like that that the Washington Post publishes each year its Challenge Index survey of Washington area public high schools. The 2003 results, to be found Thursday, Dec. 4, on this Web site's education page and in the Extra sections of the Post, allows many comparisons that some will call misleading, but I find interesting.
Many people say the Challenge Index is useless because it is too narrow. How can anyone decide which high schools are best based on just one factor? They are right that there are many qualities, most of them immeasurable, that make up a good high school. But many of the parents I speak to say they find it helpful to have a few very simple measures of things you can actually count, rather than just impressions or descriptions.
The U.S. News & World Report rankings of colleges, for instance, are too broad for me. They include dozens of factors, far more than I can sort out. I have to trust the magazine to weigh each input properly. The Challenge Index, on the other hand, is so simple that anyone, even me, can understand it and apply the same analysis to any school. And although there are very few important things that high schools do that can be quantified, I would argue that the index is by far the most useful and interesting way of measuring such schools with numbers. All the other gauges, such as average SAT scores or percentage of seniors going to four-year colleges, just tell you what the average parental income is.
To get a Challenge Index rating for a school, ask its counseling office how many Advanced Placement or International Baccalaureate tests -- college level examinations -- the school gave in May and how many seniors graduated from the school in June. In some cases I also count tests given to high school students enrolled in community colleges, if the exams can be shown to be as challenging as an AP or IB test.
Divide the number of tests by the number of graduating seniors, and see what you get. If that ratio is 1.000 or higher, the school is probably doing a good job coaxing students into college level courses and tests, which federal research shows is the best way to prepare students to survive the academic rigors of college.
Less than 4 percent of all American public high schools have reached that 1.000 level in their college level test-taking. But the 2003 Challenge Index results show that among the 158 public high schools in the Washington area that had full senior classes last June, a very impressive 55.7 percent -- a total of 88 schools -- reached that mark.
That is the good news. The bad news is that many schools in the area with the resources to lure more of their students into courses and tests that will prepare them for the trauma of university life have failed to do so, even when surrounded by good examples of how to do it.
Demographics rule, you say. If the school has a lot of low-income kids, it cannot be expected to meet the standards of schools full of lawyers' and doctors' children. Maybe yes. Maybe no. One of the best things about the Challenge Index is that it throws a light on schools from a new direction, and shows differences that appear to have less to do with family income than with other factors. Compare, for instance, Fairfax County in Virginia and Howard County in Maryland.
Both are relatively affluent suburban counties with lots of middle class parents. If you check the percentage of students poor enough to qualify for federally subsidized lunches at their high schools, Howard appears to be somewhat better off. The highest percentage of low-income kids at any of its high schools is at Oakland Mills, 14 percent. Twelve of Fairfax County's 23 high schools have low-income student percentages higher than that.
Yet which county has the higher Challenge Index? Fairfax, by far, with an average rating of 2.314 even if you do not count its super magnet school, the Thomas Jefferson High School for Science and Technology. Howard can only muster an average rating of 0.971. That is better than the vast majority of other school districts in the country, but is in the bottom half of districts in this area, many of whom share Howard's demographic advantages. The same goes for Arlington County, Va., and Anne Arundel County, Md., whose poverty profiles are similar despite the fact that Arlington's average Challenge Index rating is 2.854 and Anne Arundel's is 0.750.
Let's compare individual schools. Why should Alexandria's single high school, T.C. Williams, with 40 percent of its students qualifying for lunch subsidies, have a rating of 0.750 while Wakefield High school two miles away in south Arlington, with 52% of its students qualifying for lunch subsidies, has a rating of 1.620? Why does Glen Burnie High School in Anne Arundel County, with 28 percent of students qualifying for lunch subsidies, have a rating of 0.467 while Stuart High School in Fairfax County, with 46 percent of its students qualifying for lunch subsidies, has a rating of 1.961?
One answer is that Fairfax and Arlington counties, along with Loudoun, St. Mary's and Prince William counties, as well as the Manassas and Manassas Park school districts, have chosen to subsidize the AP and IB testing fees and require students in those courses to take those tests so that they can get the full college experience. Several people have told me that gives those school districts an unfair advantage. To which I say, would you similarly argue that a school district that pays for extra reading tutors has an unfair advantage in getting more seven year olds up to grade level? The point here is to help students learn, and a challenging high school course and test does that.
County policy set by school boards matters. It hurts me to admit this because I often complain that school boards are time-wasting debating clubs that do little to make schools better. The good effect of these AP and IB policies proves me wrong, at least in this case.
But if you look carefully at the Challenge Index list, you will see something else happening. A few schools in districts that do NOT emphasize college-level courses do much better than other schools in the same district, and the reason in nearly every case seems to be one thing -- the thinking of the people in charge of that school.
Why, for instance, does High Point High School in Prince George's County, with 49 percent of its students qualifying for lunch subsidies, have a rating of 1.071 while the average for all schools in that county is just 0.536? The answer is principal William Ryan who, aided by many administrators and teachers, strengthened the school's AP courses, urged more students to take them and made sure that their schedules did not get in the way.
Or why does River Hill High School in Howard County, with just 1 percent of its students qualifying for lunch subsidies, have a rating of 1.871 while the average for that similarly affluent county is only 0.971? The answer is principal Scott Pfeifer and his faculty, who persuaded parents that it was a silly to have so many of their students take AP courses and then, for reasons that did not make much sense, blow off the exams in May.
Or why does Clarke County High School in rural Virginia have a rating of 2.496 when other rural Virginia counties like Stafford (0.630) or Fauquier (0.437) don't even come close? The answer is superintendent Eleanor Smalley and the staff of that high school, who have created one of the strongest IB programs in rural America, overcoming initial skepticism on the part of parents who thought their kids could not handle it.
There are still difficulties in making college level courses work in high school. Veteran teacher Tom Shaffer and recent high school graduate C.J. Hebert, both from Charles County, say the growth of AP participation in their district has produced the startling phenomenon of students who enroll in the college-level courses but do little or no work in them. They appear to think getting bad grades and bad test scores will not hurt them much because having AP on their transcripts will be enough to satisfy their parents and prospective colleges.
Shaffer and Hebert are right to complain, but there are all kinds of ways to identify such slackers and either encourage them to pay attention, or transfer them to another class so that the entrance ticket to AP or IB remains what it should be -- a willingness to do the work. Outbreaks of adolescent sloth should not be used as an excuse to return to the old days of letting just A students take the most difficult courses, and shutting the door in the face of B and C students who want a taste of college academics.
Recently I turned much of this space over to Robina Bouffault, a Clarke County resident who thinks the IB program there is too hard for many students who are, she says, being forced to take it. She has yet provided no evidence of anyone being pushed into a class against their will, but she does raise one valid point. There is little research data yet on how students who struggle in AP and IB courses in high school do in college, other than the 1999 Clifford Adelman report for the U.S. Education Department that says taking any difficult course -- AP, IB, honors -- improves the chances of graduating from college.
Bouffault would like a more precise answer: do students who work hard but do poorly in AP or IB do as well in college as students who avoid those difficult courses in high school? Should we continue to insist, as most school districts with low Challenge Index ratings do, that only students with top grades should be allowed into AP and IB?
Some studies attempting to address this are in the works. I will let you know when the results appear. But in the meantime, I have the Adelman study and 20 years of interviews with less than star students, and their teachers, who say that trying college level courses in high school changed their entire perspective on learning, and made the rest of their academic lives much more fulfilling than they might have been.
At W.T. Woodson High School in Fairfax County, ranked number 7 on this year's list with a rating of 3.223, recent graduate Jeff Taves sent me an e-mail that made the point much better than I could. He was enraged to read of teachers who felt that B and C students just weren't up to the demands of AP and ought to be given easier fare.
"I am an C or B student, pretty much," he said, "not because I am not ready for the work, not because I'm not smart enough, but rather because I am a very smart student who is not very organized and does not complete my homework."
Since he was in Fairfax County, his past sins did not keep him out of AP, and his success on nine different tests was much better than many teachers would have predicted -- one 5, four 4s and four 3s, the equivalent of a college A, four college Bs and four college Cs.
He said he feels better prepared for college, and has to thank the educators who were the living examples of the county's faith in him. "The teachers I had in my school for these classes were wonderful," he said, "and pushed me to succeed."
The fact that more than half of this area's high schools give that chance to students like Taves is something parents should be very happy about and serves as an example for the schools here who have not reached that level. Creating a culture where every graduating senior has taken at least one demanding college level course and test does not seem like so much to ask. The fact that only 4 percent of U.S. schools have done it means we have a lot of work to do.
For Best Schools, Shop and Compare