Focus on Ratings Discouraged
by John Austin and L. Lamor Williams
The scores are in, the ratings are tabulated, and everyone in Texas will
know today just how good a school his or her child attends, thanks to
the state's latest research and rankings.
At least, that's the theory.
But parents, students and administrators need to look at a lot more than
the ratings to really know the score, according to several educational
"In general, tests cover only a very small part of schools' academic
curriculum, much less the broader non-academic aims of public
education," Walt Haney, an education professor at Boston College, said
in an e-mail. "And even on tests of academic subjects such as English
and math, schools' grade-level results from year to year may vary.
"As any teacher knows, one year's class of 30 or 40 kids may vary
substantially from the next year's just because of Mother Nature's
variation in which kids are born in which year," said Haney, who is also
a senior staff member of the Center for the Study of Testing, Evaluation
and Educational Policy in Boston.
Districts are assigned ratings based largely on how students perform on
the Texas Assessment of Knowledge and Skills. Each student group --
black, white, Hispanic and economically disadvantaged -- must pass the
exam, which covers reading, math, science, social studies, writing and
English/language arts. The state also considers the annual dropout rate
of seventh- and eighth-graders and the completion rate of high school
In addition to these ratings, the state tracks student progress with
TAKS scores from each administration of the exam. The federal government
also labels schools according to whether they make adequate yearly progress.
Some area administrators caution that ratings by themselves tell only
how students did on a particular day and that a handful of children
having bad days can drag down an entire district's rating. Many schools
have had their ratings drop as a result of one student's performance on
one portion of the TAKS.
"It's one of the many pieces of information that people should use, but
it can also distort the picture of what really goes on in a school,"
said Arlington Superintendent Mac Bernd.
They liken the system to giving students, who study several subjects,
only one grade on their report cards: A child who just couldn't do well
in gym class but made A's in every other subject would get a B on his
The system is a measure of the lowest performance, not the highest,
according to educators.
Administrators say it's difficult to compare districts solely on ratings.
For example, a predominantly white and wealthy district may just barely
earn the top rating, while a low-income district with many minority
students -- many who aren't native English speakers -- may do a better
job of teaching but still be rated lower.
Haney blames politicians and the press, who, he wrote, "wildly
over-promote such rankings" for some of the public's ratings fixation.
"Another reason is, many, if not most people, have a very poor
understanding of statistics and have little inkling of how much error is
associated with numerical rankings," he said.
Robert Schaeffer, public education director for the National Center for
Fair & Open Testing, is also critical of what he considers a misguided
emphasis on ratings.
"High school ratings affect real estate prices," because parents often
attempt to buy into districts they perceive to be better than others,
"And to some extent there's bragging rights.
"It becomes a positive feedback loop in which good schools appear to get
better. It drives people to judge schools based on a simple-minded set
of factors on which the ratings are based."
Like Haney, Schaeffer pointed out that the data that ratings are based
on is sometimes "phony."
"The ratings are essentially useless in many cases," Schaeffer said.
Economist Steven Levitt and author Stephen Dubner say in their
best-selling book Freakonomics that moving to a better neighborhood
doesn't improve a child's chances in school.
And Schaeffer said that while a highly rated school may be doing a great
job, that school may also have a student body made up of
upper-middle-class kids whose parents are well-educated and who
reinforce and supplement what goes on in class.
"It's called cherry-picking," he said.
At a school where students come from a lower socioeconomic group and
where parents may not be well-educated, or even speak English, teachers
may actually be doing a better job than those elsewhere, even though the
ratings don't reflect their gains: They simply have further to go.
Schaeffer also urges parents to be skeptical about claims that TAKS
scores and other test results reflect big gains for Texas students.
"Your colleges [in Texas] report more kids needing more remediation," he
said. "Now how is that?"
Anne Ware, coordinator of program evaluation for the Fort Worth school
district, agreed that ratings are only part of the picture.
"You can't just determine success or failure by one item," Ware said.
"But the test scores make the headlines."
She too urged parents to get involved, even though it can be difficult.
"If the parents start demanding more, schools will do more," she said.
Above all, remember that even in a school that is not rated "exemplary,"
there may be "extraordinary people doing great work," Schaeffer said.
IN THE KNOW
About the ratings
To assess a school's real worth, parents are advised to:
• Consider SAT scores: They help indicate whether teachers have been
"teaching to the (TAKS) test" to the exclusion of broader goals.
• Consider "real" dropout rates. Use this formula: For a given
graduating class, add the number of students in the class as freshmen
and the difference between transfers in and out over the next four
years. Then divide the number of graduates by that number. Texas does
not use this formula.
• Consider the percentage of graduates who go to college or the
percentage who are employed after high school.
• Go to school several times during the year, not just on open-house
night. Sit in on classes. Go to the library. Find out what's in it and
whether students are using it.
• See what kind of work students are doing. Are they writing more than
the few paragraphs that many standardized tests require?
• Check for art and music programs.
• Talk to other parents.
• Accountability ratings will be released at 1 p.m. today at
• National Center for Fair & Open Testing: www.fairtest.org
• Center for the Study of Testing, Evaluation and Educational Policy:
John Austin and L. Lamor Williams
Fort Worth Star-Telegram
INDEX OF NCLB OUTRAGES