An Answer to Standardized Tests
Sara Monempour was 2 when her family moved from Tehran to L.A. Then she did what most new Americans do: learned English.
Attending Los Angeles County public schools, Monempour excelled in class but scored "unbelievably low" on standardized reading tests, up to and including the SAT.
Then she noticed most of her bilingual classmates did poorly, too. "We were raised here ... and yet this pattern was always a factor," says Monempour, who spoke Farsi at home. "People who speak a different language at home or with their friends and family would have issues with testing."
Now 23 and a doctoral student at the University of California-Los Angeles, she hopes to become part of a small but growing group of elite researchers, known as psychometricians, who do little else but think about standardized tests.
Once considered an afterthought, standardized testing now drives education's most important decisions: whether students graduate, whether schools "need improvement" and even whether teachers keep their jobs.
No Child Left Behind, President Bush's education reform law, more than doubled the number of standardized tests schools must give each year, and it very likely will double it again in coming years.
Trained in both psychology and statistics, psychometricians work in school districts, education departments and private testing firms to make sure standardized tests actually test what kids know, quickly, fairly and accurately.
To meet the constant demand, the testing industry is expanding "far faster than the supply of competent people," says Bob Schaeffer of the non-profit Center for Fair and Open Testing (FairTest).
Only about a dozen universities have psychometrics programs. Most turn out only one or two graduates a year. But once they're on the job, psychometricians burn out quickly, observers say. For one thing, new federal requirements say tests given in the spring must be processed before students return in the fall, months earlier than in the past.
"It isn't the first of October, it's the first of August — and they don't mean the second," says Richard Hill, executive director of the National Center for the Improvement of Educational Assessment.
Quick turnaround times affect the quality of tests themselves, critics say. For every question that eventually appears, test developers usually field-test four. But these days, they don't always have the time for that level of scrutiny.
Also, most states now require that test questions be made public after they're given, says James Ysseldyke of the University of Minnesota's College of Education. "Once you do that, you've got to start from scratch, building a whole new test — that's incredibly expensive."
A fifth-grade teacher may be able to whip up a respectable math or reading test on a PC in half an hour, but it takes about 18 months to develop a standardized test that is challenging, fair, and comparable to previous versions and other tests.
Enter the psychometricians.
"If you're going to be good in this job, you have to be just a little bit compulsive," says Michael Walker, who works on SAT, Advanced Placement and other exams for the Educational Testing Service. "You have to pay attention to details, because we can't make a mistake."
Importance of tests
Most test questions are field-tested with kids — the SATs, for instance, include a handful of questions that won't affect students' scores but are simply being tried out for future tests.
Even the simplest math or reading problem may be the result of months of research to determine if it's clear, fair and difficult enough — but not too difficult.
When things go wrong — when, for instance, more low-scoring students than high-scoring students answer a question correctly — psychometricians call this "differential item functioning," a fancy term that means things went haywire.
"Errors aren't new," Hill says. "What is new is that the stakes are so high."
Occasionally, things go haywire on the real test, with real consequences: Thousands of students in Texas, Massachusetts, Minnesota and other states have been wrongly told they failed high school graduation exams or other key tests; some have sued.
Such problems don't affect just students. At least 20 lawsuits have been filed this year after more than 4,100 teaching candidates were incorrectly told they had failed ETS' nationally administered Praxis teacher preparation exam.
It's worth pointing out that even psychometricians say such tests shouldn't be the sole criterion determining whether teachers can teach, whether schools pass muster, students graduate or colleges accept them.
"Test scores are limited in what they tell you about a person, and test users don't always keep that in mind," Walker says.
For her part, Monempour, who's at UCLA with the help of a fellowship from testing company Harcourt, says she's surprised by how important testing has become.
"I really never thought it was going to get this huge," she says.
Studying the effects of testing on English-language learners, she says, is an issue that must be addressed.
"If we need to measure students' achievement and capabilities, we need to do it right."
Growing field fills in blanks
In the new test-driven world of education, the field of the moment is psychometrics.
Trained in psychology and statistics, psychometricians work for school districts and testing companies; while the jobs require a Ph.D., new psychometricians can often command $100,000 salaries, more than twice as much as in most other academic disciplines and far more than most teachers or principals. Many psychometricians juggle competing offers from testing firms before they've even completed their course work.
To meet the growing demand, the industry is expanding rapidly - some say too rapidly.
"Certainly the need (for tests) has increased, in good part because of No Child Left Behind," says University of Iowa professor David Frisbie, who oversees the Iowa Test of Basic Skills, one of the nation's oldest standardized skills tests. "I don't think we're coming close to filling that need."
"It's a hot, hot, hot field," says Seppy Basili of Kaplan, the test prep firm. "Nobody knows about it and there aren't very many of them, but everybody is looking for them."
At Educational Testing Service's headquarters near Princeton, N.J., Deanna Morgan, 35, works on California's high school exit exam and other state tests. She taught middle school and high school for seven years before earning her Ph.D. and believes testing is here to stay.
"It's going to happen with me or without me," she says. "I'd rather have my hand in it."
INDEX OF NCLB OUTRAGES