Here's how to talk back to the numbers purporting to show that rising SAT scores are proof high stakes testing is working.
Ohanian comment: This article is worth reading for the rebuttal supplied by Mickey VanDerwerker, leader of the grass roots student advocacy group Parents Across Virginia United to Reform SOLs.This is an activist group who don't just complain about high-stakes testing. They talk back, refuting Standardisto claims with solid research, evidence from the schools, and agony from families struggling to make school a joyful and meaningful experience for children. This group offers commitment, enthusiasm, and practical advice to others who are ready to stand up and fight.
This also illustrates an advantage of participating in the ARN-state discussion group. People doing activist work in their individual states get advice from their counterparts across the country.
Not surprisingly, this article was posted by someone in Maine. Mickey provided an answer based on her experience in Virginia. and I gave a Vermont perspective.
So read the article. Then read Mickey's response, which will be useful to resisters in all 50 states. Also note that by issuing a press release explaining the SAT scores the Virginia group also provides proof to local media that they are a group that knows its stuff, a group to be contacted on education matters.
August 31, 2003
Maine SAT scores lagging
By TOM BELL , Portland Press Herald Writer
Maine students are falling behind their peers in most New England states when it comes to taking the SAT, according to an analysis of test scores over the last five years.
Some say the scores highlight the success of the aggressive school reform tactics favored in other states in contrast to Maine's cautious, low-cost approach.
The mean score for Maine students on the SAT Scholastic Assessment Test's math test section in 2003 was lower than the scores of five other New England states. On the SAT verbal test, students in every New England state except Rhode Island outperformed Maine students. Also, Maine and Vermont were tied for the lowest participation rate.
"The rest of the country seems to have made a concerted effort to pull their scores up. Maine is simply not doing well on these scores," said Frank Heller of Brunswick, coordinator of the Maine School Choice Coalition. "I don't think the governor can ignore this slippage anymore. It's a serious problem."
The College Board, which owns the nation's most popular entrance exam, last week released the scores, which many colleges and universities use as a factor for deciding admissions.
While the SAT has long been viewed as a strong predictor of college success, critics say it mainly assesses a student's ability to take a multiple choice test. They say it provides an unfair advantage to students whose parents have higher incomes, have gone to college and can afford to pay SAT tutors to coach their children.
Maine has the lowest average incomes in New England, and the lowest percentage of college graduates. Fifty-four percent of Maine's high school graduates enroll in a college the year after graduation. In that statistic, Maine is ranked 33rd among the 50 states.
In Maine, the class of 2003 had an average SAT score of 503 on the verbal section, four 4 points lower than the national average. The average math score for Maine students was 501, which was one 1 point lower than last year and 18 points below the national average. The national average, however, is inflated because only the best students in the West and South take the test.
Bill Hiss, vice president of external affairs at Bates College, said rural students from blue-collar families tend to do poorly on the test. Bates does not require students to submit their SAT scores, and most Mainers who attend Bates don't. Yet, Maine students traditionally have excelled academically at the college, he said.
He said the cultural landscape in Maine is vastly different from the affluent suburbs of Massachusetts or the well-to-do urban neighborhoods where private schools are the norm.
He recounted a recent visit to a friend's house in San Francisco. The visit was interrupted when the an SAT coach arrived to tutor his friend's child.
"Can you think of a single kid in Maine who'd have an SAT tutor come to their house?" he asked. "It would be impossible, financially and geographically."
But the SAT trends show more than cultural differences. It appears Maine's college-bound students do not take as many advanced courses as their peers in other states.
Nationally, both math and verbal SAT scores have been increasing in recent years. The class of 2003 achieved the highest score on the verbal section in 16 years. Students' scores in the math section of the test hit were the highest in at least 36 years.
The College Board said the higher math scores could be attributed to increased participation in advanced courses such as math, physics, calculus and chemistry.
According to a Maine Department of Education analysis of course-taking patterns reported by SAT, Maine students are less likely to have taken the highest level of mathematics instruction. Nationally, for example, 25 percent of students report taking calculus, compared to only 21 percent of Maine students.
Several New England states have showedn marked improvement over the last five years in the test. The biggest jump occurred in Massachusetts, where students now must pass a statewide exam to graduate and where schools and teachers since 1998 have been held increasingly accountable for student performance. The state's math scores since 1998 have risen 14 points, and verbal scores are up 8 points.
Remarkably, Massachusetts students managed to outperform their Maine peers even though the Bay State has a much higher percentage of immigrants and minorities, groups that historically do not perform as well on the exam as whites. Also, a greater percentage of the Bay State's Massachusetts' graduating seniors - 82 percent - took the exam. Maine's participation rate was 70 percent.
Connecticut, which has focused on improving teacher quality by boosting their salaries and raising hiring standards, also showed improvement. Connecticut's mean math score rose 5 points.
Vermont's mean verbal score rose 7 points, and its math score 8 points. Rhode Island's verbal score rose 9 points.
Students in New Hampshire and Maine showed no improvement. In fact, the math scores in both states dropped one 1 point.
In a press release, Maine Education Commissioner Sue Gendron focused on the positive, noting that the rate of student participation in Maine moved up one percentage point from 2002. She said the continued growth in the participation rate indicates that the number of students who expect to go to college is increasing.
Deputy Education Commissioner Patrick Phillips said Maine's teachers and students are dong a great job and are working hard to meet the state's Learning Results standards. But he added that state officials are concerned that Maine's SAT scores are not keeping pace with national trends.
He noted that Maine has chosen to avoid "draconian" reform measures, such as high-stakes testing, and is giving local schools discretion on how to assess standards.
He said limited state funding is also a factor contributing to Maine's stagnant scores. Although the Maine Legislature has defined what schools must do to provide an adequate education, he said, the additional state funds that schools need to meet those standards won't fully arrive until the end of this decade.
He said officials plan to study the policies of states that have seen their SAT scores jump. He noted that Maine girls perform as well as Massachusetts girls on the verbal test, but Maine boys have fallen significantly behind. He said he wants to find out what Massachusetts is doing to boost the scores of its boys.
John Kennedy, who analyzes testing data for the state, noted that the large gaps in achievement between rich and poor school districts in Massachusetts do not exist in Maine. Also, minorities in Maine do as well as whites, he said.
The biggest gap in Maine, he said, is between girls and boys. Boys do better on the math test, and girls are better on the verbal test.
Duke Albanese, the education commissioner during Gov. Angus King's administration, said the National Assessment of Education Progress test is a better indicator of the high quality of Maine's schools. The test is designed like a poll. While only a portion of the state's students take the test, the results are considered to represent the performance of all the students.
Massachusetts and Maine students both routinely score well above the national average on the tests, which are is given to students in fourth and and eighth grade. On that test, the scores of Massachusetts students have improved over the past decade, while the Maine scores have remained about the same.
In those that tests, the math scores of Massachusetts"s fourth- and eighth-grade students improved between 1992 and 2000, and reading scores improved between 1992 and 2002.
The reading scores of Maine's fourth- and eight-graders declined slightly during that period, and fourth-grade math scores also declined a bit. Only eighth-grade math scores improved.
Mickey VanDerwerker responds:
Remarkably, Massachusetts students managed to outperform their Maine peers even though the Bay State has a much higher percentage of immigrants and minorities, groups that historically do not perform as well on the exam as whites. Also, a greater percentage of the Bay State's Massachusetts' graduating seniors--82 percent-- took the exam. Maine's participation rate was 70 percent.
Be careful of the statistics games being played. Now I have not gone and looked at raw numbers BUT here are the things to think about.
1. If the lowest achieving students stopped taking SATs, average scores go up.
2. If we have forced/encouraged those lowest achievers out before senior year, then the number of seniors is smaller than it used to be.
3. If the senior class is smaller because we have encouraged the low achievers out (by way of high stakes testing and other interesting tactics), then the smaller, more high achieving group that is left has more kids in it who are likely to take SATs. Voila, the state can claim a higher percentage of kids take SATs when actually what is happening is pushouts. Figure out the proportion of SAT takers compared to the ninth graders three years earlier.That'll tell a
different story, betcha.
4. Watch how they report SAT scores. The SAT report combines both private and public schools. Private schoolers, at least those here in VA, score way higher than public schoolers. If a good percentage of your takers are private
schoolers, voila again, the average rises. And if you are comparing to a state with fewer private schools, then voila your rises look so much better.
If you want to know where to look for your figures, give me a holler. Here's our VA take:
"Overstating the SAT case"
You'll be reading in your Wednesday paper that SAT scores in VA and across the nation are rising. For students who have their sights set on colleges that require SAT scores (about 400 and counting no longer do), that news is good.
However, once again, the VA Department of Education (DOE) uses a plethora of raw numbers mixed with percentages and proportions and combines public and private school data to overstate its case for rising achievement due to SOLs.
The DOE press release appears to credit the SOL for increases in SAT scores that include the scores of private school as well as public school students.
Overall, Virginia seniors, public and private, scored an average 514 on verbal and 510 on math. Students in private schools score higher than seniors in public school (which isn't surprising given the well-documented correlation between
socio-economic factors and standardized test scores). Including their scores obviously inflates the average score for all Virginia seniors.
Average scores for public school seniors alone were 512 on verbal and 508 on math, a rise of 6 and 4 points, respectively. Nationally, public school seniors scored 504 on verbal and 516 on math, a gain of 4 points on each.
While public school SAT scores have risen, private school scores have risen even higher. Still, the DOE credits SOLs and more challenging coursework for the growth in public school scores, even though private schools do not require SOL tests, nor are their curricula aligned with the SOLs. (Between 1998 and
2003, scores for public school seniors have risen 8 points in verbal and 11 points in math. For private religious schools, the rise has been 12 points in both verbal and math. For private independent schools, the rise has been 11 points
in verbal and 18 points in math).
The DOE also mixes and shifts between raw numbers and percentages in what seems an attempt to befuddle the average reader. For example, its press release notes that there were 3,528 more students taking SAT I this time. The number of
students taking SAT I is rising, but so is the total number of seniors in Virginia's public schools. As a proportion of the senior class (based on the September 30 enrollment figures), there has been a slight decrease,1%, in test
takers since 2002.
Similarly, the DOE notes that the number of African American test takers increased over last year. Again, using the raw numbers, this is an accurate statement. However, the proportion of African-American seniors who took SAT I actually decreased, from 46% in 2001 and 2002, to 44% in 2003 (using figures from DOE's Report of Fall Membership by Grade and Ethnicity). The mean combined score for African Americans rose, though still not to the levels achieved in 1995.
Meanwhile, the score gap between white students and African American students did not get any smaller.
For the SAT II Math and English, which more colleges are beginning to require, the percentage of all Virginia seniors taking English writing remains flat at 14%, while the percentage taking Math IC and IIC is down. The information on the College Board site does not allow for determining how public school students alone fare. Starting in March 2005, the SATI will include an SATII-type writing test.
In any case, we would once again reiterate that no test scores - including SAT - should be used as definitive measures of student achievement, school quality or rankings, because so many factors influence test scores and go into school and educational quality (as even the College Board, like other test makers and test experts, explicitly states). Education is more than a score.