Reporters Find Science Journals Harder to Trust, but Not Easy to Verify
Now news organizations say they are starting to look at the science journals a bit more skeptically. One can hope this skepticism might spill over to government agents who travel with the title expert in reading. Hope but don't hold your breath. Reporters are entirely credulous when it comes to pedagogy, relying on their own experiences at school and/or their own bias. To my knowledge, a reporter has never questioned the claims of "science" attached to research involved in Reading First. Why is "science" attached to the work of Barbara Foorman and not the work of, say, Ken Goodman?
No reporter has ever mentioned the preponderance of people with doctorates in psychology and/or learning disabilities [with subsequent involvement in scripted curricula] on the Reading First panel of 73 experts, nobody mentioned the lack of anyone with expertise in children's literature, and so on.
By Julie Bosman
When the journal Science recently retracted two papers by the South Korean researcher Dr. Hwang Woo Suk, it officially confirmed what he had denied for months: Dr. Hwang had fabricated evidence that he had cloned human cells.
But the editors of Science were not alone in telling the world of Dr. Hwang's research. Newspapers, wire services and television networks had initially trumpeted the news, as they often do with information served up by the leading scientific journals.
Now news organizations say they are starting to look at the science journals a bit more skeptically.
"My antennae are definitely up since this whole thing unfolded," said Rob Stein, a science reporter for The Washington Post. "I'm reading papers a lot more closely than I had in the past, just to sort of satisfy myself that any individual piece of research is valid. But we're still in sort of the same situation that the journal editors are, which is that if someone wants to completely fabricate data, it's hard to figure that out."
But other than heightened skepticism, not a lot has changed in how newspapers treat scientific journals. Indeed, newspaper editors openly acknowledge their dependence on them. At The Los Angeles Times, at least half of the science stories that run on the front page come directly from journals, said Ashley Dunn, the paper's science editor. Gideon Gil, the health and science editor for The Boston Globe, said that two of the three science stories that run on a typical day were from research that appeared in journals.
Beyond newspapers, papers from journals are routinely picked up by newsweeklies, network news, talk radio and Web sites.
"They are the way science is conducted, they're the way people share information, they're the best approximation of acceptance by knowledgeable people," said Laura Chang, science editor for The New York Times. "We do rely on them for the starting point of many of our stories, and that will not change."
There are limits to the vetting that science reporters, who are generally not scientists themselves, can do. Most journal articles have embargoes attached, giving reporters several days to call specialists in the field, check footnotes on an article and scrutinize the results.
"Scientific discoveries are more difficult because they often require in the generalist reporter a good deal of study, follow-up interviews and some guidance on how to make sense of technical matters," said Roy Peter Clark, a senior scholar at the Poynter Institute, which studies journalism. "But I think the scandals do require both a new level of skepticism on the part of the reporter and also maybe some new protocols between scientists and journalists."
The Hwang case was not the first time journals had been duped: recently, editors at The New England Journal of Medicine said they suspected two cancer papers they published contained fabricated data. In December, the same journal said that the authors of a 2000 study on the painkiller Vioxx had omitted the fact that several patients had had heart attacks while taking the drug in a trial. A study on the painkiller Celebrex that appeared in The Journal of the American Medical Association was discredited when it was discovered that the authors had submitted only six months of data, instead of the 12 months of data they had collected.
While the journals have a peer review process that is in part meant to filter out fallacious papers by checking research techniques and conclusions, perhaps the greatest difficulty for science reporters is trying to catch what journal editors have missed.
After hearing the news of Dr. Hwang's fabrications, Mr. Gil of The Globe said he immediately remembered his newspaper's coverage of the stem cell papers.
"We were blown away, in part because we had written those stories on Page 1," Mr. Gil said. "And when we wrote them, we called the leading experts in the world on all this embryonic stem cell stuff, who are here in Boston. And they were as hoodwinked as anybody else."
Despite the fraud cases, most of what the journals publish is basically credible, said David Perlman, the science editor of The San Francisco Chronicle. Among the most prestigious science journals that reporters consult regularly are Nature, Science, The New England Journal of Medicine and The Journal of the American Medical Association.
"I think they and we have been burned enough that they're making efforts," Mr. Perlman said. "They're being more careful now, and I think reporters are too. I definitely have more of a 'Hey, let's look more carefully' attitude now that I did 5 or 10 years ago."
Donald Kennedy, the editor of Science, said in a statement in December that the journal itself was not an investigative body. But when reporting on journal findings, most news outlets fail to caution that studies must be replicated to be truly authenticated.
"Beyond Hwang, the more fundamental issue is that journals do not and cannot guarantee the truth of what they publish," said Nicholas Wade, a science reporter for The New York Times. "Publication of a paper only means that, in the view of the referees who green-light it, it is interesting and not obviously false. In other words, all of the results in these journals are tentative."
The journals' own peer review processes, which are intended to be the first barrier against fraud, have come under criticism lately. A cover story in the February issue of The Scientist said that the top-tier journals were receiving more submissions every year, overtaxing peer reviewers and weakening the screening process.
After the Hwang scandal, Science announced it was considering a set of changes to better prevent fraud: Dr. Kennedy said in January that new rules could include "requiring all authors to detail their specific contributions to the research submitted, and to sign statements of concurrence with the conclusions of the work," as well as "implementing improved methods of detecting image alteration, although it appears improbable that they would have detected problems in this particular case." (Through a spokeswoman, Dr. Kennedy declined to be interviewed and said the editors were currently conducting a review of the episode.)
Some newspapers have adopted guidelines of their own to check for conflicts of interest involving authors of journal articles. The Globe instituted guidelines last July requiring reporters to ask researchers about their financial ties to studies, and to include that information in resulting articles. In its weekly health and science section, The Globe outlines any shortcomings of a study under the heading "Cautions."
Kit Frieden, the health and science editor for The Associated Press, said: "We've always had our own peer review process, where on the major studies we seek outside expert comment. We've always regarded scientific research cautiously because mistakes can be made, and I don't think that's changed."
The growing competition for the most important research among the journals may contribute to mistakes and fabrications, even in the most prestigious of the bunch. But in the end, the severe consequences of presenting fraudulent research generally act as a deterrent, said Mr. Dunn of The Los Angeles Times.
"Unlike financial fraud, where you can bamboozle somebody of their money and disappear and then start over again, in science the researchers are in one place," he said. "If they get caught in this type of thing, their careers are over."
New York Times
INDEX OF NCLB OUTRAGES