Evidence-Based Practice, Best Practices, and Other Lies

Follow the Money

Publication Date: 2009-11-30

  By Susan Ohanian  

Teachers, who are often told they should aspire to be more like doctors, might have something to learn from Dr. Groopman's comment on recent changes forced on the medical community.

Dr. Jerome Groopman holds the Dina and Raphael Recanati Chair of Medicine at Harvard Medical School and is Chief of Experimental Medicine at Beth Israel Deaconess Medical Center in Boston. He is also a staff writer for The New Yorker. His latest book is How Doctors Think. He makes this observation in an interview in the Dec. 17, 2009 New York Review of Books.

As you read this excerpt, think about how closely it relates to the current Race to the Top and LEARN (sic) initiatives coming from Washington, D. C.

Jerome Groopman: [T]here are important reasons for having a scientific statistical analysis of evidence. I'm a scientist. I'm a professor at Harvard. I've done the clinical trials in my own field that have led to such "evidence." But I'm also acutely aware of their limitations. Statistical analysis is not a substitute for thinking. [emphasis added] Unfortunately, to my mind, because I voted for President Obama and certainly support many of the current reform efforts, there is a very powerful group with an ideology emphasizing evidence-based medicine, what they call "best practices." That is a wonderful term, because how can you argue with best practices?

But if you look at some of the bills, like the House bill, HR3200, and you look at many of the incentives in the Baucus bill from the Senate Finance Committee, they clearly want doctors not to think and lead, but to simply follow. And the incentives are that you are paid more by adhering to specific guidelines, and according to some proposals, your malpractice liability will be tied to whether you follow guidelines or not.

Now many times, there are patients whose illness don't conform to the direction of guidelines. Many people do not realize that in general the committees that draw up clinical guidelines force a consensus and there are often experts who disagree with some aspects of the guidelines or contend that they are flawed. There are numerous examples of this that are familiar to the public. One was the treatment of nearly all women after menopause with estrogen to prevent heart disease and dementia. We now know that the case for such treatment is far from clear and some credible experts had doubts about it from the start. A recent analysis of more than a hundred evidence-based conclusions about clinical practice reported that after two years more than a quarter of the conclusions were contradicted by new data, and that nearly half of the "best practices" were overturned at five years. This shows that guidelines are not gospel from a scientific point of view. Also, patients have different goals with respect to how much treatment they want, what kinds of treatment, and frankly, how much they are willing to comply with prescribed treatment. And you are punished in this system if your patients don't comply.

And so what's happened in Massachusetts is that patients who are in most need of a caring and communicative doctor, patients who are confused about their treatment, patients who are resistant, patients who don't like to take pills, diabetics who are too poor to eat healthy food--all of these patients now may be shunned by physicians because of the risk that you're going to look bad on a report card. I just learned of an older woman who was very fragile and in the midst of heart attack whose cardiologist hesitated to perform a necessary procedure to open the coronary arteries because her outcome might well be poor, and this could be counted against him in assessing his performance. These are the unintended consequences of much of the movement for what is called "pay-for-performance."

The parallel perils with teachers are even more dire. The LEARN (sic) legislation, introduced by Washington Senator Patty Murray and now sitting at the Senate Committee on Health, Education, Labor, and Pensions, stipulates that "SCIENTIFICALLY VALID RESEARCH" will be employed by people getting money from this act. NCLB has given us long, hard experience with what the U. S. government means by "scientifically valid research." The U. S. Inspector General called federal practices corrupt, but that hasn't put DIBELS out of business. And now this new legislation looks to push DIBELS-type practices into the middle grades and high school.

Phrases like "explicit, intentional and systematic" are peppered throughout the Act.

The Act makes this extraordinary claim:

The intellectual and linguistic skills necessary for writing and reading must be developed through explicit, intentional, and systematic language activities, to which many low-income and minority students do not currently have access.

There is plenty of evidence that low-income and minority students are subjected to entirely too much "explicit, intentional, and systematic language activities," e.g. workbooks, skill-drill sheets, test prep activities. What they lack is access to libraries, access to self-selection of reading materials and time to read books of their own choosing.

The act makes numerous references to "research-based" instructional methods. They fail to say "whose research." Sad experience with NCLB has shown us that when one set of research triumphs, shutting out all dissent, the results are disastrous for many children. Dr. Groopman points to the perils of "evidenced-based" medicine. We could hope that organizations such as the National Council of Teachers of English would learn from history and be very wary of legislation that decrees teachers will follow "research-based" instructional practice. No method works for every child. What makes us professional is the ability to decide when to use what research--and with whom.

As a longtime middle school teacher, I am very frightened by what the LEARN (sic) Act wants to decree for middle graders:

improving student achievement by establishing adolescent literacy initiatives that provide explicit and systematic instruction in oral language, reading, and writing development across the curriculum;

providing direct and explicit comprehension instruction;

providing direct and explicit instruction that builds academic vocabulary and strategies and knowledge of text structure for reading different kinds of texts within and across core academic subjects

Sorry, but I don't see any systematic instruction that would work for an 8th grader who read his first book in my class, Dr. Seuss's Hop on Pop (My principal wouldn't let me use petty cash to join the Beginner Reader Book Club so I joined on my own); Arnold, who, when he wasn't cackling like a chicken, did nothing but take spelling tests for three months (I got lists from the honors classes, from national spelling contests, etc.); Pete, whose letter exchange with me appeared in Education Week; five students who formed their own discussion group for Soul Brothers and Sister Lou; Richard, who was so angry he couldn't read that some days he screamed and other days he curled up on the floor in a fetal position (he also got kicked out of the State University reading clinic).

And then there was the boy who was legally blind and brilliant. He didn't show up to school for the first couple of weeks. I kept asking where he was and kids told me, "Oh, he hasn't come to school since first grade." I set the truant officer on him. And suddenly there he was, one of the most lovely boys I've ever encountered.

He'd broken his very thick glasses, so my teaching partner and I arranged for him to get a new pair through the Lions Club. After he'd been in our class for a few weeks I asked him why he hadn't come to school. He confessed he'd gotten mad in first grade and thrown a chair at the teacher. They kicked him out and he never went back.

I asked him where he got the terrific vocabulary. He claimed it was from watching TV all day. But when I also learned that he had a brother at Harvard, I began to question such cliches as stated in the LEARN (sic) act:

Many low-income children from birth through kindergarten entry lack oral and print language-rich environments in their homes, and early learning programs are often not sufficiently responsive to the range of skills such children may develop in their home environments.

This boy could not see well enough to take the 8th grade CTB/McGraw-Hill achievement test. So we gave him the 6th grade test, which had larger print. He answered every question correctly.

I phoned CTB/McGraw-Hill in California, asking them what score I could put on his test. They hemmed and hawed and said, "You can say he reads at least on a 10th grade level."

I asked them to put it in writing, since the high school guidance counselor was refusing to put the boy in an honors program. CTB/McGraw-Hill refused to put anything in writing. I begged for a letter, telling the CTB/McGraw-Hill how vital it was to this boy's educational future. I think of this refusal when I read LEARN (sic) Act imperative for teachers:

using screening assessment, diagnostic assessment, formative assessment, and summative assessment to identify learning needs, inform instruction, and monitor student progress and the effects of instruction

Will someone in the U. S. Congress tell CTB/McGraw-Hill to cooperate?

Will someone tell the U. S. Congress that screening assessments, diagnostic assessments, formative assessments and summative assessments are not a substitute for thinking?

The existing standardized tests are so incredibly bogus that it is the height of impropriety for NCTE and other so-called professional organizations to pretend that tests exist "to identify learning needs, inform instruction, and monitor student progress and the effects of instruction." Never mind that the last part of this claim bespeaks a factory/delivery model of instruction: the teacher delivers instruction, the student receives it, and the effects of that instruction can be measured.

Why are NCTE, IRA, and others engaged in such a thing? I think the answer lies right here:

(11) HIGH QUALITY PROFESSIONAL DEVELOPMENT- The term 'high quality professional development' means professional development that--
(A) is job-embedded, ongoing, and based on scientifically valid research;
(B) is sustained, intensive, and classroom-focused, and is not limited in scope to a 1-day or short-term workshop or conference;
(C) is designed to increase the knowledge and expertise of teachers, principals, and other school leaders. . .

Aha! As Deep Throat advised in All the President's Men, "Follow the money. . . ." Populist commentator Jim Hightower once advocated "reveal the money," proposing that the 1996 Presidential candidates should disclose the money contributed to their campaigns: "Like NASCAR race drivers or PGA golfers, why not require each of the candidates to cover their clothing, briefcases and staff with the logo patches of their corporate sponsors?"

Anybody giving testimony that teachers need professional development (more, say, than schools need libraries staffed by professional librarians) should wear patches with the names of the districts from which they receive consulting fees. Special badges could indicate associations with publishers and outfits like Achieve.

I have several specific concerns about this call for professional development:

1) It feeds into the corporate-politico drumbeat that teachers aren't qualified.

2) It is likely not the best use of taxpayer dollars in very difficult economic times.

3) Will there be an "approved" list issued by the U. S. Department of Education? Whether it's scripted instruction or whole language, I don't think the federal government has any business telling local districts how to teach their students.

4) There is substantial research showing the direct benefit to students provided by libraries and professional librarians. Where's the research showing the direct benefit from staff development days?

Using Dr. Groopman's language, I would tell the people supporting this legislation that for twenty years I taught students who didn't conform to the direction of guidelines. I have written several books about just how far these students sat outside the guidelines. Jack, the high schooler who played himself in Scrabble for six months. He did that and he read half an hour a day--a book of his own choosing. That was his curriculum. Or Charles, the third grader who read Rumpelstiltskin 16 days in a row--and then I stopped counting. Or Joanna, the timid child repeating third grade, who patiently worked with Leslie, the deaf child in public school for the first time, teaching her knock-knock riddles. I had no idea if such a thing was appropriate for a deaf child, but these children were so determined that I followed their lead. And one day Leslie screamed, "I get it! I get it! Let me read it to the class!" And she did. And the class roared their approval. And Leslie screamed, "Let me read another one!" And she did.

Where are the screening assessments, diagnostic assessments, formative assessments and summative assessments for that? For Leslie and for Joanna.

Where's the "SCIENTIFICALLY VALID RESEARCH" that will allow a teacher today to follow the lead of a child repeating third grade and her deaf friend?

My version of Follow the Money here in Vermont is to point out that Reading First cost $600 per child to implement here. I think communities should have sat down and figured out how best to spend that $600. With Vermont being the 4th hungriest state in the Union, not to mention one with severe winters requiring high fuel bills, I think there would have been a variety of responses. And after all, that is our money. Every person should realize this simple fact. I'd like to see whole communities question the rules for the allocation of Federal dollars in their schools.

Follow the money.

Question the guidelines.

Ask the NCTE Executive Committee to act professionally, admitting they made a mistake and to withdraw their support of the LEARN (sic) legislation immediately. To ERR is human, to admit it, divine.

Past-President, Kylene Beers: kBeers@prodigy.net
Past pres., Kathleen Yancey: kyancey@fsu.edu
Pres., Carol Jago: cjago@caroljago.com
Pres.-elect, Yvonne Siu-Runyan: hanalei@indra.com
Vice-pres., Keith Gilyard: rkg3@PSU.EDU