The Mismanagement of Reading First: Summary of Evidence, Part 2
Susan Notes: This report documents why Reading First is known as the Enron of education policy.
Elsewhere, the legislation defines what kind of research a program must have to be "proven to teach reading:"
• Employs systematic, empirical methods that draw on observation or experiment;
• Involves rigorous data analyses that are adequate to test stated hypotheses;
• Relies on measurements or observational methods that provide valid data…;
• Has been accepted by a peer-reviewed journal or approved by a panel of independent experts through a comparably rigorous, objective, and scientific review."
According to P.L. 107-110, ss. 1201 (20 USC 6361), the purpose of Reading First is "To provide assistance to State educational agencies and local educational agencies in selecting or developing effective instructional materials (including classroom-based materials to assist teachers in implementing the essential components of reading instruction), programs, learning systems, and strategies to implement methods that have been proven to prevent or remediate reading failure within a State."(emphasis added). In 2002, when then-Assistant Secretary of Elementary and Secondary Education Susan B. Neuman invited states to apply for Reading First, she wrote, "Reading First is the academic cornerstone of the No Child Left Behind Act, which recognizes the importance of both improving student achievement and implementing programs and strategies scientifically proven to be effective." Indeed, the Department repeats this message on its web site: "Reading First is a prime example of the No Child Left Behind law's emphasis on programs and teaching methods that have been proven to work." The Department noted:
• States and local schools have the flexibility to determine how reading programs are selected, as long as the selected program has been scientifically proven to work (emphasis added).
• There is no federally prescribed reading program.
• States are responsible for the quality of the local programs they fund, and for ensuring that these programs rely on scientifically based reading research. http://www.ed.gov/programs/readingfirst/nclb-reading-first.html.
However, apart from repeated rhetoric, it is clear that the US Department of Education has acted to minimize, and in many cases prevent the use of programs that meet these research standards in favor of those without any evidence of effectiveness.
Among the five top-selling basal textbooks (the "Michigan list"), there is only one text examined in a single study that met these standards. In contrast, there are dozens of rigorous experimental-control comparison studies published in peer-reviewed journals showing the effectiveness of Success for All, Direct Instruction, and a few others. With such a clear focus on rigorous research, how could Reading First have ended up virtually mandating programs lacking such evidence?
One answer is that in practice, Reading First administrators applied a much less rigorous definition of "based on scientifically based research." This definition requires only that programs incorporate elements that have been rigorously evaluated. In practice, this lesser standard has meant that programs must emphasize the five elements of effective reading instruction derived by the National Reading Panel (2000): phonemic awareness, phonics, vocabulary, comprehension, and fluency. Since virtually every reading program contains at least some focus on these elements, this sets an extremely low standard.
1. The U.S. Department of Education re-purposed a "Consumer's Guide," developed by the University of Oregon under a previous contract, to assist states in evaluating instructional programs. Although Ray Simon wrote in a letter to Senator Lugar that "states were under no obligation to use the Consumer's Guide as their review criteria, and many chose not to," there is significant evidence to conclude that states who wished to examine any reading program other than the five basals included by the State of Michigan were compelled to use this Consumer's Guide. The Consumer's Guide is specifically required in many reviews of state applications, and an e-mail from Sandi Jacobs, Chris Doherty's assistant, told Missouri officials to "say more about [the Consumer's Guide]. It is one of the main things the reviewers are looking for."
In its review of Reading First, the Center for Education Policy (CEP) found that:
Use of the original form of the Consumer's Guide is itself not problematic. It had two sections. The first section asked the reviewer to determine if there was sufficient scientific evidence on the program that had determined it to be effective. The second section, to be used only if the first section did not find evidence on the program itself, asked reviewers to examine the extent to which the program addressed the five components of reading instruction identified by the National Reading Panel.
According to numerous state proposals, Drs. Simmons and Kame'enui and others from the University of Oregon provided extensive training to many states in use of a version of the Consumer's Guide that removed the first section, thereby removing any examination of the evidence on any instructional program. This follows the method that apparently was used by the University of Oregon when the Oregon review was carried out. Programs were rated only on section two, the extent to which programs addressed the five components of reading instruction. In fact, section one of the Consumer's Guide was used only in one case we were able to document – the coalition review comprised of Washington, Alabama, Montana, and Idaho. Simmons and Kame'enui officially revised the Consumer's Guide in late 2004 to eliminate section one entirely.
2. The Florida Center for Reading Research (FCRR) was also given the task of evaluating programs, initially for use in Florida and then as a resource for other states. FCRR reviewed not only the programs, but also the research on those programs. Originally, its web site contained two charts, one on the five components and one on the research evidence behind each program reviewed. The research evidence chart was removed in mid-2004 and has not re-appeared. Further, as noted earlier, FCRR's web site "approved" the six top basals without review and only reviewed research on other programs.
3. The US Department of Education's promotion of the Oregon list and the version of the Oregon Consumer's Guide that excluded evidence of effectiveness as a criterion sent a clear message to states that they did not need to, and should not, consider evidence of effectiveness when reviewing materials. If the US Department had instead promoted the Washington coalition review process, which examined evidence in a meaningful way, or the original (later suppressed) Florida Center for Reading Research review, it would have had the opposite effect on state actions. In fact, if the US Department of Education had not promoted any one of these reviews, one would expect that states would have adhered to the legislation and guidance which, as excerpted above, are clear about requirements for evidence of effectiveness.
The US Department of Education also ensured that evidence of effectiveness would not be used through its Reading First Expert Review Panel, which reviewed all state applications for funding. Across the applications, most states initially deferred to local control, stating that their districts would review instructional programs according to their correspondence with scientifically based reading research. However, this was not sufficient, according to both the reviewers and ED staff. They had to use Section 2 of the Consumer's Guide, as noted earlier.
The review panels further required that the districts' ratings of programs be reviewed by "national experts." The review panels did not sign off on any state application for funding until the state guaranteed that it would evaluate districts' choices in consultation with "national experts in scientifically based reading instruction." In some cases, those national experts are even named. With few exceptions, the named reviewers were closely connected to the University of Oregon and University of Texas individuals who are authors or consultants to commercial basal and supplementary texts (Kame'enui, Simmons, Good, Vaughn), or they were connected to Louisa Moats of Sopris West, another commercial publisher.
4. The State of Oklahoma attempted to
follow the legislation in writing its state application. Oklahoma submitted four iterations of its proposal. In the first two, Oklahoma wrote that it would require districts to use instructional programs that had at least three years of longitudinal research on their effectiveness, a criterion that is fully in keeping with the legislation. The state was urged by reviewers twice not to do this because it would limit districts to only a few programs. Oklahoma relented on the third application, and in the fourth application – which was funded – Oklahoma agreed to restrict schools to the traditional basals on its state adoption list.
Other states, however, specified lists of instructional programs that were as limited in numbers of qualifying programs as Oklahoma's would have been. California, for example, approved only 2 programs – the two textbooks that had already been approved through their state adoption process. There are no comments by Federal reviewers on the limitations of the California list.
Scientific review was not necessary if states chose the five top basals. Michigan specified in its response to a public information request by the Success for All Foundation that it had simply "place[d] phone calls to all major publishers requesting examination copies for review." Neither reviewers nor the US Department of Education objected to this procedure that explicitly ignored scientific evidence. In fact, the Florida state web site explicitly excused the six top basals from any need for review.
The U.S. Department of Education's promotion and virtual requirement of the three-tier model is itself a policy in opposition to the use of research on program effectiveness. The three-tier model, which has never been evaluated in even a single study in comparison to a control group, reinforces the idea that research is only relevant when the Reading First leadership wants it to be. There is no need for research on an instructional model's effectiveness – it is enough, according to the US Department of Education, that the model be "based on research," regardless of research on outcomes or effectiveness, and even this minimal requirement was waived if states promised to use any of the top five basal texts.
Have There Been Conflicts of Interest
Among Reading First Leaders and Consultants?
The story of the mismanagement of Reading First is one of extraordinary disregard for commonsense ethical standards regarding conflicts of interest. Reading First contractors are authors of major basal series that have benefited enormously from Reading First funding. Several are consultants to Voyager, which has aggressively exploited the supplemental text niche created by Reading First. In an early 2005 conversation with Reading First director Chris Doherty, we pointed out these conflicts. He was undisturbed, said the individuals had all been vetted by the ED, and pointed out that if you want "the best people in reading," they were bound to be involved with publishers. Yet most of the conflicted individuals were virtually unknown in the reading world before Reading First; most who had any reputation at all had it within special education, not reading. There are many reading experts without such conflicts. The disregard for ethical standards is illustrated by one small but particularly extraordinary fact. On the Oregon Reading First technical assistance center web site under the heading "curriculum," there was for several years a link to the Scott Foresman web site, including information on the Scott Foresman Early Reading Intervention program authored by former center directors Ed Kame'enui and Deborah Simmons. It is unconscionable, of course, for a federally funded technical assistance center to openly promote a product that pays royalties to its directors. This link was mentioned in a press report in August, 2005. Yet for many months thereafter, the link was still there. More importantly, many of the conflicts discussed here have been reported in the press in articles going back to April, 2005. Yet none had been acknowledged or addressed six months later. An October 11, 2005 letter from RF Director Chris Doherty to RMC Director Everett Barnes did ask him to ask contractors in the technical assistance centers with conflicts of interest to recuse themselves from conversations about their programs. Yet by this late date, all states and almost all schools had already received their funding and adopted their programs.
Based on voluminous correspondence among these individuals and between the U.S. Department of Education and state Reading First programs, Reid Lyon, Edward Kame'enui, Deborah Simmons, Sharon Vaughn, and Louisa Moats were in a tremendously influential position. Correspondence attests to the fact that these individuals approved providers of instructional materials or professional development used in many states. For example, the federally approved Alabama proposal required that every reviewer of local proposals had to be approved by "one of the chairpersons of the Secretary's Reading Leadership Academy": Kame'enui, Vaughn, Moats, and Carnine. Three of these four (all but Carnine) have well-documented conflicts of interest. North Carolina's and Washington's approved proposals contain identical language and an identical list.
1. Edward Kame'enui and Deborah Simmons. Edward Kame'enui and Deborah Simmons were until 2005 the directors of the Reading First technical assistance center at the University of Oregon. They have a particularly long list of conflicts of interest. Both are authors of the Scott Foresman Early Reading Intervention program widely adopted under Reading First. Kame'enui has now joined the U.S. Department of Education as Commissioner of Research on Special Education, and he filed a financial disclosure form on taking his new job. It showed that he was earning $100,000 to $150,000 per year from this textbook. In addition, Kame'enui and Simmons are authors of Reading Street, the 2007 Scott Foresman basal series. They were both members of the "design team" for Voyager, the popular commercial "tier 2" supplementary program under Reading First. Kame'enui only resigned his Voyager consultancy in 2005, when he accepted his new job. Both Kame'enui and Simmons have long careers involved with the Reading Mastery program. Despite these many involvements with commercial publishers, Kame'enui and Simmons led the Oregon review panel, and served on those of several other states. They are the authors of the Consumer's Guide virtually required in most states to evaluate Reading First reading programs. They served as consultants to a large number of states on selecting instructional programs and materials, and recommended current and former colleagues to serve in other influential positions in Reading First at the state level, such as technical advisors to states during the application process for Reading First.
The fact that Kame'enui and Simmons were Scott Foresman authors is particularly important in light of the evolution of the "Oregon List" of basal textbooks that states and districts across the country used to determine which programs would be implemented under Reading First funding. As noted earlier, the original "Oregon List" of 2002 listed Success for All fifth, ahead of Scott Foresman. Scott Foresman was felt by many in the Reading First world to have too little emphasis on phonics, and the Florida web site recommended using Scott Foresman only with a phonics supplement.
In 2004, the Oregon center, led by Kame'enui and Simmons, revised its list of recommended basals. The only change was to move Success for All from fifth to seventh, and to move Scott Foresman from seventh to fifth. This change now made it possible for states to require the "top five" or "top six" basals on the Oregon list and thereby include all of the five favored basals, including Scott Foresman. Recall that the publisher of Scott Foresman, Pearson, was already paying Kame'enui and Simmons $100,000 to $150,000 per year in royalties, and had made them authors of America's top-selling reading text, ensuring them personal royalties of many hundreds of thousands of dollars per year.
The low ethical standards of the Oregon group is also illustrated by the fact that until recently, the Oregon Reading First Technical Assistance Center had on its web site a link to the Scott Foresman web site, conveniently set up so that the Kame'enui and Simmons Early Reading Intervention could be "added to your shopping cart."
As noted earlier, Kame'enui has now taken a position at the U.S. Department of Education and Simmons has moved to Texas A & M University, but both continue to be very influential in Reading First and continue to profit from sales of programs promoted by Reading First.
2. Sharon Vaughn. Until June 2005, Sharon Vaughn was the director of the University of Texas Reading First technical assistance center. She was the chair of the design team for Voyager, a major beneficiary of Reading First policies, and remains involved with Randy Best, Voyager's founder. External employment forms from her university reveal that for several years she has spent 10 percent of her time as a Voyager consultant. She is also an author of Reading Street, the 2007 Scott Foresman basal series, and is a frequent consultant for Pearson, the publisher of Scott Foresmanï€ª. Scott Foresman is another major beneficiary of Reading First policies. Vaughn is a major proponent of the three-tier model and has been particularly outspoken in promoting this model within Reading First. Like Kame'enui and Simmons, despite her apparent conflicts of interest she has played a central role in organizing national Reading First conferences, serving as a consultant to many states, and recommending others as consultants.
3. Roland Good. Roland Good is a professor at the University of Oregon and is the main developer of DIBELS. Although DIBELS is available free on the web, most schools purchase it from the for-profit Sopris West company, which acquired it in 2003. Voyager Passport is built around DIBELS, and the virtual requirement that RF schools use DIBELS has therefore been a major selling point for Voyager. He has also been a member of the Voyager design team. Good served on the Oregon Reading First review team and on the panel that reviewed early reading assessments (and approved DIBELS but not other widely used measures). He has served on several other state review teams and as a consultant to several states.
4. Louisa Moats. Louisa Moats is an employee of Sopris West, a for-profit publishing company that publishes her LETRS training program as well as DIBELS and programs often used as supplementary texts under Reading First, especially one called Read Well. Although Reading First has been careful to keep vendors out of all national Reading First events, Moats, a vendor herself, has been central to planning and execution of Reading First trainings. Michigan listed Moats as the only person capable of providing the specific expertise required by the U.S. Department of Education, and offered her a sole-source contract. State officials typically preferred trainers from their own states but were usually forced to choose either LETRS, or training from Kame'enui and Simmons or the Texas Reading Academy.
5. Reid Lyon. Reid Lyon was a primary architect of Reading First. He was a division chief at the National Institute of Child Health and Human Development (NICHD). Through his involvement with then-Governor George Bush in the Texas Reading Initiative, he developed a personal relationship with President Bush, and was on loan to the White House for a period of time when Reading First was being rolled out. This relationship gave him enormous informal influence on Reading First, and many of the leaders of Reading First were personal friends and co-authors of his (especially Louisa Moats and Sharon Vaughn). Lyon helped plan the national Reading First annual meetings, has consulted with several states on Reading First, and has made speeches all over the country on the program.
Unlike the others discussed in this section, Lyon does not receive royalties from textbook publishers. The only form of conflict of interest we know about relates to his departure from government in 2005 to join Randy Best, the entrepreneur who created Voyager and then sold it for $380 million in 2005. (A Dallas newspaper estimated the value of Voyager in 2000, before Reading First, at $5 million.) Lyon has been identified in press reports as having praised Voyager long before joining Best, and when Reading First funding for New York City was at risk because they insisted on using a reading approach weak in phonics, Lyon was reported to have negotiated a deal in which they could receive their funding if they also adopted Voyager Passport districtwide as a supplementary text.
6. Rod Paige. Rod Paige, Secretary of Education during the establishment of Reading First, also joined Voyager founder Randy Best's company.
It is important to point out that the Department of Education had to be aware of at least some of these conflicts, which clearly violate the Department's own ethics policies for contractors. If, as seems apparent, the Department did know about these conflicts and went forward anyway, it must bear much of the responsibility for the ethical problems that appear throughout administration of Reading First.
The documents we have assembled on the Reading First program provides overwhelming evidence that the leaders and major contractors to Reading First had a clear idea of what they wanted the program to support, and they have used every means at their disposal to manipulate states, districts, and schools into adhering to this plan. The main elements of the plan are as follows:
Core Basal Programs: Macmillan, Scott Foresman, Harcourt Trophies, Houghton-Mifflin, Open Court, Reading Mastery
Louisa Moats (Sopris West)
Texas Reading Academy
Instructional Model: Three-Tier Model
Deviations from the above program elements do occur, but states, districts, and schools generally have had to fight for them and continue to this day to receive pressure to drop them. In contrast, states, districts, and schools that propose to use any of the above elements are not criticized for doing so, and federal funds continue to be used to develop and support extra training and materials designed by the Reading First technical assistance centers to support only these products.
Despite the clear emphasis in the legislation, research has played little or no effective role in establishing or administering the elements of Reading First. Reviewers and monitors do criticize schools for using programs "not consistent with scientifically-based reading research (SBRR)," but the practical definition of consistency with SBRR is whatever the Reading First leaders want it to be. Programs with strong and widely acknowledged evidence of effectiveness have been routinely shunted aside in favor of traditional basal texts that lack such evidence.
The levels of conflict of interest among the Reading First contractors are extraordinary. We do not have documentary evidence showing that particular consultants actively promoted materials from which they derive income, but it is at least a remarkable coincidence that the basal and supplementary textbooks, training programs, and assessments that pay substantial royalties or consulting fees to key RF contractors have received the bulk of Reading First funding, and that the two highest-placed government leaders of Reading First, Reid Lyon and Rod Paige, have joined a company that profited massively from Reading First.
In Reading First, Congress intended to provide substantial resources to see that research-proven reading programs would go to millions of children in low-achieving, high-poverty schools. Instead, they ended up providing a windfall to the same commercial textbook publishers that have always used marketing, not research, to promote their wares.
The cynical manipulation of this well-intentioned program continues to cheat millions of vulnerable children now, and by making a mockery of the concept of evidence-based reform, it cheats millions of children in the future who could have had access to proven programs that would teach them to read.
Adams, G.L., & Engelmann, S. (1996). Research on Direct Instruction: 25 years beyond DISTAR. Seattle, WA: Educational Achievement Systems.
Borman, G.D., Hewes, G.M., Overman, L.T., & Brown, S. (2003) Comprehensive school reform and achievement: A meta-analysis. Review of Educational Research, 73 (2), 125-230.
Borman, G.D., Slavin, R.E., Cheung, A., Chamberlain, A., Madden, N., & Chambers, B. (2005a). Success for All: First year results from the National Randomized Field Trial. Educational Evaluation and Policy Analysis, 27 (1), 1-22.
Borman, G., Slavin, R. E., Cheung, A., Chamberlain, A., Madden, N. A., & Chambers, B. (2005b). The national randomized field trial of Success for All: Second-year outcomes. American Educational Research Journal 42 (4), 673-696.
Borman, G.D., Slavin, R.E., Cheung, A., Chamberlain, A., Madden, N.A. & Chambers, B. (2006). Final reading outcomes of the national randomized field trial of Success for All. Manuscript submitted for publication.
Brownstein, A., & Hicks, T. (2005 a, August 29). IG targets conflicts of interest: Limits on local control. Title I Monitor.
Brownstein, A., & Hicks, T. (2005b, August 29). When research goes to market, is it a good thing for education? Title I Monitor.
Comprehensive School Reform Quality Center (2005). CSRQ Center report on elementary school comprehensive school reform models. Washington, DC: American Institutes for Research.
Foorman, B.R., Francis, D.J., Fletcher, J.M., Schatschneider, C., & Mehta, P. (1998). The role of instruction in learning to read: Preventing reading failure in at-risk children. Journal of Educational Psychology, 90 (1), 37-55.
Herman, R. (1999). An educator's guide to schoolwide reform. Arlington, VA: Educational Research Service.
Manzo, K. (2005 a, June 22). Complaint filed against reading initiative. Education Week.
Manzo, K. (2005 b, September 8). States pressed to refashion Reading First grant designs. Education Week.
Manzo, K. (2005 c, October 12). GAO to probe federal plan for reading. Education Week.
National Reading Panel (2000). Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction. Rockville, MD: National Institute of Child Health and Human Development.
Salvato, N. (January 18, 2006). Effective reading programs share common characteristics: An interview with Reid Lyon. Education News.
Slavin, R. E., Madden, N. A., Cheung, A., Borman, G.D., Chamberlain, A. & Chambers, B. (2006). A three-year randomized evaluation of Success for All: Final reading outcomes. Baltimore, MD: Center for Data-Driven Reform in Education, Johns Hopkins University.
Toppo, G. (2005 a, August 8). Federally funded Reading First called into question. USA Today.
Toppo, G. (2005 b, October 10). Reading program raises questions for lawmakers. USA Today.
Traub, J. (1999). Better by design? A consumer's guide to schoolwide reform. Washington, DC: Thomas Fordham Foundation.
U.S. Department of Education (2002). Guidance on the Comprehensive School Reform Program. Washington, DC: Author.
FAIR USE NOTICE
This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available in our efforts to advance understanding of education issues vital to a democracy. We believe this constitutes a 'fair use' of any such copyrighted material as provided for in section 107 of the US Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit to those who have expressed a prior interest in receiving the included information for research and educational purposes. For more information click here. If you wish to use copyrighted material from this site for purposes of your own that go beyond 'fair use', you must obtain permission from the copyright owner.