Orwell Award Announcement SusanOhanian.Org Home


NCLB Outrages

Final Audit Report

Ohanian Comment: Be careful what you wish for. The auditor concludes that the Department did not adequately assess issues of bias and lack of objectivity when approving individuals to be technical assistance providers before and after the NCRFTA contract was awarded. This may well be a call for more "scientific reading," not more teacher professional choice.

The Departmentâs Administration of Selected
Aspects of the Reading First Program



ED-OIG/A03G0006
February 2007

Our mission is to promote the efficiency, effectiveness, and integrity of the Departmentâs Programs and Operations.

U.S Department of Education
Office of Inspector General
Philadelphia, PA

NOTICE

Statements that managerial practices need improvements, as well as other conclusions and recommendations in this report, represent the opinions of the Office of Inspector General. Determinations of corrective action to be taken will be made by the appropriate Department of Education officials.
In accordance with Freedom of Information Act (5 U.S.C. § 552), reports issued by the Office of Inspector General are available to members of the press and general public to the extent information contained therein is not subject to exemptions in the Act.

February 22, 2007

Memorandum

TO: Ray Simon
Deputy Secretary
Lead Action Official

Lawrence A. Warder
Chief Financial Officer
Office of the Chief Financial Officer

FROM: John P. Higgins, Jr. /s/

SUBJECT: Final Audit Report
The Departmentâs Administration of Selected Aspects of the Reading First Program
Control Number ED-OIG/A03G0006

Attached is the subject final audit report that covers the results of our review of selected aspects of the Departmentâs administration of the Reading First program during the period August 1, 2001, through September 30, 2004. An electronic copy has been provided to your Audit Liaison Officers. We received your comments, which concurred with the recommendations, but only concurred in part with the findings in our draft report.

Corrective actions proposed (resolution phase) and implemented (closure phase) by your offices will be monitored and tracked through the Departmentâs Audit Accountability and Resolution Tracking System (AARTS). ED policy requires that you enter your final corrective action plan (CAP) for our review in the automated system within 30 days of the issuance of this report.

In accordance with the Inspector General Act of 1978, as amended, the Office of Inspector General is required to report to Congress twice a year on the audits that remain unresolved after six months from the date of issuance.

In accordance with the Freedom of Information Act (5 U.S.C. §552), reports issued by the Office of Inspector General are available to members of the press and general public to the extent information contained therein is not subject to exemptions in the Act.


400 MARYLAND AVE., S.W. WASHINGTON, D.C. 20202-1510
We appreciate the cooperation given us during this review. If you have any questions, please call Bernard Tadley, Regional Inspector General for Audit, at 215-656-6279.

Attachment

cc: Anne Campbell, Chief of Staff, Office of Elementary and Secondary Education

electronic cc: Dolores Warner, Audit Liaison Officer, Office of Elementary and Secondary Education

Gail Cornish, Audit Liaison Officer, Office of the Chief Financial Officer

TABLE OF CONTENTS
Page

EXECUTIVE SUMMARYâ¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦.. 1

BACKGROUNDâ¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦...â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦ 3

AUDIT RESULTSâ¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦...â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦. 6

FINDING NO. 1 â Sessions at The Secretaryâs Reading Leadership
Academies Focused on a Select Number of Reading
Programsâ¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦.. 6

FINDING NO. 2 â The Secretaryâs Reading Leadership Academy Handbook
and Guidebook Appeared to Promote the Dynamic
Indicators of Basic Early Literacy Skills Assessment
Testâ¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦..12

FINDING NO. 3 â The Department Did Not Adequately Assess Issues
of Bias and Lack of Objectivityâ¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦14

OTHER MATTERSâ¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦. 17

OBJECTIVE, SCOPE, AND METHODOLOGYâ¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦â¦. 19

ATTACHMENT 1 â The Secretaryâs Reading Leadership Academy 1, Day 2,
Participant Evaluations

ATTACHMENT 2 â The Secretaryâs Reading Leadership Academy 2, Day 2,
Participant Evaluations

ATTACHMENT 3 â The Secretaryâs Reading Leadership Academy 3, Day 2,
Participant Evaluations

ATTACHMENT 4 â Auditeeâs Comments

EXECUTIVE SUMMARY

As part of the U.S. Department of Education's (Department) efforts to equip states with the information and resources needed to implement the Reading First provision of the No Child Left Behind Act of 2001 (NCLB Act), the Department and the National Institute for Literacy (NIFL) sponsored three major reading academies, the Secretaryâs Reading Leadership Academies (RLAs). The RLAs were held in Washington, D.C., in January and February 2002, and hosted policymakers and key education leaders from every state and territory in the nation. The academies were designed to help state leaders gear up for the implementation of Reading First, the Departmentâs program to improve the quality of reading instruction in kindergarten through third grade. The Department also provided support to states and districts in their Reading First program implementation by funding the National Center for Reading First Technical Assistance (NCRFTA) contract.

The objective of our audit was to determine whether the Department carried out its role in accordance with applicable laws and regulations in administering the RLAs and related meetings and conferences, the NCRFTA contract award process, and its website and guidance for the Reading First program.

Our audit disclosed that the Department generally administered its Reading First website, and its Guidance for the Reading First Program, dated April 2002, in accordance with applicable laws and regulations. With regard to the RLAs, we concluded that the Department did not have controls in place to ensure compliance with the Department of Education Organization Act (DEOA) and NCLB Act curriculum provisions. Specifically, we found that: 1) the âTheory to Practiceâ sessions at the RLAs focused on a select number of reading programs; and 2) the RLA Handbook and Guidebook appeared to promote the Dynamic Indicators of Basic Early Literacy Skills (DIBELS) Assessment Test. With regard to RMC Research Corporationâs (RMC) technical proposal for the NCRFTA contract, we concluded that the Department did not adequately assess issues of bias and lack of objectivity when approving individuals to be technical assistance providers before and after the NCRFTA contract was awarded.

We recommend that the Assistant Secretary for Elementary and Secondary Education â

⢠Establish controls to ensure compliance with, and avoid the appearance of violating the DEOA and the NCLB Act curriculum provisions, especially when organizing conferences where specific programs of instruction are likely to be formally discussed or presented at Department sponsored events;
⢠Establish controls to ensure it does not promote curriculum or create the appearance that it is endorsing or approving curriculum in its conference materials and related publications; and
⢠In coordination with the Chief Financial Officer, establish controls to ensure adequate assessments of bias and lack of objectivity for individuals proposed to perform Department contract work are conducted by the Department and its contractors.
The Department concurred with the recommendations in the draft report and provided proposed corrective actions to address each recommendation. However, it only agreed in part with the findings in the report. The Department stated that the report did not present a balanced summary of the activities reviewed, asserting that the report did not recognize the positive aspects of the activities and challenges faced in planning these activities (the RLAs). We acknowledge that there were positive aspects of the RLAs; however, the purpose of this report is to identify opportunities for improvement. To address some of the Departmentâs concerns, we included the RLA Participant Evaluations for day two of the second RLA in this final report (see Attachment 2), so that all of the participantsâ comments (positive and negative) are included in the report. The Departmentâs specific comments are addressed after each finding, and the full text of its response is provided as Attachment 4 to this report.

We also found that there is interest in placing more emphasis on a reading programâs scientific evidence of effectiveness in determining its eligibility for Reading First funds. We suggest that the Department and Congress, during the next reauthorization of the law, clarify whether reading programs need to have scientific evidence of effectiveness in order to be eligible for funding under Reading First. This issue is discussed in the Other Matters section of the report.

BACKGROUND
In 2000, the Partnership for Reading, now authorized by the Elementary and Secondary Education Act (ESEA), as amended by the NCLB Act (Public Law 107-110), was established. The Partnership was a collaborative effort by three federal agencies: NIFL, the National Institute of Child Health and Human Development (NICHHD), and the Department; to bring the findings of evidence-based reading research to the educational community, families, and others with an interest in helping all people learn to read well. According to NIFL, the Partnership's mission was to disseminate evidence-based research, a focus that made it substantively different from earlier information dissemination efforts and clearinghouses. This mandate to use evidence-based research as the basis for making decisions about reading instruction was continued by the work of the National Reading Panel (NRP), assigned by Congress in 1997 to review the available research. The NRP examined more than 460 studies to extract the essential findings about what has been scientifically proven to work in reading instruction.
The ESEA, as amended by the NCLB Act on January 8, 2002, established the Reading First program. Title 1, Part B, Section 1002(b)(1) of the ESEA authorized an appropriation for Reading First of $900,000,000 for fiscal year 2002 and âsums as may be necessary for each of the five succeeding fiscal years.â The appropriations for fiscal years 2003, 2004, 2005, and 2006 were $993,500,000, $1,023,923,000, $1,041,600,000, and $1,029,234,000, respectively. From January 2002, through September 2006, two Department officials in the Departmentâs Office of Elementary and Secondary Education (OESE) administered Reading First, a Reading First Director and an Education Program Specialist.

In April 2002, the Department issued guidance for the Reading First program and placed this guidance on the Departmentâs website. The website provided information about Reading First for teachers, principals, parents, state and local education officials, and anyone with an interest in improving reading instruction and increasing student achievement. According to the Reading First guidance, the Reading First program focused on putting proven methods of early reading instruction in classrooms. Through Reading First, states and districts received support to apply scientifically based reading research (SBRR)âand the proven instructional and assessment tools consistent with this researchâto ensure that all children learned to read well by the end of third grade. The Department provided formula grants to states that submitted an approved application. State Educational Agencies (SEAs) then awarded subgrants to eligible Local Educational Agencies (LEAs) on a competitive basis. SEAs funded subgrants that showed the most promise for raising student achievement and for successful implementation of reading instruction, particularly at the classroom level. Only programs that were based on SBRR were eligible for funding through Reading First.

Title 1, Part B, Section 1208(6) of the ESEA defines SBRR as research that:
A. Applies rigorous, systematic, and objective procedures to obtain valid knowledge relevant to reading development, reading instruction, and reading difficulties; and
B. Includes research that-
i. Employs systematic, empirical methods that draw on observation or experiment;
ii. Involves rigorous data analyses that are adequate to test the stated hypotheses and justify the general conclusions drawn;
iii. Relies on measurements or observational methods that provide valid data across evaluators and observers and across multiple measurements and observations; and
iv. Has been accepted by a peer-reviewed journal or approved by a panel of independent experts through a comparably rigorous, objective, and scientific review.
NIFL, an interagency group composed of the Secretaries of Education, Labor, and Health and Human Services (HHS), was authorized under the NCLB Act to help children, youth, and adults learn to read by supporting and disseminating SBRR. In 1998-1999, NIFL was appropriated $5 million in funds under the Reading Excellence Act (REA) program to conduct a National Reading Research Dissemination Campaign (NRRDC). In September 2000, NIFL contracted with RMC to perform tasks for the NRRDC.
In August 2001, five months prior to the Reading First program legislation being signed into law, the Department started gearing up its Reading First program and wanted to disseminate the findings about SBRR to the states. The Department did not have the funding to hold conferences on SBRR so it requested NIFLâs help.
In September 2001, the RLAs were added as a task under NIFLâs NRRDC contract with RMC. The Department and NIFL sponsored three reading academies, the Secretaryâs RLAs, which were held in Washington, D.C., on January 23-25, February 13-15, and February 20-22, 2002. In planning the RLAs, the Assistant Secretary (former) for OESE and other Department officials worked with NIFL, RMC, and numerous individuals outside of the Department. Several of these individuals were associated with the Direct Instruction program. The RLAs featured presentations on topics, such as effective reading instruction, the selection of reading programs, accountability and assessment in reading, professional development for reading teachers and others, and instructional leadership. Department officials and researchers from research institutions made presentations and led discussions with participants on teaching reading based on scientific research.
The goals of the RLAs were to help states:
⢠Develop a knowledge base about scientifically based reading instruction;
⢠Build capacity to design and sustain professional development for teachers in scientifically based reading instruction;
⢠Acquire the knowledge and tools to implement the Reading First initiative; and
⢠Partner with school districts and the Department to improve reading achievement.
The RLAs also included a session, titled âTheory to Practice: A Panel of Practitioners.â The speakers discussed how implementing a scientifically based reading program had brought about great improvements in the reading skills of their kindergarten through third grade students. In addition, there was a luncheon speaker on the second day of the first and third RLA who discussed the improved reading scores of his schoolâs students since implementing its reading program.
At each RLA, attendees were provided an RLA Handbook. The book included copies of the presentations from each RLA session. There were three different versions of the RLA Handbook, as it was specific to the particular RLA attended. At the end of each day of each RLA, attendees were provided with an evaluation form to critique that dayâs training. The evaluation forms were provided to the Department.
The Department provided support to states and districts in their Reading First program implementation by funding the NCRFTA contract. The NCLB Act authorized the NCRFTA to provide comprehensive technical assistance to states and districts over five years. The total cost to the Department for full performance of the contract was estimated to be $6,891,035. RMC was the only firm that responded to the solicitation. RMC was awarded the contract on September 30, 2003, and it currently serves as the national coordinator for Reading First technical assistance. The purpose of the contract was to establish three regional centers located in the western, central, and eastern regions of the United States. RMC subcontracted with the Institute for the Development of Educational Achievement (University of Oregon, Eugene), the Texas Center (Vaughn Gross Center) for Reading and Language Arts (University of Texas, Austin), and the Florida Center for Reading Research (Florida State University, Tallahassee) to provide technical assistance to states and districts in the western, central and eastern regions of the United States, respectively.
The Department is generally prohibited from exercising control over any schoolâs curriculum or program of instuction. Section 3403(b) of the DEOA provides that Department officials shall not construe any provision of a Department program as authorizing the Department to exercise any direction, supervision, or control over the curriculum or program of instruction of any school, or school system. Section 9527(a) of the ESEA, as amended by the NCLB Act, prohibits Department officials from construing any provision of the NCLB Act as authorizing such officials to mandate, direct, or control an SEA, LEA, or schoolâs curriculum, or program of instruction. Further, Section 9527(b) prohibits the Department from using any NCLB Act program funds to endorse, approve, or sanction any curriculum.
A number of complaints have been made that the Department exercised undue influence in its administration of the Reading First program, by promoting or endorsing specific reading programs, materials, assessment instruments, and models of instruction. In addition to this audit, the OIG conducted an inspection of the Departmentâs administration of the Reading First grant application process, a series of audits of selected statesâ (Wisconsin, New York, and Georgia) implementation of the Reading First program, as well as an audit of RMCâs administration of the Reading First program contracts.

AUDIT RESULTS

Our audit disclosed that the Department generally administered its Reading First website and its Guidance for the Reading First Program, dated April 2002, in accordance with applicable laws and regulations. With regard to the RLAs, we concluded that the Department did not have controls in place to ensure compliance with the DEOA and NCLB Act curriculum provisions. Specifically, we found that: 1) the âTheory to Practiceâ sessions at the RLAs focused on a select number of reading programs; and 2) the RLA Handbook and Guidebook appeared to promote the DIBELS Assessment Test. With regard to RMCâs technical proposal for the NCRFTA contract, we concluded that the Department did not adequately assess issues of bias and lack of objectivity when approving individuals to be technical assistance providers before and after the NCRFTA contract was awarded.

The Department concurred with the recommendations in the draft report and provided proposed corrective actions to address each recommendation. However, it only agreed in part with the findings in the report. The Departmentâs specific comments are addressed after each finding, and the full text of its response is provided as Attachment 4 to this report.

Finding No. 1 â Sessions at The Secretaryâs Reading Leadership Academies Focused on a Select Number of Reading Programs

We concluded that the Department did not have controls in place to ensure compliance with the DEOA and NCLB Act curriculum provisions. We found that: 1) only a select number of reading programs were discussed during the âTheory to Practice: A Panel of Practitionersâ sessions; 2) at the first and third RLAs, the luncheon speakerâs presentation featured one of the few reading programs discussed during the âTheory to Practiceâ sessions; and 3) participants at the first and third RLAs expressed concerns that certain programs were being endorsed and promoted by the Department.

In January and February 2002, the Department and NIFL sponsored three Secretaryâs RLAs in Washington, D.C. According to NIFL, the RLAs were designed to help state leaders gear up for swift and successful implementation of Reading First. The RLAs included a session titled âTheory to Practice: A Panel of Practitioners.â

The Director of NIFL informed us that the intent of the RLAs was to disseminate information on the SBRR findings to the states. The Director also stated that the RLAs started as a joint effort between the Department and NIFL. However, the Director stated that after the Assistant Secretary (former) for OESE proposed that the Secretary of Education (former) sponsor the RLAs, the Department became much more involved in the planning of the RLAs and controlled the meetings. The Department also exercised control over the content and presenters. In planning the content of the RLAs, the Department worked with numerous individuals outside of the Department. Several of these individuals were associated with the Direct Instruction program. The RLAs became known as the Secretaryâs RLAs.

Composition of the âTheory to Practiceâ Panels
The Department included only a select number of reading programs for discussion by the âTheory to Practiceâ panel members. The following chart shows the limited number of programs presented by âTheory to Practiceâ panels at the RLAs:

Reading Leadership Academy (RLA)
Panel Member - Position at the time of the RLA Reading Programs
Discussed
RLA 1, Jan. 23-25, 2002 Principal, City Springs Elementary, Baltimore, MD Direct Instruction
Reading Facilitator, Parham School, Cincinnati, OH Direct Instruction
Principal, Parham School, Cincinnati, OH Direct Instruction
Teacher, Tovashal Elementary School, Murrieta, CA Open Court

RLA 2, Feb. 13-15, 2002 Principal, City Springs Elementary, Baltimore, MD Direct Instruction
Asst. Administrator, Washington Reads, WA Office of the Superintendent of Public Instruction, Olympia, WA Open Court, Read Well
Principal, Weaver Elementary School, Weaver, AL Houghton-Mifflin,8 Soar to Success

RLA 3, Feb. 20-22, 2002 Principal, City Springs Elementary, Baltimore, MD Direct Instruction
Principal, Parham School, Cincinnati, OH Direct Instruction
Asst. Superintendent, L.A. Unified School District, CA Open Court

During the âTheory to Practiceâ sessions, each of the panel members discussed the reading success stories of the students in his or her school or district. A question and answer period with the audience followed. At the first RLA, held on January 23-25, 2002, three of the four panel members talked about the Direct Instruction program. The fourth panel member discussed the Open Court program, held up the Open Court Reader for the audience to view, and read a brief passage from it.

After the âTheory to Practiceâ session, there were many comments by attendees indicating they felt the RLA was biased toward Direct Instruction and Open Court. Consequently, the next day, the Director of the Reading First Office (former) explained to the audience at the first RLA that the intention of the panelists was not to âplugâ certain programs. The Department invited some of the same panel members back to participate on the âTheory to Practiceâ panels at the second and third RLAs. However, the Department did not instruct any of the panel members to refrain from naming the programs that they were using.

At the second RLA, held on February 13-15, 2002, the âTheory to Practiceâ panel included three panelists. One of the three panelists was a participant from the first RLA. Once again, this panelist discussed the Direct Instruction program.

At the third RLA, held on February 20-22, 2002, the âTheory to Practiceâ panel was composed of three panelists. Two of the three panelists had participated in previous panels and once again they discussed the Direct Instruction program. The third panel member discussed the Open Court program and read a series of letters from a teacher, which described how the teacher was at first skeptical of Open Court, but took the training, taught it in class, and became a big supporter of the program.

An email, to the Director of the Reading First Office (former), from a colleague outside of the Department, dated January 15, 2002, with a subject line of âFwd: School Using DI in Cinn.â suggested the Department may have intentionally wanted to showcase the Direct Instruction program by selecting âTheory to Practiceâ panelists who used Direct Instruction successfully in their district or at their schools. The email stated â

When [an RLA consultant] had me looking for possible administrators to present at RF [Reading First] academies [the former principal of Wesley Elementary School in Houston, TX, which has one of the longest continuous Direct Instruction implementations in the country] recommended the principal of this school [Parham School, Cincinnati, OH].

The principal of Parham School was a panelist for two of the three âTheory to Practiceâ sessions, and also discussed the Direct Instruction program.

Selection of the Luncheon Speaker
In addition to the âTheory to Practiceâ session, there was a luncheon speaker on the second day of each academy. The luncheon speaker at the first and third RLAs was the principal of the Seed Academy and Harvest Preparatory School. The principalâs presentation focused on one reading program, Direct Instruction. During both of his presentations, the principal discussed how he was looking for a model of successful instruction to implement at his school when he learned of Direct Instruction. The principal then used slides to show the improved reading scores of his schoolâs students since implementing Direct Instruction.

An email, dated January 14, 2002, from the Director of the Reading First Office (former) inviting the principal of the Seed Academy and Harvest Preparatory School to speak at the RLAs indicated that the Department may have favored Direct Instruction. The email stated â

We have chatted several times at ADI [Association for Direct Instruction] Eugene conferences, and recently (I think) about the DI [Direct Instruction]-based middle school my wife and I started in rural Africa several years ago. I am still the President of the Board of that school . . . . Iâve also been working with a bunch of public schools in my home of Baltimore, trying to do what youâve done at Seed.

. . . we would hugely appreciate it if you could make time in your schedule to speak a bit at our never-been-done before Reading Leadership Academies.

The Government Accountability Officeâs (GAO) Standards for Internal Control in the Federal Government requires that agencies develop detailed policies and procedures, and practices to fit their agencyâs operations and to ensure that they are built into and are an integral part of operations. Policies and procedures are a part of the control activities that enforce managementâs directives and are an integral part of an entityâs planning, implementing, reviewing, and accountability for stewardship of government resources and achieving effective results. Control activities help to ensure that actions are taken to address risk. As a part of these control activities, policies and procedures should have been developed and implemented to minimize the risks the Department may face when it engages external speakers at conferences it sponsors. One such risk was the risk of the perception that the Department favored a particular program, tool, or entity. Another risk was the risk of violating the DEOA, which established the Department and describes the Federal-State relationship in education as follows â

Section 3403 (b) No provision of a program administered by the Secretary or by any other officer of the Department shall be construed to authorize the Secretary or any such officer to exercise any direction, supervision, or control over the curriculum, program of instruction, administration, or personnel of any educational institution, school, or school system . . . over the selection or content of . . . textbooks, or other instructional materials by any educational institution or school system, except to the extent authorized by law.

Section 9527(b) of the NCLB Act (20 U.S.C. Section 7907) reinforces the language in the DEOA. Specifically, it prohibits funds provided to the Department from being used by the Department to endorse, approve, or sanction any curriculum designed to be used in an elementary or secondary school.

RLA Participant Comments
As a result of the Department not having controls to ensure compliance with the DEOA, and the NCLB Act prohibitions against endorsing or promoting programs of instruction, some attendees at the RLAs felt that the Department was endorsing the Direct Instruction and Open Court reading programs. The comments expressed on the evaluation forms from the first and third RLAs included â

⢠âThe . . . Theory to Practice Panel â was very poor. It sounded like a sales job for a program as opposed to a description of enabling teachers to teach reading.â
⢠âI felt like it was simply a push for a national curriculum. I think Iâll go buy shares in Open Court!â
⢠âPanel was a sales job for Direct Instruction and Open Court.â
⢠âPlease do not promote a program (Open Court) (Direct Instruction). This is not the Department of Educationâs place to do.
⢠âI felt like I was in a Direct Instruction sales pitch all day. Thanks for including at least one other program.â
⢠âI felt it was wrong to showcase one specific program (D.I.) excessively . . ..â
⢠âTodayâs sessions may have given an excessive government endorsement to Direct Instruction.â
In addition, there was a perception by some states that the programs discussed at the RLAs were part of a Department âapproved listâ of Reading First programs. As a result, in April 2002, the Reading First Office was compelled to put a notice on its website clarifying that the Department did not have an approved list of reading programs for use with Reading First funds.

Nonetheless, the view that there was a Department âapproved listâ of Reading First reading programs persisted, and in a letter to Reading First State Directors, dated October 11, 2005, over two years after Reading First applications were approved, the Director of the Reading First Office (former) wrote â

The U.S. Department of Education has consistently and openly stated that there is no Federal âapproved listâ of reading programs for use under the Reading First program. The Department has not developed or endorsed any list of instructional programs as appropriate for use, nor has the Department indicated a preference for certain kinds of instructional programs, such as basal textbook-based programs.

The sessions at the RLAs focused on a select number of reading programs, which gave attendees the impression that the Department was promoting these specific reading programs. Since the Department wanted senior level state leaders, who would be Reading First program policymakers, to attend the RLAs, it should have taken all action possible to minimize the perception that it could be viewed as endorsing any reading program. The Department had control of the agenda and presenters for the RLAs and should have ensured that nothing in the RLAs could be viewed as endorsements or approval of any particular program. As one of the sponsors, the Department had a responsibility to avoid all appearances of violating the letter and spirit of the DEOA or Section 9527(b) of the NCLB Act. The appearance that the Department is endorsing or approving curriculum in its conference sessions can damage the integrity of the event and the reputation of the Department.

The evidence above calls into question whether the Department violated the DEOA and the NCLB Act curriculum provisions.

Recommendation
We recommend that the Assistant Secretary for Elementary and Secondary Education â

1.1 Establish controls to ensure compliance with, and avoid the appearance of violating the DEOA and NCLB Act curriculum provisions, especially when organizing conferences where specific programs of instruction are likely to be formally discussed or presented at Department sponsored events.

Department Comments

The Department concurred with the recommendation and provided proposed corrective action to address it. However, the Department agreed only in part with the finding, stating that the finding failed to provide a balanced summary of the activities reviewed.

The Department explained â

While we agree that the audience should have been adequately advised that the Department was not promoting or endorsing a specific program, we do not think that there is any problem in having panelists highlight certain successful programs or identify the specific programs they are implementing. . . . To the contrary, we have received positive feedback that this is the kind of helpful information that applicants and grantees need to make their programs more effective. Furthermore, when panelists failed to specifically say what program they were implementing, participants during the question and answer session asked for the program to be named . . . . Thus, it is reasonable for the panelists to mention the names of the programs they were using, so that the Department can carry out its important statutory responsibilities to disseminate information on effective and promising practices.

The Department further stated â

. . . [it] cannot be expected to present information on all possible programs. It is reasonable that the Department should be able to present information on certain programs, as long as it is made clear that the programs presented are merely examples of the types of programs that might be supported with Reading First funds and that the presentation was not intended to be an endorsement or promotion of a specific program.

The Department also stated it was concerned that steps taken by Department officials to clarify the false impression that was left with some of the conference attendees were represented in the report as further evidence of its initial failure to indicate that it was not endorsing or promoting particular programs. These actions should have been recognized as positive steps that Department personnel expeditiously took to clarify this false impression. These actions included:

⢠Informing the audience at the first RLA that the purpose of the âTheory to Practiceâ sessions was not to promote or endorse any particular program; and
⢠Posting a clarifying note on the Reading First website and sending a letter to Reading First State Directors indicating that there was not an âapproved listâ of reading programs that could be funded under Reading First.

OIG Response

We acknowledge that there were positive comments on the RLA evaluation forms regarding the âTheory to Practiceâ sessions, (see Attachments 1, 2, and 3). However, the negative comments should have alerted the Department that some attendees felt the Department was endorsing or promoting the Direct Instruction and Open Court reading programs. Prior to conducting the second and third RLAs, the Department should have done more to mitigate this perception. The Department was advised about this perception in the following emails:

⢠In an email attachment to the Director of the Reading First Office (former), from the facilitator of the first RLA, dated January 27, 2002, titled âNotes from Debriefing of the First RLA,â the notes include the following point: âToo much emphasis on Direct Instruction.â

⢠Another email, dated February 4, 2002, to the Director of the Reading First Office (former), from the President of RMC, included feedback from a consultant who was tasked with analyzing the RLA participant evaluations from the first RLA. It stated, âAnd, as everyone knows, Open Court and Direct Instruction canât be the only shows in town.â

Although, additional reading programs were discussed during the second RLAâs âTheory to Practiceâ session, as stated in the finding, for the third RLA the Department again only selected panelists that used the Direct Instruction and Open Court reading programs. This again gave the impression that the Department was endorsing or promoting these programs, and that they were approved Reading First programs. Our concern is that only a few specific reading programs were discussed when there were other reading programs available.

We do not believe the actions taken by the Department to clarify that it was not endorsing or promoting any particular programs have been misrepresented in the audit finding. As stated in the finding, the perception of an âapproved listâ still persisted well after the RLAs. Although OESE put a note on its website in 2002 to address the perception, it was still compelled to take further action to address the perception in 2005, over three years after the RLAs were held. This is evidence that in planning for the first RLA, the Department did not address the risk of appearing to promote or endorse a particular program. Further, once it became evident that there was a perception by some of the attendees of the first and third RLAs that the Department was endorsing or promoting reading programs, not enough was done to mitigate this perception.

Finding No. 2 - The Secretaryâs Reading Leadership Academy Handbook and Guidebook, Appeared to Promote the Dynamic Indicators of Basic Early Literacy Skills Assessment Test

The Reading First statute required the use of screening, diagnostic, and classroom-based reading assessments so that teachers could effectively screen, identify, and overcome reading barriers facing their students. We found that the Department appeared to promote DIBELS by including an article featuring it in the RLA Handbook and Guidebook.

At each RLA, attendees were given a RLA Handbook that contained the PowerPoint slides for each speakerâs presentations. The RLA Handbook also included an article titled, âThe Importance and Decision-Making Utility of a Continuum of Fluency-Based Indicators of Foundational Reading Skills for Third-Grade High-Stakes Outcomes.â This 29-page article featured DIBELS, one of many screening tools on the market that could have been used for performing Reading First assessments. The Department also included this article in the RLA Guidebook. The RLA Guidebook, published in September 2002, was a collection of the presentations and resources that formed the basis of the RLAs. While numerous assessment instruments were listed in the RLA Handbook and Guidebook, only DIBELS was featured in an article in both books.

The RLA Handbook was provided at the RLAs, which were held to assist states in gearing up for their implementation of the Reading First program. The RLA Guidebook was provided shortly thereafter, when states were preparing their Reading First applications. As a result, states may have been given the impression that DIBELS was the assessment test the Department preferred for use in the Reading First program. In fact, 43 states indicated in their Reading First applications that they intended to use DIBELS as one of their assessment test instruments.

As cited in Finding 1, federal laws and regulations applicable to this finding are in the DEOA, and the NCLB Act, which contain provisions prohibiting the Department from exercising direction or control over curriculum, or from endorsing any curriculum or program of instruction. Also as cited in Finding 1, GAOâs Standards for Internal Control in the Federal Government requires that agencies develop detailed policies and procedures, and practices to fit their agencyâs operations and to ensure that they are built into and are an integral part of operations. Policies and procedures are a part of the control activities that enforce managementâs directives and are an integral part of an entityâs planning, implementing, reviewing, and accountability for stewardship of government resources and achieving effective results. Control activities help to ensure that actions are taken to address risk. As a part of these control activities, policies and procedures should have been developed and implemented to minimize the risks the Department may face when it provides guidance to state-level administrators and policy makers. One such risk was the risk of the perception that the Department favored a particular program, tool, or entity.

We concluded that the Department did not have controls in place to ensure it was not promoting curriculum or creating the appearance that it was endorsing or approving curriculum in its conference materials and related publications.

The appearance that the Department is endorsing or approving curriculum in its conference materials and related publications can damage the integrity of the event and the reputation of the Department.

Recommendation
We recommend that the Assistant Secretary for Elementary and Secondary Education â

2.1 Establish controls to ensure it does not promote curriculum or create the appearance that it is endorsing or approving curriculum in its conference materials and related publications.

Department Comments

The Department concurred with the recommendation and provided proposed corrective action to address it. The Department did not provide any specific comments in its response indicating agreement or disagreement with this finding.

Finding No. 3 â The Department Did Not Adequately Assess Issues of Bias and Lack of Objectivity

We found that the Department did not adequately assess issues of bias and lack of objectivity when approving individuals to be technical assistance providers before and after the NCRFTA contract was awarded. Specifically, the Department did not: 1) adequately vet proposed technical assistance providers resumes, and 2) follow up on reading related contracts held by technical assistance providers in order to determine whether views and positions taken could be largely motivated from the close identification or association of an individual with a particular point of view or the positions or perspectives of a particular group.

When RMC submitted its proposal for the NCRFTA contract, it included the names and resumes of key personnel, including Regional Directors. Although issues of bias and lack of objectivity of some proposed key personnel existed, the Department did not address this issue. For example, the current and former directors of the Western Regional Technical Assistance Center had associations with Prentice Hall, which included Prentice Hall publishing a reading text authored by these individuals in 1997.

In addition, as a part of the NCRFTA contract negotiations, the Department requested RMC provide a list of all reading related contracts held by the staff of each technical assistance center. The following contracts were reported to RMC: the Central Regional Technical Assistance Centerâs former director had been a paid consultant to the reading program Voyager prior to her work with the Center; and the Eastern Regional Technical Assistance Centerâs current director has a current publishing contract for reading intervention materials with the publisher SRA/McGraw-Hill. The Department did not follow up on these issues.

The Departmentâs main reason for reviewing technical assistance providersâ resumes was to determine whether the proposed individuals had sufficient SBRR expertise andor were being paid with Reading First funds for performing other work related to Reading First. The Department did not consider whether the proposed individuals had associations with reading program publishers and or reading programs. In addition, the Department did not require RMC to vet consultant resumes for issues of bias and lack of objectivity.

We concluded that the Department did not have controls to ensure potential sources of bias and lack of objectivity were adequately assessed. In addition, the Department did not consider associations with reading program publishers as a potential source of bias because officials thought that it would limit the pool of technical assistance providers with expertise in SBRR. Consequently, appearances of bias and lack of objectivity contributed to the complaints surrounding the administration of the Reading First program and led to the perception that some individuals may have been promoting the reading products they were associated with and may have influenced the products that were being selected by SEAs and LEAs.

To respond to the ongoing perception of the appearances of bias and lack of objectivity, in a letter sent to RMC, dated October 11, 2005, two years after the NCRFTA contract was awarded, the Reading First Director (former) wrote â

In providing this assistance, [technical assistance], it is essential that the National Center avoid all conflicts of interest among employees, subcontractors, and consultants who may have connections to particular instructional programs or materials used under Reading First. Although I understand that the National Center does not recommend programs or materials, it is important that steps are taken to avoid the appearance of conflicts of interest in this area. Individuals associated with the National Center or the three regional centers must recuse themselves from participating in technical assistance activities that appear to benefit commercial entities with whom they are personally connected.

As cited in Finding 1, federal laws and regulations applicable to this finding are in the DEOA, and the NCLB Act, which contain provisions prohibiting the Department from exercising direction or control over curriculum, or from endorsing any curriculum or program of instruction. Also as cited in Finding 1, GAOâs Standards for Internal Control in the Federal Government requires that agencies develop detailed policies and procedures, and practices to fit their agencyâs operations and to ensure that they are built into and are an integral part of operations. Policies and procedures are a part of the control activities that enforce managementâs directives and are an integral part of an entityâs planning, implementing, reviewing, and accountability for stewardship of government resources and achieving effective results. Control activities help to ensure that actions are taken to address risk.

Since the DEOA and the NCLB Act prohibit the Department from endorsing or promoting curriculum, it should have had a process in place to assess potential sources of bias and lack of objectivity. Without an adequate assessment of bias and lack of objectivity for individuals proposed to perform Department contract work, the Department could be placed in a situation where the public could reasonably question and perhaps discount or dismiss the work performed simply because of the existence of a potential bias. Further, the Department should ensure its contractors also perform this assessment.

Recommendation
We recommend that the Assistant Secretary for Elementary and Secondary Education, in coordination with the Chief Financial Officer â

3.1 Establish controls to ensure adequate assessments of bias and lack of objectivity for individuals proposed to perform Department contract work are performed by the Department and its contractors.

Department Comments

The Department concurred with the recommendation and provided proposed corrective actions to address it. The Departmentâs response stated â

We acknowledge that the Department should have in place procedures to assess issues of bias and lack of objectivity on the part of potential technical assistance providers. However, the mere fact that certain individuals may have expertise with respect to particular programs should not preclude them from serving as technical assistance providers in any capacity. On the other hand, we acknowledge the Department should take steps to ensure that such individuals do not provide advice in areas in which they may have a financial conflict of interest.

OIG Response

The finding does not suggest that an individual should be excluded from serving as a technical assistance provider due to the mere fact he or she may have expertise with regard to particular programs. The main point of the finding, agreed to by the Department, is that the Department should have had policies and procedures in place and should have taken steps to assess issues of bias and lack of objectivity.

OTHER MATTERS

In order for a reading program to be used in a stateâs Reading First program, it must include explicit and systematic instruction in the five essential components of reading (phonemic awareness, phonics, vocabulary development, reading fluency (including oral reading skills), and reading comprehension strategies), as identified by the statute. According to the Assistant Secretary (former) for OESE and the Chief (former) of the Child Development and Behavior Branch (CDBB) within the NICHHD at the National Institute of Health (NIH), a limited number of reading programs themselves had been rigorously tested and proven to be effective when the Reading First program began in 2002. However, since the initiation of the Reading First legislation, there seems to be some movement to place more emphasis on the scientific evidence of effectiveness of reading programs rather than just the inclusion of the five essential components of reading. For example, a letter from Senator Richard G. Lugar to Secretary Spellings, dated September 8, 2005, stated â

In my view, the Department must do a better job of providing clear information to states and districts regarding the definition of research based programs. Programs that have been rigorously evaluated should be emphasized under Reading First. I would encourage the Department to send a mailing to its technical assistance centers and States directing them to fully consider the scientific evidence of effectiveness for programs, not just program components, in considering programs for use under Reading First.

In addition, while a large portion of grants have already been awarded, perhaps state Reading First directors should be asked to revise criteria for awarding funding to better reflect the above definition of research-validated programs and to include competitive preference points to applicants proposing to implement research-validated programs.

In addition, the language included in the 2006 Labor-HHS-Education appropriations bill (Public Law 109-103), dated July 2005, appears to support this point of view. The 2006 Labor-HHS-Education appropriations bill stated â

The Committee intends for funds available under the Reading First program to encourage and support the use of reading programs with the strongest possible scientific evidence of effectiveness. The Committee urges the Department to provide clear guidance to its technical assistance centers and the States to: fully consider scientific evidence of effectiveness in rating programs for use under Reading First; contemplate expanded lists of allowable programs that include innovative programs with scientific evidence of effectiveness; when awarding new grants, consider giving preference to those schools that select programs with strong, scientific evidence of effectivenessâ¦and allow comprehensive reading programs that have scientific evidence of effectiveness to be implemented in full, as they have been researched, without modification to conform to other models of instruction...The Committee notes that Reading First materials decisions are to be made at the school level, subject to the approval of the State.

This point of view is consistent with an August 2006 statement made to us, by the Chief (former) of the CDBB within the NICHHD at the NIH. The Chief (former) stated that publishers certainly have had the time to test their programs to determine causal effectiveness. He further stated that by now, there should be a larger group of programs that have been tested. With the impending reauthorization of the NCLB Act, he believes there should be a push towards the ânext level.â Changes should be made to tighten up the criteria under which programs are eligible for funding. He also recommended, in an August 16, 2006, email to us, that â

Federal funds should only be used for those programs, combination of programs, instructional strategies, reading methods, hybrid instructional models (Mathes et al., 2005), that have been found to be effective using experimental research designs that can determine the causal impact of the programs, strategies, methods, and/or models on student learning and achievement in reading. Effectiveness should be established using the most rigorous experimental designs that provide the strongest evidence of causal validity with priority placed on randomized clinical trials (RCT) and regression discontinuity studies. Quasi-experimental research designs could be utilized when randomization is not possible but with the recognition that causal validity must be interpreted with caution. Studies of effectiveness should also address the issue of intervention fidelity, the reliability and validity of the measures employed, the extent to which relevant people, settings, and measurement timings are included in the study, and the extent to which the effects of the program, strategies, methods, and/or models can be tested with subgroups within the population under study.
Since the legislation is scheduled for reauthorization in 2007, Congress has an opportunity to clarify whether reading programs should be funded on the basis of program effectiveness. Congress will also be able to determine what it means for a program to be âbased on scientific reading researchâ and whether this definition is consistent with program effectiveness. Information obtained and deliberated upon, as part of the reauthorization process, should enable Congress to make the legislation more responsive to the needs of children by ensuring that quality programs are funded with Reading First funds.
We suggest that the Department and Congress, during the next reauthorization of the law, clarify whether reading programs need to have scientific evidence of effectiveness in order to be eligible for funding under Reading First.


— Inspector General
U. S. Department of Education
2007-02-22
http://www.ed.gov/about/offices/list/oig/auditreports/a03g0006.doc


INDEX OF NCLB OUTRAGES


FAIR USE NOTICE
This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available in our efforts to advance understanding of education issues vital to a democracy. We believe this constitutes a 'fair use' of any such copyrighted material as provided for in section 107 of the US Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit to those who have expressed a prior interest in receiving the included information for research and educational purposes. For more information click here. If you wish to use copyrighted material from this site for purposes of your own that go beyond 'fair use', you must obtain permission from the copyright owner.