The Enhanced Reading Opportunities Study, Second Report
Findings from the Second Year of Implementation
This report presents findings from the Enhanced Reading Opportunities (ERO) study — a demonstration and rigorous evaluation of two supplemental literacy programs that aim to improve the reading comprehension skills and school performance of struggling ninth-grade readers. The U.S. Department of Education’s (ED) Office of Elementary and Secondary Education (OESE) is funding the implementation of these programs, and its Institute of Education Sciences (IES) is responsible for oversight of the evaluation. American Institutes for Research (AIR), MDRC and Survey Research Management (SRM) are conducting the evaluation in partnership.
The present report — the second of three — focuses on the second of two cohorts of ninth-grade students to participate in the study and discusses the impact that the two interventions had on these students’ reading comprehension skills through the end of their ninth-grade year. The report also describes the implementation of the programs during the second year of
the study and provides an assessment of the overall fidelity with which the participating schools adhered to the program design as specified by the developers. While this report focuses primarily on implementation and impacts in the second year of the study, comparisons between the first and second year of the study are also provided. The key findings discussed in the report include the following:
- On average, across the 34 participating high schools, the supplemental literacy programs improved student reading comprehension test scores by 0.08 standard deviation. This represents a statistically significant improvement in students’ reading comprehension (p-value = 0.042).
- Seventy-seven percent of the students who enrolled in the ERO classes in the second year of the study were still reading at two or more years below grade level at the end of ninth grade, relative to the expected reading
achievement of a nationally representative sample of ninth-grade students. One of the two interventions — Reading Apprenticeship Academic Literacy (RAAL) — had a positive and statistically significant impact on reading comprehension test scores (0.14 standard deviation; p-value = 0.015). Although not statistically significant, a positive impact on reading comprehension (0.02 standard deviation) was also produced by the other intervention, Xtreme Reading. The difference in impacts between the two programs is not statistically significant, and thus it cannot be concluded that RAAL had a different effect on reading comprehension than Xtreme Reading.
- The overall impact of the ERO programs on reading comprehension test scores in the second year of implementation (0.08 standard deviation) is not statistically different from their impact in the first year of implementation (0.09 standard deviation), nor is each intervention’s impact in the second year of implementation statistically different from its impact in the first year.
- The implementation fidelity of the ERO programs was more highly rated in the second year of the study than in the first year. In comparison with the first year, a greater number of schools in the second year of the study were deemed to have programs that were well aligned with the program developers’ specifications for implementation fidelity (26
schools in the second year, compared with 16 schools in the first year), and fewer schools were considered to be poorly aligned (one school in the second year, compared with 10 schools in the first year).