The Impacts of Success for All on Reading Achievement

As millions of Californian students enjoy their summer break, an important reality is that in the fall, many students will not return to their previous school with their classmates but instead will attend a brand new one.  Such non-promotional student mobility has negative consequences for mobile students themselves, but it also challenges educators, who must meet the learning needs of these students despite instructional discontinuity. These issues are especially acute in the types of schools where student mobility is most prevalent, especially in urban elementary schools serving predominantly poor and minority populations.  Unfortunately, there is relatively little rigorous evidence about what school practices work for this important student population.

A new paper published in Educational Evaluation and Policy Analysis by Paul Hanselman and Geoffrey Borman presents new relevant evidence.  The authors focus on the Success for All program (SFA), which is a comprehensive school reform model emphasizing early literacy skills for kindergarten through grade 5.  The authors exploit a pragmatic feature of the large-scale experimental evaluation of SFA to estimate the true effectiveness of the instructional program in the later elementary grades.  Because all schools in the national trial wanted to implement the program, half were randomly assigned to implement SFA instruction in grades K–2 and half to implement in grades 3–5.  In previous analyses, the K-2 group has been the focus, with the 3–5 providing the experimental comparison required to demonstrate positive program effects in the early grades.  However, in the new paper, the authors flip the comparison, using the K–2 group as an experimental control to estimate the true effect of the program in the later grades.

In short, the authors find no effects-positive or negative-of the SFA instruction in grades 3–5 compared to business as usual in the comparison schools.  They also rule out the possibility that the 3–5 instruction was particularly beneficial to either initially high- or low-achieving students and demonstrate that the non-effects in the later grades are significantly lower than the positive effects in the lower grades seen in the exactly same experimental trial.  The implication of these results is that even though SFA has demonstrated benefits in the early grades, it is less effective in the later grades, where it may be no better or worse than the likely alternative.

These results imply that greater attention should be given to instructional design in later elementary grades, since successful approaches in the early grades may not translate directly to other contexts.  However, it is important to remember that these results do not reflect the intended Success for All implementation.  Neither schools nor the designers would choose to start students’ exposure to the K–5 instructional sequence in third grade, as the experimental trial mandated.  But as Hanselman and Borman point out, this scenario approximates the experiences of the many mobile students who will start next year in the later elementary grades at a brand new school.  Ultimately, their paper concludes that we still need to know much more about what schools can do to meet the needs of the important mobile student population.

The full study can be found in Paul Hanselman and Geoffrey D. Borman, The Impacts of Success for All on Reading Achievement in Grades 3–5, Does Intervening During the Later Elementary Grades Produce the Same Benefits as Intervening Early? Educational Evaluation and Policy Analysis, June 2013, vol. 35 no. 2, 237–251.