The Unintended Consequences of a Middle School Literacy Support Intervention

Increased learning time is a strategy that has proliferated in the era of No Child Left Behind (NCLB), as a way to bolster the literacy and numeracy outcomes of K–12 students, particularly for students whose initial performance was below average. Research evidence has documented that double dose strategies have been effective at improving math outcomes, especially for lower-performing students, but the evidence is less numerous and less conclusive for literacy. Furthermore, there is little evidence on whether and how such strategies can improve outcomes for average or above-average students as a way to improve their college readiness.

This study investigated the impact of supplemental literacy instruction on subsequent cognitive outcomes for middle school students near the national median for literacy achievement. The underlying theory of action held that bolstering the performance of average readers in middle school would improve their performance in high school and, in turn, college readiness. The district is situated in a suburb of a major metropolitan area with roughly 40% identifying as white, and another 40% identifying as Black and roughly one-third qualifying for free- or reduced-price meals. Students in district middle schools were assigned to an additional literacy course in middle school, rather than taking an exploratory world language course, if they fell below the 60^th national percentile on the literacy component of Iowa Test of Basic Skills (ITBS) in 5^th grade. Using random variation in a student’s position relative to the chosen cutoff, I was able to identify the causal effect of participating in the supplemental literacy course rather than electing a world language course. While initial estimates suggested no impact of the program, analysis to test for differential effects by race indicated strong negative impacts for Black students, and noisy positive impacts for white, Latino, and Asian students. These negative effects for Black students were most pronounced in predominantly white schools and were evident on both state tests used for accountability as well as low-stakes nationally normed measures (8^th grade ITBS).

These findings suggest that additional literacy instruction in middle school can have markedly different effects on students, and that program differentiation or augmentation may be necessary to prevent harm for middle-school students who were initially of average literacy ability in fifth grade. The counter-intuitive effects of the policy suggest the importance of carefully considering how broad policies can have differing impacts when they are deployed at the building level. This is particularly true when policies are developed at the state or district level, and especially where there is considerably heterogeneity in the student population, by race, income, or gender, across districts or buildings. Policy makers and practitioners should also be reminded that messaging and marketing the rationale and intent of policies are important elements of deploying those policies. In addition, though using cutoff scores on established criteria to assign students to different programs or educational settings seems fair, it may create otherwise artificial distinctions between students who are similar in prior experience and impact their subsequent self-perceptions and academic performance.

The full study is in Shaun M. Dougherty , “Bridging the Discontinuity in Adolescent Literacy? Mixed Evidence from a Middle Grades Intervention,” Education Finance and Policy, Spring 2015, Vol. 10, No. 2, Pages 157-192.