Evaluation of the Breakthrough Success Community

The CORE Districts received a Networks for School Improvement grant from the Gates Foundation to launch a network focused on improving ninth-grade on-track rates. CORE based its approach on the Institute for Healthcare Improvement’s Breakthrough Collaborative model, where a hub identifies areas where practices diverge from research and translates those research findings into actionable recommendations and measurements to gauge success. CORE invested significantly in measures to support the network, developing the Breakthrough Success Community (BTSC) on-track metric, which improved on binary metrics by estimating five lanes of “on-trackness.” However, CORE paused the metric in 2019–20 due to concerns and introduced the Developmental Relationships Survey as a measure of adult–student relationships. The revised on-track metric returned in winter 2021–22, and although educators valued the measure, there were questions about its validity and transparency.

From 2018–19 to 2021–22, CORE recruited three cohorts of high schools (41 schools total) to BTSC, with each cohort receiving 2–4 years of support during the period covered by this evaluation. COVID-19 interrupted Cohorts 1 and 2, and Cohort 3 launched after the pandemic. Recruitment saw limited engagement and did not include system analyses to help schools identify gaps for BTSC to address or see existing system assets that could serve as a foundation for improvement.

CORE’s review of research led to five focus areas, or “drivers”: adult teaming, relationships, grading, transitions, and master schedule. BTSC provided a Key Actions Checklist to guide testing ideas in these drivers, emphasizing one-on-one adult–student interactions and building on student assets over scaling system-level changes. Some schools made progress with BTSC, characterized by strong relational trust, engaged administrators, adaptation of BTSC ideas, and monitoring of student outcomes. However, most schools did not fully implement BTSC practices across the ninth grade.

To assess student outcomes, we compared BTSC schools in the CORE Districts to a matched set of schools. Analysis of trends in BTSC on-track metric points and ninth-grade D/F rates showed comparable patterns for BTSC and comparison schools. Regression analyses found no evidence of an impact of BTSC on on-track points in either 2021–22 or 2022–23 and no effects on the D/F rate in 2022–23. The results were consistent across a range of specifications of the regression models.

Since our data collection concluded, BTSC has completed another year, and CORE will enroll a fourth cohort. Recent adjustments to BTSC may have addressed some issues. We conclude with recommendations to improve the success of similar initiatives in education.