Generating Traction with Continuous Improvement

Lessons Learned: Gaining Traction for Improvement

During the 2020–21 school year, PACE examined two hubs leading continuous improvement networks focused on increasing the rate at which students are on track to graduate high school positioned for postsecondary success. Both hubs offered:

• coaching, with expectation of monthly meetings;
• professional development from hub staff and external content experts;
• frameworks, tools, and common data to promote reform efforts; and
• access to a network of sites doing similar work.

To analyze the impact of work at these sites, PACE employed a framework developed by Cynthia Coburn (2003).1 Coburn examined attempts at education reform that had a lasting impact on student outcomes and, based on that research, suggested that we broaden our conceptualization of what it means to “scale” reform to include four components:

• Spread: Principles central to the reform are taken up by increasing numbers of people across a system and are embedded in formal policies and organizational culture.
• Depth: Consequential reform shifts education system beliefs, norms, and pedagogy.
• Ownership: Ownership transfers to districts, schools, and teachers, even if the reform was initially required and/or supported by external mandates or organizations.
• Sustainability: The reform persists over time, driving long-term impact.

During 2020–21, 51% of teams gained some traction. For teams that built traction, we saw a consistent sequence of traction initiation when teams in districts or schools began to own the improvement work and were testing ideas for ways to change their systems that had sufficient depth to improve student outcomes (36% of participating teams). After this point, we saw traction build further if they were able to spread the work outside the team to other departments, which typically required administrative support and buy-in from well-respected teacher leaders (15% of participating teams). We did not see any teams achieve sustainability during that year. Forty-nine percent of participating teams did not gain any traction toward scale (that is, there was no evidence of progress toward attaining the four dimensions of scale). During 2020–21, many California schools—including most in our study—offered very limited in-person instruction, which likely hampered the work. Additionally, there were 2 years left on the grant funding the work, so we would not have expected the work to be self-sustaining at this point. From our analysis of how the hubs interacted with the districts and schools in their networks, we offer six lessons for how hubs can best support improvement.

Coaches started by building trusting relationships to understand and support the sites they were serving.

While it is widely recognized that establishing trust provides a critical foundation for learning and change, it can be challenging for hub staff to build trust both because they are charged with getting teams to dig deep to do difficult work and because some sites’ cultures are resistant to external influence. Starting work with a system analysis supported relationship development because it gave sites and their coaches a chance to set goals for improvement based on current outcomes and build a shared understanding of site priorities, assets, and challenges. When sites found that coaches listened to them and valued their expertise during an initial system analysis, that built a foundation for meaningful engagement with the work.

Hubs played a critical role by holding space for teams to meet and providing accountability for moving work forward.

Educators have many demands on their time, and the COVID-19 pandemic has exacerbated many educators’ feelings of stress and burnout. In this context, many teams struggled initially to get into a regular rhythm of improvement work. Especially when work is getting started, hubs can provide crucial support by creating a sense of accountability for teams to meet and accomplish tasks.

When the work with the hub was coherent with sites’ priorities and context, it was more likely to gain traction.

One conundrum facing improvement efforts is that some schools or districts engage in them only somewhat voluntarily. This was true in these networks as well. Some schools felt “voluntold” to join the networks PACE studied by their districts; more broadly, some districts engaged in improvement work because of persistent challenges in performance identified on the state dashboard. Regardless of how schools and districts come into the work, it is critical that the hub finds ways to weave the work into existing site priorities, which can build ownership for both teacher leaders and administrators. By leveraging their knowledge of the local context and their knowledge of the hub’s plan, improvement coaches could identify ways to mesh local priorities and assets with the work of the improvement network. Network leaders played a critical role as well by designing programs that provided focus for efforts while offering guardrails within which sites could adapt the work.

Hubs were better positioned to affect systems when sites developed cross-functional teams.

Selecting a cross-functional team (with team members who had varied roles within the organization) to lead improvement work was beneficial for multiple reasons. Administrators could elevate and champion the work, make key decisions, allocate resources, and keep the work aligned with other parts of the system. Teachers and counselors provided invaluable insights into potential implementation challenges and served as initial testers of new approaches so that change ideas could be adapted for feasibility in the local context before they were spread. Finally, once the work had sufficient depth and ownership, a multifunctional team was best positioned to spread the work into other parts of the organizational structure (for example, into additional departments, grade levels, or other schools in the district).

Continuous improvement gained traction when hubs presented it as straightforward logic for making decisions using varied data, with simple tools and minimal jargon.

Continuous improvement requires thoughtful data use, and improvement methodologies typically have technical approaches and tools to help scaffold the work. But when educators are first starting improvement work, sophisticated measures, complicated data systems, and detailed tools can be overwhelming and can exceed both the data capacity of school systems and the comfort level of educators. What educators seem to miss if the initial presentation of improvement work is too technical is that improvement centers on basic logic:

1. Can a promising change be implemented well in a given context?
2. Might the idea matter for student outcomes when implemented reasonably well?
3. When the idea is spread to others, does it affect interim measures of student outcomes and ultimately lead to progress toward the aim?

Each of these questions requires evidence. Especially in initial pilots of a new approach, data should be captured frequently and easily. Improvement work gained traction when coaches helped teams to build regular habits of testing, examine evidence, and ask whether the change appeared to support progress toward the aim, providing complicated measures and technical tools only when they moved teams’ work forward.

Hubs can improve the depth of work by supporting sites with selecting promising ideas to test.

Hubs can play an important role in helping sites to select promising practices to test so that teams do not spend time reinventing the wheel or testing ideas that are tangential to their aim. Continuous improvement processes are ideally suited for helping educators determine how to adapt an idea that worked elsewhere to their context, figure out which ideas cannot be implemented well in their context, and decide when an idea is ready to scale across a school or district. The greatest catalyst for improvement work was evidence of positive impacts on students, and by suggesting promising ideas for change, hubs helped sites make more progress faster.

Recommendations

To help hubs enact these six lessons and support sites in doing improvement work that scales, we offer seven recommendations to external support providers, county offices of education, or districts working with their schools to improve student outcomes at scale:

• Prompt participant ownership of the work early and often.
• Approach continuous improvement principles as practical and common sense.
• Flexibly integrate tools, structures, frameworks, and data so that teams can more easily digest and ultimately sustain them.
• Value the existing expertise and the assets and constraints in participating local educational agencies and take on a stance as a partner in the work.
• Center improvement work on teacher knowledge.
• Focus on the few actions that will lead to the greatest impact (that is, the Pareto principle).
• Transform student outcomes by testing changes that improve instruction.

Ultimately, we believe that continuous improvement holds great potential as an approach for hubs working with schools and districts as long as hubs meet sites where they are and avoid overly technical solutions unless they prove to be necessary for the given circumstances.

The authors would like to thank the CORE Districts and California Education Partners for welcoming us to their public events and internal planning sessions. We are also grateful for the time and insights that the staff at these organizations and the busy educators in partner districts provided during interviews. We would like to thank Shawn Bernardo for his steady support of our data collection and the entire PACE team for their ongoing thought partnership. This report, like all PACE publications, has been thoroughly reviewed for factual accuracy and research integrity. The authors assume full responsibility for the accuracy of the report contents.

• 1Coburn, C. E. (2003). Rethinking scale: Moving beyond numbers to deep and lasting change. Educational researcher, 32(6), 3–12.