Designing School Systems to Encourage Data Use and Instructional Improvement

Today’s educators are inundated with different forms of data, with the expectation that they will use them routinely and systematically to support instruction in schools. Driving this “educational data movement” are new data management systems, consultants, coaches, data teams, protocols to facilitate data-driven conversations, and advocacy-oriented, “how-to” books. But, despite the press from advocates to incorporate data into decision-making, the research base lags behind. Specifically, what are the important organizational conditions that shape educators’ use of data and their ability to mobilize resources to support this organizational goal?  

In this study, recently published by Educational Administration Quarterly, I present data collected from four school systems in California—two traditional districts and two charter management organizations (CMOs)—to bring these organizational dimensions into relief.  First, data use in all four systems was related to federal and state accountability systems. Described by one teacher as “the ghost that is chasing us,” educators in all four pointed to data from state assessment scores (CSTs) and benchmark/interim assessments designed to model these high-stakes assessments as key sources of data because of the link to program improvement status and resulting penalties. Additionally, the CMOs’ data use was tied by accountability to their authorizers, the educational market, and their school communities. For charters, data were used as evidence of success of the school model, useful for attracting new families and for supporting reauthorization with their authorizers.

Second, certain organizational conditions—namely, structure and distribution of decision-making rights, size and growth trajectory, financial resources, degree of regulation—both enabled and constrained leaders to allocate resources to support data use within the organization. The state’s difficult financial environment had a similar limiting influence across all systems. For CMOs, their decentralized network structure, smaller size, and fewer regulations seemed to enable their resource allocation in three key areas: human capital development, tools and technologies, and organizational processes and routines.

What do these findings offer for policymakers? First, they point to tensions that arise when educators are faced with multiple forms of accountability, whether from federal or state actors, the educational market and local community, or other third-parties, like authorizers. These competing demands privileged the use of certain forms of data over others, creating complex tensions for educators. Particularly in response to changes related to Common Core State Standards and Smarter Balanced assessments, policymakers may want to consider the metrics chosen to demonstrate success when revising state and district accountability policies.

This study also lays the groundwork for the diffusion of promising practices across school districts and CMOs, as well as school districts like Los Angeles Unified that experiment with portfolio management models. For instance, districts had found ways of engaging resources (e.g., professional learning communities) to support teachers to help ameliorate the stresses associated with data use in a high-stakes accountability environment, an area where the CMOs, with their younger, less experienced teachers, still struggled. CMOs’ investments in advanced data management systems could provide a model for districts regarding how to merge multiple databases to enable more sophisticated data analysis.

For the full study, see Caitlin Farrell, “Designing School Systems to Encourage Data Use and Instructional Improvement: A Comparison of School Districts and Charter Management Organizations,” Educational Administration Quarterly. An ungated version is available here.