The Importance of Formative Evaluation in California’s Universal Pre–K Rollout

California’s plan to expand the existing Transitional Kindergarten program as part of a universal pre–K program for all four-year-olds marks a substantial investment in the state’s children and families. In order to ensure a successful rollout, California should learn from recent research on other states’ pre–K programs and fund research infrastructure for formative evaluation that will help identify and address implementation challenges, drive continuous quality improvement, and ultimately ensure the provision of rich, developmentally appropriate classrooms for all California children.

High quality, pre–Kindergarten programs can provide substantial benefits for young children, both in terms of immediate readiness for school and lifelong well-being. Research from across the United States, including Boston, Tulsa, New Jersey, Michigan, South Carolina, West Virginia, and North Carolina has demonstrated benefits at the end of the pre–Kindergarten year; and, in some evaluations, through middle school, high school, and beyond.

Drawing on this evidence regarding the benefits of preschool, California is planning the expansion of Transitional Kindergarten (TK)—a year of early education provided in public schools, currently available only to 4-year-old children born between September and December—into a universal pre–K program for all 4-year-olds (CA UPK). Previous research has shown that California’s TK program supports children’s literacy and mathematics skills more than other available educational options, marking the expansion to UPK as a substantial investment into both the development of the state’s youngest learners and the well-being of working families, whose struggles accessing and paying for child care in the wake of COVID-19 have been documented extensively, for example by The Education Trust—West, the Pew Research Center, CalMatters, and National Public Radio.

But such an expansion is not easy. School leaders are already grappling with many challenges related to UPK, including: finding and creating appropriate spaces for serving young children, locating qualified early educators, building educator capacity with professional development and coaching, identifying developmentally appropriate curricula and assessments, and aligning these with efforts already in place for older children.

Earlier this year, leaders looking to research to guide their decision-making were met with troubling news in the form of results from a study exploring the impact of another state’s pre–K expansion: Tennessee’s Statewide Voluntary Pre-K (TN-VPK) program. The TN-VPK study was conducted to evaluate Tennessee’s investment in state pre–K, and began shortly after the state scaled its pilot program statewide. The study is one of the few randomized-control trials able to track long-term pre–K outcomes, and as such has been watched closely by researchers, politicians, and practitioners. The most recent follow-up found that in sixth grade, children offered a slot in TN-VPK in 2009–11 had lower academic skills, increased rates of disability identification, and more disciplinary experiences than students who applied but were not offered a slot. Effect sizes were modest but meaningful. For example, children who were offered a slot in TN-VPK were 2.2% more likely to have a disciplinary infraction on their record than those who were not. Children’s math and reading scores were lower than those of their non-TN-VPK peers, with effect sizes ranging from 0.15 to 0.20.

These findings highlight the challenges inherent to program expansion—challenges that California policymakers and school leaders currently face. Young children require high-quality learning environments in order to thrive, but quality is difficult to scale. What’s more, as previous policy efforts have shown, difficulty with scaling is particularly likely to impact children from minoritized, low-income, or otherwise marginalized backgrounds. The provision of low-quality or developmentally inappropriate care, as the findings from Tennessee remind us, may not give California children the start they deserve.

For California, the key question raised by the TN-VPK study is: How can such an outcome be avoided as we scale-up to universal pre-K? Unfortunately, the Tennessee researchers had limited data available to answer this question. For example, though research consistently highlights the importance of teacher–child interactions, sequenced curricula and professional development, and time engaged in learning, these factors went unmeasured in most study classrooms. Data that did exist suggested low to moderate quality; in fact, researchers from the TN-VPK project have hypothesized that the study result stemmed from developmentally inappropriate classrooms and substantial time lost to transitions. Researchers have been unable, however, to link classroom features to children’s outcomes over time. Additionally, the study did not document where children who were not attending TN-VPK spent their time, preventing the researchers from understanding whether TN-VPK children were underperforming children who spent time at home, in other preschool settings, or in some other type of care—a factor that has proved essential in understanding other early childhood experiments. In sum, Tennessee’s high-quality impact study, while informative, was not by itself positioned to clearly guide Tennessee public schools in efforts to improve TN-VPK. Indeed, in the years following the study, researchers in Tennessee have done extensive data collection on what kinds of classrooms best help students, with the goal of both better understanding the findings from the impact evaluation and better supporting ongoing improvement efforts.

Both the lingering questions about and follow-up research on what happened in Tennessee should guide California as it moves forward with CA UPK. California should continue to look to research on high-quality state pre–K programs operating in public schools—e.g., in Tulsa, Oklahoma, and Boston, Massachusetts—and retain an age-appropriate developmental focus despite rolling out CA UPK in environments originally designed to develop the academic skills of older students.

But perhaps more importantly, California must fund a research infrastructure for formative evaluation that allows for regular data collection and collation at the child, classroom, and school level statewide. Such a system would provide essential information on the implementation of CA UPK in its early years and would set targets for improvement.

The state must establish a system for collecting data on:

• structural features of classrooms (e.g., teacher qualifications, curricula use, teacher–child ratios, space for outdoor play);
• observations of CA UPK classroom interactions and children’s experiences (e.g., quality of teacher–child interactions, time spent on academic content, instructional formats);
• and information on where children spend their wraparound care hours.

This information, alongside data on schoolwide supports for teachers and regular, holistic assessments of children’s developmental progress, would present rich opportunities for the state to identify what works and to support continuous improvement.

This infrastructure can be built in a variety of ways, but funding and support from the state is essential. For legislators, this means an up-front investment in a formative evaluation infrastructure, including purchasing and conducting trainings for established evaluation tools (e.g., classroom assessment measures) and funding the development of aligned reporting systems for districts that allow for consistency and ease of data collection statewide. This may also include funding research positions at the state or district level.

At the state level specifically, this means integrating UPK data into the new cradle-to-career data system and linking disparate data systems so school systems can assess the characteristics of CA UPK schools, families, and teachers, as well as information on what happens in CA UPK classrooms, all in the same database.

For district leaders, this may mean reaching out to researchers who could support data collection and evaluation efforts. Partnerships between researchers and practitioners have become more common in the early childhood sector, and have driven key improvements in many places (e.g., North Carolina and Louisiana, both of which have invested substantially in ongoing data collection efforts).

Researchers in universities, think tanks, and other settings should reach out to local school systems and build connections to support formative evaluation. In particular, researchers in the state university systems could consider collaborations across sites to help build a statewide portrait of CA UPK classrooms as expansion occurs.

California’s expansion of TK into a statewide, universal pre–K program is an important step in supporting the development of young children—one that, if implemented well, could contribute to closing opportunity gaps that have pervasive and damaging effects. But effective implementation at scale is not easy, and challenges will inevitably arise. By investing in systematic data collection and formative evaluation up front, the state can learn from TN-VPK and ensure the ongoing learning needed to create a high-quality system for California children and families.

Anna J. Markowitz is an assistant professor in the UCLA Graduate School of Education and Information Studies. Jade V. Marcus Jenkins is an associate professor in the UCI School of Education.