• An IRT Mixture Model for Rating Scale Confusion Associated with Negatively Worded Items in Measures of Social-Emotional Learning

    Daniel M. Bolt, Yang Caroline Wang, Robert H. Meyer, Libby Pier. October 2019

    We illustrate the application of mixture IRT models to evaluate the possibility of respondent confusion due to the negative wording of certain items on a social-emotional learning (SEL) assessment. Using actual student self-report ratings on four social-emotional learning scales collected from students in grades 3-12 from CORE districts in the state of California, we also evaluate the consequences of the potential confusion in biasing student- and school-level scores as well as correlational relationships between SEL and student-level variables. Models of both full and partial confusion are examined. Our results suggest that (1) rating scale confusion due to negatively-worded items does appear to be present; (2) the confusion is most prevalent at lower grade levels (3rd-5th); and (3) the occurrence of confusion is positively related to reading proficiency and ELL status, as anticipated, and bias estimates of SEL correlations with these student-level variables. For these reasons, we suggest future iterations of the SEL measures use only positively oriented items. To maintain measurement continuity, we suggest bias corrections based on the studied mixture model may be useful, although the precision of such corrections is sensitive to the nature of confusion (e.g., full versus partial).

  • Students with Growth Mindset Learn More in School: Evidence from California’s CORE School Districts

    Susana Claro, Susanna Loeb. October 2019

    While the importance of social-emotional learning for student success is well established, educators and researchers have less knowledge and agreement about which social-emotional skills are most important for students and how these skills distribute across student subgroups. Using a rich longitudinal dataset of 221,840 fourth through seventh grade students in California districts, this paper describes growth mindset gaps across student groups, and confirms, at a large scale, the predictive power of growth mindset for achievement gains, even with unusually rich controls for students’ background, previous achievement, and measures of other social-emotional skills. Average annual growth in English language arts and math corresponding to differences between students with fixed and growth mindset in a same school and grade level is 0.07 and 0.05 standard deviations respectively, after adjusting for students’ characteristics and previous achievement. This estimate is equivalent to 48 and 35 additional days of learning.

  • Learning and Practicing Continuous Improvement: Lessons from the CORE Districts

    H. Alix Gallagher, Benjamin W. Cottingham. October 2019

    Continuous improvement has become a leading method of changing the way schools and districts foster better student learning and success. As part of the CORE-PACE Research Partnership, PACE spent a year studying the CORE Districts’ approach to implementing continuous improvement with a focus on two key questions: 1) What do we know about how to support educators in learning continuous improvement? 2) What conditions support continuous improvement in districts and schools? The findings are presented in a report that provides an overview of lessons learned in building a successful continuous improvement culture, and three detailed case studies exploring key factors to that success, including leadership, systems of support, and structures and processes.

  • Bridging the Knowing-Doing Gap for Continuous Improvement: The Case of Long Beach Unified School District

    Vicki Park. Policy Analysis for California Education. October 2019

    Successful continuous improvement requires educators to have a shared clarity of purpose, integrated systems of support, and a clear vision for instruction across the system with the central goal of improving classroom instruction. This case study examines how Long Beach Unified School District, one of the CORE districts involved in the CORE-PACE research project on continuous improvement, fosters these efforts. It is a portrait of a learning system that emphasizes improvement towards high-quality, rigorous instruction for all students through professional learning and capacity-building. Their efforts offer critical insights and reflection for other systems and leaders interested in supporting continuous improvement for both student and adult learning.

  • A Student-Centered Culture of Improvement: The Case of Garden Grove Unified School District

    Benjamin W. Cottingham, Angela Gong, H. Alix Gallagher. Policy Analysis for California Education. October 2019

    A crucial but challenging requirement of successful continuous improvement involves transforming the system’s culture. This case study explores how Garden Grove Unified School District built a culture that puts kids first; nurtures commitment, drive, and loyalty among teachers and other district personnel; and views both student and adult learning as important. This case examines four structures and processes used by Garden Grove leadership to establish and maintain a culture of improvement that has resulted in rising student achievement. The lessons learned could be implemented in many districts by changing leadership approaches and reallocating existing resources.

  • Leadership that Supports Continuous Improvement: The Case of Ayer Elementary

    Kate E. Kennedy, H. Alix Gallagher. Policy Analysis for California Education. October 2019

    Ayer Elementary School in Fresno is an exemplar of leadership practice necessary for successfully building and maintaining a culture of continuous improvement. This case study examines the leadership practices that teachers say allowed them to undertake the challenging work of using data for evidence-based changes that are steadily improving student outcomes in this ethnically diverse, high-poverty school. The report offers insights into how leaders can foster a culture of risk-taking, teacher agency, and collective efficacy. It also raises questions about how to support more principals in learning the leadership skills necessary to support the desired spread of continuous improvement in California.

  • How Would Test Opt-out Impact Accountability Measures? Evidence from the CORE Districts and the PACE/USC Rossier Poll

    Edward J. Cremata. Policy Analysis for California Education. September 2019

    The number of students opting out of standardized tests has grown in recent years. This phenomenon poses a potential threat to our ability to accurately measure student achievement in schools and districts. This brief documents the extent to which opting out is observed in the CORE districts and models how higher opt-out levels could affect various accountability measures. More students opting out could significantly impact some accountability measures in use in California, but the CORE districts’ growth measure is largely unaffected, as it reports the impact of schools on individual students’ achievement. In contrast, accountability metrics that track student achievement by cohort are at risk of becoming biased even with relatively low absolute levels of opting out. This brief suggests that districts should consider explicitly adjusting for the characteristics of the students who actually sit for tests when designing school accountability systems.

  • On Growth Models, Time for California to Show Some Improvement

    Morgan S. Polikoff. September 2019

    California is one of just two states (with Kansas) that does not use a student-level growth model to measure school performance. This brief lays out a number of common beliefs about growth models and provides evidence that these beliefs are inaccurate or unsupported. In so doing, the brief makes a positive case that the state should adopt such a model and replace the current "change" metric in the California School Dashboard.

  • Self-Management Skills and Student Achievement Gains: Evidence from California’s CORE Districts

    Susana Claro, Susanna Loeb. Policy Analysis for California Education. September 2019

    Existing research on self-management skills shows that measures of self- management predict student success. However, these conclusions are based on small samples or narrowly defined self-management measures. Using a rich longitudinal dataset of 221,840 fourth through seventh grade students, this paper describes self-management gaps across student groups, and confirms, at a large scale, the predictive power of self-management for achievement gains, even with unusually rich controls for students’ background, previous achievement, and measures of other social-emotional skills.

  • 12th Grade Course-taking and the Distribution of Opportunity for College Readiness in Mathematics

    Minahil Asim, Michal Kurlaender, Sherrie Reed. Policy Analysis for California Education. August 2019

    In this report we explore the patterns in mathematics course-taking among California public high school seniors. We describe what courses students are enrolled in and how course participation varies by key student characteristics, such as race/ethnicity, socioeconomic status, and performance level on the state’s 11th grade assessments. We also explore course-taking patterns for students eligible for California’s public four-year colleges—California State University (CSU) and the University of California (UC), and for applicants and admitted students at the CSU and UC.

Creative Commons License


PACE thanks these funders and sponsors for their financial support

PACE Funders and Sponsors