CORE-PACE Research Partnership Publications

  • Can We Measure Classroom Supports for Social-Emotional Learning?

    Robert H. Meyer, Libby Pier, Jordan Mader, Michal Christian, Andrew B. Rice, Susanna Loeb, Hans Fricke, Heather Hough. October 2019

    This brief applies value-added models to student surveys in the CORE Districts to explore whether social-emotional learning (SEL) surveys can be used to measure effective classroom-level supports for SEL. The authors find that classrooms differ in their effect on students’ growth in self-reported SEL—even after accounting for school-level effects. Results suggest that classroom-level effects within schools may be larger than school-level effects. However, the low explanatory power of the SEL models means it is unclear that these are causal effects that have appropriately controlled for student-level characteristics. Finally, there are generally low correlations between classroom-level growth in SEL and classroom-level growth in English language arts (ELA) or math, suggesting the SEL measures may capture growth not measured by academic test scores. Although results are preliminary, they indicate there might be measurable student growth in SEL impacted by the environment of classrooms within schools.

  • Assessing Survey Satisficing: The Impact of Unmotivated Questionnaire Respondents on Data Quality

    Christine Calderon Vriesema, Hunter Gehlbach. October 2019

    Education researchers use surveys widely. Yet, critics question respondents’ ability to provide high-quality responses. As schools increasingly use student surveys to drive policymaking, respondents’ (lack of) motivation to provide quality responses may threaten the wisdom of using surveys for data-based decision-making. To better understand student satisficing (sub- optimal responding on surveys) and its impact on data quality, we examined the pervasiveness and impact of this practice on a large-scale social-emotional learning survey administered to 409,721 students in grades 2-12. Findings indicated that despite the prevalence of satisficing in our sample, its impact on data quality appeared more modest than anticipated. We conclude by providing an accessible approach for defining and calculating satisficing for researchers, practitioners, and policymakers working with large-scale datasets.

  • A Middle School Drop: Consistent Gender Differences in Students’ Self-Efficacy

    Erin M. Fahle, Monica G. Lee, Susanna Loeb. October 2019

    Academic self-efficacy is a student’s belief in their ability to perform within a school environment. Prior research shows that students experience a drop in academic self-efficacy during middle school that is particularly steep for female students and results in lower self-efficacy for girls than boys throughout middle and high school. In this brief, we probe whether this pattern is consistent across student groups defined by demographics, achievement level, and school of attendance. We find unusual consistency: while non-white, low-achieving, and poor students show somewhat lower self-efficacy than other students, the differential drop in middle school is essentially universal across student groups. Similarly, while schools vary meaningfully in their students’ level of self-efficacy, they also do not differ much in this trend.

  • An IRT Mixture Model for Rating Scale Confusion Associated with Negatively Worded Items in Measures of Social-Emotional Learning

    Daniel M. Bolt, Yang Caroline Wang, Robert H. Meyer, Libby Pier. October 2019

    We illustrate the application of mixture IRT models to evaluate the possibility of respondent confusion due to the negative wording of certain items on a social-emotional learning (SEL) assessment. Using actual student self-report ratings on four social-emotional learning scales collected from students in grades 3-12 from CORE districts in the state of California, we also evaluate the consequences of the potential confusion in biasing student- and school-level scores as well as correlational relationships between SEL and student-level variables. Models of both full and partial confusion are examined. Our results suggest that (1) rating scale confusion due to negatively-worded items does appear to be present; (2) the confusion is most prevalent at lower grade levels (3rd-5th); and (3) the occurrence of confusion is positively related to reading proficiency and ELL status, as anticipated, and bias estimates of SEL correlations with these student-level variables. For these reasons, we suggest future iterations of the SEL measures use only positively oriented items. To maintain measurement continuity, we suggest bias corrections based on the studied mixture model may be useful, although the precision of such corrections is sensitive to the nature of confusion (e.g., full versus partial).

  • Students with Growth Mindset Learn More in School: Evidence from California’s CORE School Districts

    Susana Claro, Susanna Loeb. October 2019

    While the importance of social-emotional learning for student success is well established, educators and researchers have less knowledge and agreement about which social-emotional skills are most important for students and how these skills distribute across student subgroups. Using a rich longitudinal dataset of 221,840 fourth through seventh grade students in California districts, this paper describes growth mindset gaps across student groups, and confirms, at a large scale, the predictive power of growth mindset for achievement gains, even with unusually rich controls for students’ background, previous achievement, and measures of other social-emotional skills. Average annual growth in English language arts and math corresponding to differences between students with fixed and growth mindset in a same school and grade level is 0.07 and 0.05 standard deviations respectively, after adjusting for students’ characteristics and previous achievement. This estimate is equivalent to 48 and 35 additional days of learning.

  • Learning and Practicing Continuous Improvement: Lessons from the CORE Districts

    H. Alix Gallagher, Benjamin W. Cottingham. October 2019

    Continuous improvement has become a leading method of changing the way schools and districts foster better student learning and success. As part of the CORE-PACE Research Partnership, PACE spent a year studying the CORE Districts’ approach to implementing continuous improvement with a focus on two key questions: 1) What do we know about how to support educators in learning continuous improvement? 2) What conditions support continuous improvement in districts and schools? The findings are presented in a report that provides an overview of lessons learned in building a successful continuous improvement culture, and three detailed case studies exploring key factors to that success, including leadership, systems of support, and structures and processes.

  • Bridging the Knowing-Doing Gap for Continuous Improvement: The Case of Long Beach Unified School District

    Vicki Park. Policy Analysis for California Education. October 2019

    Successful continuous improvement requires educators to have a shared clarity of purpose, integrated systems of support, and a clear vision for instruction across the system with the central goal of improving classroom instruction. This case study examines how Long Beach Unified School District, one of the CORE districts involved in the CORE-PACE research project on continuous improvement, fosters these efforts. It is a portrait of a learning system that emphasizes improvement towards high-quality, rigorous instruction for all students through professional learning and capacity-building. Their efforts offer critical insights and reflection for other systems and leaders interested in supporting continuous improvement for both student and adult learning.

  • A Student-Centered Culture of Improvement: The Case of Garden Grove Unified School District

    Benjamin W. Cottingham, Angela Gong, H. Alix Gallagher. Policy Analysis for California Education. October 2019

    A crucial but challenging requirement of successful continuous improvement involves transforming the system’s culture. This case study explores how Garden Grove Unified School District built a culture that puts kids first; nurtures commitment, drive, and loyalty among teachers and other district personnel; and views both student and adult learning as important. This case examines four structures and processes used by Garden Grove leadership to establish and maintain a culture of improvement that has resulted in rising student achievement. The lessons learned could be implemented in many districts by changing leadership approaches and reallocating existing resources.

  • Leadership that Supports Continuous Improvement: The Case of Ayer Elementary

    Kate E. Kennedy, H. Alix Gallagher. Policy Analysis for California Education. October 2019

    Ayer Elementary School in Fresno is an exemplar of leadership practice necessary for successfully building and maintaining a culture of continuous improvement. This case study examines the leadership practices that teachers say allowed them to undertake the challenging work of using data for evidence-based changes that are steadily improving student outcomes in this ethnically diverse, high-poverty school. The report offers insights into how leaders can foster a culture of risk-taking, teacher agency, and collective efficacy. It also raises questions about how to support more principals in learning the leadership skills necessary to support the desired spread of continuous improvement in California.

  • How Would Test Opt-out Impact Accountability Measures? Evidence from the CORE Districts and the PACE/USC Rossier Poll

    Edward J. Cremata. Policy Analysis for California Education. September 2019

    The number of students opting out of standardized tests has grown in recent years. This phenomenon poses a potential threat to our ability to accurately measure student achievement in schools and districts. This brief documents the extent to which opting out is observed in the CORE districts and models how higher opt-out levels could affect various accountability measures. More students opting out could significantly impact some accountability measures in use in California, but the CORE districts’ growth measure is largely unaffected, as it reports the impact of schools on individual students’ achievement. In contrast, accountability metrics that track student achievement by cohort are at risk of becoming biased even with relatively low absolute levels of opting out. This brief suggests that districts should consider explicitly adjusting for the characteristics of the students who actually sit for tests when designing school accountability systems.

  • Self-Management Skills and Student Achievement Gains: Evidence from California’s CORE Districts

    Susana Claro, Susanna Loeb. Policy Analysis for California Education. September 2019

    Existing research on self-management skills shows that measures of self- management predict student success. However, these conclusions are based on small samples or narrowly defined self-management measures. Using a rich longitudinal dataset of 221,840 fourth through seventh grade students, this paper describes self-management gaps across student groups, and confirms, at a large scale, the predictive power of self-management for achievement gains, even with unusually rich controls for students’ background, previous achievement, and measures of other social-emotional skills.

  • The Properties of Non-Academic School Performance Measures

    Rachel S. White, Morgan S. Polikoff. Policy Analysis for California Education. May 2019

    Although there is a robust body of literature studying targets for academic indicators within school quality systems few studies explore target setting for non-academic indicators. Focusing on elementary schools within the CORE districts, we investigate how moving performance targets for non-academic indicators affects school quality ratings.

  • Making Sense of Social-Emotional Survey Results Using the CORE Districts' Benchmarking Data

    Katie Buckley, Ed.D.. May 2019

    The CORE districts have been measuring SEL via self-report student surveys since 2015. Any district, state, or school looking to use these surveys can now build from what we have learned in the CORE districts. In this paper we provide benchmarking data, including means and standard deviations by construct, grade level and subgroup, and examples of how to use these data in practice. The data come from nearly half a million students across the 8 CORE districts, in grades 4 through 12, who took the survey in the 2015-16 school year.

  • Measuring School Contributions to Growth in Social-Emotional Learning

    Hans Fricke, Susanna Loeb, Robert H. Meyer, Andrew B. Rice, Libby Pier, Heather Hough. Policy Analysis for California Education. May 2019

    School value-added models are increasingly used to measure schools’ contributions to student success. At the same time, policymakers and researchers agree that schools should support students’ social-emotional learning (SEL) as well as academic development. Yet, the evidence regarding whether schools can influence SEL and whether statistical growth models can appropriately measure this influence is limited.

  • Roll Call: Describing Chronically Absent Students, the Schools they Attend, and Implications for Accountability

    Heather Hough. Harvard Education Press. February 2019

    Student absenteeism has recently entered the national spotlight with its emphasis in the Every Student Succeeds Act, and here in California with its inclusion in the School Dashboard. Yet many questions remain about who chronically absent students are and how they are concentrated within schools. In chapter 1 (of the edited book, Absent from School), the author uses data from the CORE districts—which serve nearly one million students in over 1,000 schools in California–to better understand differences across students and schools, comparing these measures to a broader set of school performance indicators. First, the author describes attendance at the student level and how it varies by student characteristics. Then, she shows how schools perform on this metric, by school type, by subgroup, and across time. Finally, she describes how schools’ performance on chronic absence metrics corresponds to other accountability metrics and the related implications for reporting school-level measures of chronic absenteeism.

    Purchase book

  • Engaging District, School, and Teacher Leaders in Improvement

    H. Alix Gallagher, Angela Gong, Heather Hough, Kate E. Kennedy, Taylor N. Allbright, Eupha Jeanne Daramola. Policy Analysis for California Education. January 2019

    California’s shift towards continuous improvement in education makes understanding how districts and schools can learn to improve a more pressing question than ever. The CORE Improvement Community (CIC), a network of California school districts engaged in learning about improvement together, is an important testing ground to learn about what this work entails. 

  • Using Data for Improvement: Learning from the CORE Data Collaborative

    Heather Hough, Erika Byun, Laura Steen Mulfinger. Getting Down to Facts II. September 2018

    Experts agree that effective data use is critical for continuous improvement. However, there is a lack of understanding statewide about how data use for continuous improvement, with its adaptive and iterative nature, differs from data use for other purposes. In this paper, the authors discuss what data are most useful to inform continuous improvement at all levels of the system and provide a case study of how the CORE data collaborative uses a multiple-measures approach to support decision-making.

  • Measuring Social Emotional Learning Through Student Surveys in the CORE Districts: A Pragmatic Approach to Validity and Reliability

    Hunter Gehlbach, Heather Hough. Policy Analysis for California Education. May 2018

    As educational practitioners and policymakers expand the range of student outcomes they assess, student perception surveys—particularly those targeting social-emotional learning—have grown in popularity. Despite excitement around the potential for measuring a wider array of important student outcomes, concerns about the validity of the inferences that might be drawn from student self-reports persist. One of the most ambitious attempts to incorporate student perception surveys into a larger assessment framework has occurred through CORE—a consortium of school districts in California. Pulling from CORE’s data and their use within these districts, we summarize the evidence for validity and reliability of CORE’s student-report surveys on social-emotional learning through a pragmatic approach.

  • Trends in Student Social Emotional Learning: Evidence from the CORE Districts

    Martin R. West, Libby Pier, Hans Fricke, Heather Hough, Susanna Loeb, Robert H. Meyer, Andrew B. Rice. Policy Analysis for California Education. May 2018

    Mounting evidence demonstrates that social-emotional skills are important for students’ academic and life success, yet we have limited evidence on how these skills develop over time and how this development varies across student subgroups. In this study, we use the first large-scale panel survey of social-emotional learning (SEL) to describe how four SEL constructs—growth-mindset, self-efficacy, self-management, and social awareness—develop from Grade 4 to Grade 12, and how these trends vary by gender, socioeconomic status, and race/ethnicity.

  • School Effects on Social-Emotional Learning: Findings from the First Large-Scale Panel Survey of Students

    Susanna Loeb, Michael S. Christian, Heather Hough, Robert H. Meyer, Andrew B. Rice, Martin R. West. Policy Analysis for California Education. May 2018

    Measures of school-level growth in student outcomes are common tools used to assess the impacts of schools. The vast majority of these measures are based on standardized tests, even though emerging evidence demonstrates the importance of social-emotional skills (SEL). This paper uses the first large-scale panel surveys of students on SEL to produce and evaluate school-level value-added measures by grade for growth mindset, self-efficacy, self-management, and social awareness.

  • Measuring Students’ Social-Emotional Learning Among California’s CORE Districts: An IRT Modeling Approach

    Robert H. Meyer, Caroline Wang, Andrew B. Rice. Policy Analysis for California Education. May 2018

    With an increased appreciation of students’ social-emotional skills among researchers and policy makers, many states and school districts are moving toward a systematic process to measure Social-Emotional Learning (SEL). In this study, we examine the measurement properties of California's CORE Districts’ SEL survey administered to over 400,000 students in grades 3 to 12 during the 2015-16 school year.

  • Enacting Social-Emotional Learning: Practices and Supports Employed in CORE Districts and Schools

    Julie A. Marsh, Susan McKibben, Heather Hough, Michelle Hall, Taylor N. Allbright, Ananya M. Matewos, Caetano Siqueira. Policy Analysis for California Education. April 2018

    Social-emotional learning refers to the beliefs, attitudes, personality traits, and behaviors that students need to succeed in school and life. Our study looks closely at ten “outlier schools” in California’s CORE districts whose students report strong social-emotional learning outcomes compared to other, similar middle schools.

  • Building Systems Knowledge for Continuous Improvement: Early lessons from the CORE districts

    Michelle Nayfack, Vicki Park, Heather Hough, Larkin Willis. Policy Analysis for California Education. November 2017

    In California, recent policy shifts have created a high degree of local control with the expectation that school districts will think differently about school and district improvement. However, many districts lack the individual expertise and organizational capacity to support these changes at scale. In large part, this is due to a lack of a shared understanding of the routines, structures, and supports needed for school systems to develop and implement change ideas that dramatically improve student outcomes.

  • Continuous Improvement in Practice

    Heather Hough, Jason Willis, Alicia Grunow, Kelsey Krausen, Sylvia Kwon, Laura Steen Mulfinger, Sandra Park. Policy Analysis for California Education. November 2017

    Calls for “continuous improvement” in California’s K-12 education system are central to current discussions about school improvement in the state. Yet, definitions of continuous improvement vary, and knowledge of what continuous improvement looks like in practice is limited. To advance the conversation, this brief helps to define continuous improvement both in theory and in practice.

  • Exploring Improvement Science in Education: Promoting College Access in Fresno Unified School District

    Jorge Aguilar, Michelle Nayfack, Susan Bush-Mecenas. Policy Analysis for California Education. June 2017

    California’s Local Control Funding Formula (LCFF) requires districts to report multiple measures of student performance that reflect success in the goal of preparing students for college, career, and citizenship. As they engage in the Local Control Accountability Plan (LCAP) process, they are expected to use state and local indicator data from California’s School Dashboard to monitor student progress.

  • Using Surveys of Students' Social-Emotional Skills and School Climate for Accountability and Continuous Improvement

    Heather Hough, Demetra Kalogrides, Susanna Loeb. Policy Analysis for California Education. March 2017

    This report and accompanying policy brief show that there is good reason to pursue the measurement of social-emotional learning (SEL) and school culture/climate (CC) as a way to better understand student and school performance. Using data from California's CORE districts, we show that SEL and CC measures demonstrate reliability and validity, distinguish between schools, are related to other academic and non-academic measures, and also illuminate dimensions of student achievement that go beyond traditional indicators.

  • Local Control in Action: Learning from the CORE Districts' Focus on Measurement, Capacity Building, and Shared Accountability

    Julie A. Marsh, Susan Bush-Mecenas, Heather Hough. Policy Analysis for California Education. October 2016

    California and the nation are at the crossroads of a major shift in school accountability policy. At the state level, California’s Local Control and Accountability Plan (LCAP) encourages the use of multiple measures of school performance used locally to support continuous improvement and strategic resource allocation. Similarly, the federal Every Student Succeeds Act (ESSA) reinforces this local control, requiring more comprehensive assessment of school performance and a less prescriptive, local approach to school support. These changes represent a major cultural shift for California schools and districts.

  • Identity crisis: Multiple measures and the identification of schools under ESSA

    Heather Hough, Emily Penner, Joe Witte. Policy Analysis for California Education. August 2016

    The Every Student Succeeds Act (ESSA) makes sweeping changes to the way school performance is measured. Using the innovative measurement system developed by the CORE Districts in California, the authors explore how schools can be identified for support and improvement using a multiple measures framework.

  • Making Students Visible: Comparing Different Student Subgroup Sizes for Accountability

    Heather Hough, Joe Witte. Policy Analysis for California Education. May 2016

    With the passage of the Every Student Succeeds Act (ESSA) of 2015, California state policymakers are tasked with determining the subgroup threshold for school-level reporting. To inform this decision, this policy brief explores the implications of utilizing various subgroup sizes using data from the CORE Districts. The authors find that the 20+ subgroup size presents clear advantages in terms of the number of students represented, particularly in making historically underserved student populations visible.

  • Using Chronic Absence in a Multi-Metric Accountability System

    Heather Hough. Policy Analysis for California Education. April 2016

    With the passage of the Every Student Succeeds Act (ESSA) of 2015, California must integrate additional measures of student and school performance into the state-wide accountability system. To support the conversation as policymakers consider if/how to include chronic absenteeism data in the state’s accountability system, PACE has conducted an analysis of the CORE Districts’ student chronic absenteeism data.

Twitter

  •  
  • 1 of 864

PACE thanks these funders and sponsors for their financial support