The Impact of Unmotivated Questionnaire Respondents on Data Quality
Publication authors
Published
Summary
Education researchers use surveys widely. Yet, critics question respondents’ ability to provide high-quality responses. As schools increasingly use student surveys to drive policymaking, respondents’ (lack of) motivation to provide quality responses may threaten the wisdom of using surveys for data-based decision-making. To better understand student satisficing (sub-optimal responding on surveys) and its impact on data quality, we examined the pervasiveness and impact of this practice on a large-scale social-emotional learning survey administered to 409,721 students in grades 2-12. Findings...
Evidence from California’s CORE School Districts
Publication authors
Published
Summary
While the importance of social-emotional learning for student success is well established, educators and researchers have less knowledge and agreement about which social-emotional skills are most important for students and how these skills distribute across student subgroups. Using a rich longitudinal dataset of 221,840 fourth through seventh grade students in California districts, this paper describes growth mindset gaps across student groups, and confirms, at a large scale, the predictive power of growth mindset for achievement gains, even with unusually rich controls for students’...
Published
Summary
We illustrate the application of mixture IRT models to evaluate the possibility of respondent confusion due to the negative wording of certain items on a social-emotional learning (SEL) assessment. Using actual student self-report ratings on four social-emotional learning scales collected from students in grades 3-12 from CORE districts in the state of California, we also evaluate the consequences of the potential confusion in biasing student- and school-level scores as well as correlational relationships between SEL and student-level variables. Models of both full and partial confusion are...
Consistent Gender Differences in Students’ Self-Efficacy
Published
Summary
Academic self-efficacy is a student’s belief in their ability to perform within a school environment. Prior research shows that students experience a drop in academic self-efficacy during middle school that is particularly steep for female students and results in lower self-efficacy for girls than boys throughout middle and high school. In this brief, we probe whether this pattern is consistent across student groups defined by demographics, achievement level, and school of attendance. We find unusual consistency: while non-white, low-achieving, and poor students show somewhat lower self...
Published
Summary
This brief applies value-added models to student surveys in the CORE Districts to explore whether social-emotional learning (SEL) surveys can be used to measure effective classroom-level supports for SEL. The authors find that classrooms differ in their effect on students’ growth in self-reported SEL—even after accounting for school-level effects. Results suggest that classroom-level effects within schools may be larger than school-level effects. However, the low explanatory power of the SEL models means it is unclear that these are causal effects that have appropriately controlled for student...
Findings From the First Large-Scale Panel Survey of Students
Published
Summary
Measures of school-level growth in student outcomes are common tools for assessing the impacts of schools. The vast majority of these measures use standardized tests as the outcome of interest, even though emerging evidence demonstrates the importance of social-emotional learning (SEL). This article presents results from using the first large-scale panel surveys of students on SEL to produce school-level, value-added measures by grade for growth mind-set, self-efficacy, self-management, and social awareness. The article finds substantive differences across schools in SEL growth, with...
Evidence from California’s CORE Districts
Publication authors
Published
Summary
Existing research on self-management skills shows that measures of self-management predict student success. However, these conclusions are based on small samples or narrowly defined self-management measures. Using a rich longitudinal dataset of 221,840 fourth through seventh grade students, this paper describes self-management gaps across student groups, and confirms, at a large scale, the predictive power of self-management for achievement gains, even with unusually rich controls for students’ background, previous achievement, and measures of other social-emotional skills. Self-management is...
Insights from Outlier Schools
Published
Summary
There is a growing consensus in education that schools can and should attend to students’ social-emotional development. Emerging research and popular texts indicate that students’ mindsets, beliefs, dispositions, emotions, and behaviors can advance outcomes, such as college readiness, career success, mental health, and relationships. Despite this growing awareness, many districts and schools are still struggling to implement strategies that develop students’ social-emotional skills. The purpose of this paper is to fill this gap by examining the social-emotional learning (SEL) practices in ten...
Publication authors
Published
Summary
Although there is a robust body of literature studying targets for academic indicators within school quality systems few studies explore target setting for non-academic indicators. Focusing on elementary schools within the CORE districts, we investigate how moving performance targets for non-academic indicators affects school quality ratings. We ask: (1) How does school performance on CORE’s school quality improvement measures vary across schools and over time?; and (2) How does the setting of targets on CORE’s non-academic indicators at various levels impact the number and types of schools...
Publication author
Published
Summary
The CORE districts have been measuring SEL via self-report student surveys since 2015. Any district, state, or school looking to use these surveys can now build from what we have learned in the CORE districts. In this paper we provide benchmarking data, including means and standard deviations by construct, grade level and subgroup, and examples of how to use these data in practice. The data come from nearly half a million students across the 8 CORE districts, in grades 4 through 12, who took the survey in the 2015-16 school year. While not a true national sample, the CORE benchmarking sample...
Published
Summary

School value-added models are increasingly used to measure schools’ contributions to student success. At the same time, policymakers and researchers agree that schools should support students’ social-emotional learning (SEL) as well as academic development. Yet, the evidence regarding whether schools can influence SEL and whether statistical growth models can appropriately measure this influence is limited. Recent work shows meaningful differences across schools in changes in SEL scores by grade, but whether these differences represent the effects of schools is still unclear. The current paper...

Publication author
Published
Summary

Children’s physical and mental health play critical roles in their development. Research shows that poor health in childhood adversely affects future success and that children in lower-income households are more likely to suffer health problems. Improvements in child health can lead to higher future economic growth and can improve the upward mobility of children from low-income families. California’s financial commitment to health care, including greater access to Medicaid, has reduced but not eliminated health gaps associated with poverty. Insurance coverage alone does not ensure access to...

A Pragmatic Approach to Validity and Reliability
Publication authors
Published
Summary
As educational practitioners and policymakers expand the range of student outcomes they assess, student perception surveys—particularly those targeting social-emotional learning—have grown in popularity. Despite excitement around the potential for measuring a wider array of important student outcomes, concerns about the validity of the inferences that might be drawn from student self-reports persist. One of the most ambitious attempts to incorporate student perception surveys into a larger assessment framework has occurred through CORE—a consortium of school districts in California. Pulling...
Evidence from the CORE Districts
Published
Summary
Mounting evidence demonstrates that social-emotional skills are important for students’ academic and life success, yet there is limited evidence on how these skills develop over time and how this development varies across student subgroups. This study uses the first large-scale panel survey of social-emotional learning (SEL) to describe how four SEL constructs—growth mindset, self-efficacy, self-management, and social awareness—develop from Grade 4 to Grade 12, and how these trends vary by gender, socioeconomic status, and race/ethnicity. The results are based on self-report student surveys...
An IRT Modeling Approach
Published
Summary
With an increased appreciation of students’ social-emotional skills among researchers and policy makers, many states and school districts are moving toward a systematic process to measure Social-Emotional Learning (SEL). In this study, we examine the measurement properties of California's CORE Districts’ SEL survey administered to over 400,000 students in grades 3 to 12 during the 2015-16 school year. We conduct analyses through both classical test theory and item response theory frameworks, applying three different polytomous IRT models on both the full student sample and on separate samples...
Findings from the First Large-Scale Panel Survey of Students
Published
Summary
Measures of school-level growth in student outcomes are common tools used to assess the impacts of schools. The vast majority of these measures are based on standardized tests, even though emerging evidence demonstrates the importance of social-emotional skills (SEL). This paper uses the first large-scale panel surveys of students on SEL to produce and evaluate school-level value-added measures by grade for growth mindset, self-efficacy, self-management, and social awareness. We find substantive differences across schools in SEL growth, of magnitudes similar to those for academic achievement...
Practices and Supports Employed in CORE Districts and Schools
Published
Summary
Social-emotional learning refers to the beliefs, attitudes, personality traits, and behaviors that students need to succeed in school and life. Our study looks closely at ten “outlier schools” in California’s CORE districts whose students report strong social-emotional learning outcomes compared to other, similar middle schools. The brief and infographic—based on a longer technical report—describe the surprising breadth and variety of social-emotional learning practices found in these outlier schools, as well as commonalities in their approaches and implementation challenges that some are...
Published
Summary
States and school districts across the U.S. are seeking to expand their definition of student success to include social-emotional learning. The CORE Districts, a collaborative of California districts that has developed a system of school accountability and continuous improvement that includes measures of social-emotional skills based on student self-reports, exemplify this trend. In this case study, we provide an overview of CORE's School Quality Improvement System, which was implemented in the 2015–16 school year across six districts serving roughly one million students.
Published
Summary

This report and accompanying policy brief show that there is good reason to pursue the measurement of social-emotional learning (SEL) and school culture/climate (CC) as a way to better understand student and school performance. Using data from California's CORE districts, we show that SEL and CC measures demonstrate reliability and validity, distinguish between schools, are related to other academic and non-academic measures, and also illuminate dimensions of student achievement that go beyond traditional indicators. We also show how the SEL and CC measures can be used to identify areas of...

Published
Summary
Although all students change schools when they are promoted from one school level to another, some students also move from one school to another for reasons other than promotion. The practice of students making non-promotional school changes is referred to as student mobility. Past research has documented that student mobility is widespread in the United States and often detrimental to the educational achievement of students. Yet little of this research has focused on the secondary level or examined mobility from the school perspective. This study examined three important aspects of student...
Published
Summary
Although all students change schools when they are promoted from one school level to another, some students also move from one school to another for reasons other than promotion. The practice of students making non-promotional school changes is referred to as student mobility. Past research has documented that student mobility is widespread in the United States and often detrimental to the educational achievement of students. Yet little of this research has focused on the secondary level or examined mobility from the school perspective. This study examined three important aspects of student...
Implications for Equity, Practice, and Implementation
Published
Summary
In July 1996, California embarked on its largest ever education reform: a nearly $1 billion class size reduction effort to improve literacy in the primary grades. Now in its second year, the Class Size Reduction (CSR) initiative provides $800 (up from $650 the first year) per student to schools that reduce class size to 20 students or fewer in first grade, second grade, and then third grade and/or kindergarten. The impetus for CSR came from several factors. A revived state economy created a revenue windfall. Under Proposition 98, a minimum amount of this surplus must be allocated to education...
The Influence of Household Support, Ethnicity, and Parental Practices
Published
Summary
Accumulating evidence shows that young children benefit develop­mentally by participating in quality childcare centers and preschools. But we know little about which family characteristics and home practices influence parents' selection of a center-based program. This article reports on the influence of the family's social-structural attributes, ethnicity, and parental practices on the likelihood of selecting a center-based program, after taking into account economic characteristics. The odds that parents enroll their child in a center-based program are greatest when mothers are more highly...
Child Care and Development Services for Children and Families: Phase III Final Report, Part 2
Published
Summary
Phase III was designed to further analyze the preliminary recommendations and to utilize the assistance of childcare and development community to re­design childcare and development policy. Professional judgment and experience was sought from representatives from the three lead agencies and the childcare and development field. For six of the nine tasks included in the project, work groups were assembled to discuss new proposals and ideas for improving services within the state. Over 80 people spent thousands of hours contributing their experience and expertise to these efforts. Three...
Child Care and Development Services for Children and Families: Phase III Final Report, Part 1
Published
Summary

Phase III was designed to further analyze the preliminary recommendations and to utilize the assistance of childcare and development community to re­design childcare and development policy. Professional judgment and experience was sought from representatives from the three lead agencies and the childcare and development field. For six of the nine tasks included in the project, work groups were assembled to discuss new proposals and ideas for improving services within the state. Over 80 people spent thousands of hours contributing their experience and expertise to these efforts. Three...