Strengthening Community Schools Through Improved Data Systems

Introduction

At a time when students of families living in poverty have experienced the worst of the economic trauma from the COVID-19 pandemic, community schools have reemerged as a promising intervention for addressing lack of access to quality education, health care, healthy foods, and economic prosperity (Castrechini & London, 2012; Jacobson et al., 2018; Maier et al., 2017; Voight & Hanson, 2017).

Community schools represent a place-based school improvement strategy in which “schools partner with community agencies and local government to provide an integrated focus on academics, health and social services, youth and community development, and community engagement” (Maier et al., 2017). President Joe Biden and leaders of several states have significantly increased investments in community schools, including the $443 million of President Biden’s Build Back Better Act budget allocated to their expansion (U.S. Department of Education, 2021). The California budget includes $2.8 billion in funding to expand community schools (California Department of Education, 2021), and Vermont, New York, and other states are making similar investments (Camera, 2021). Some policy leaders are calling for creating and expanding “community school systems” as a mechanism for moving the model beyond pockets of innovation (McDaniels, 2018). Attention to systemic challenges has also emerged, with calls for “getting the data right, and using it,” to better assess students’ health, social, and emotional needs and the efficacy of programs designed to address those needs (Starr, 2018).

These dynamics, combined with California’s investment in a statewide, cross-agency Cradle-to-Career data system, create a unique opportunity to bring discussion of data as a significant component of advancing educational equity into closer conversation with research on how community schools

address educational opportunity gaps. California’s Cradle-to-Career data system will link K–12, college, and career data systems and will eventually include health and welfare systems, finally allowing school systems access to information on students’ full academic and life trajectory—access that has limited even the most data-savvy school systems from monitoring progress towards academic goals. In this brief, we share findings from a 2021 research project that involved close collaboration between university researchers with experience in data systems and county administrators implementing an evidenced-based community school model at scale. Next, we briefly describe the research on community schools and data- based decision-making that guided the design of the county-university collaboration, and we make recommendations for how leaders in other counties and districts can develop data systems that support the implementation and improvement of community-schooling strategies.

LACOE Community Schools Initiative

The Los Angeles County Office of Education (LACOE) started the Community Schools Initiative (CSI) pilot implementation at 15 school sites during the 2019–20 school year with the goal of helping schools to close opportunity gaps and address long-standing inequities. Starting in the fall of 2019, LACOE partnered with each school and its respective school district as well as with the Los Angeles County Department of Mental Health, the Los Angeles County Workforce Development Board, and many other key countywide and school community organizations. A community school site coordinator at each school site was hired to lead implementation, establish advisory councils at each school, and initiate and sustain a number of LACOE-wide and school-specific partnerships based on LACOE CSI school needs. School site coordinators and, at some sites, education specialists were charged with conducting semiannual needs assessments, establishing and implementing an annual CSI plan, forming school-specific partnerships to meet the needs of students and families, and serving as a primary referral service to connect caretakers and students with established community partnerships. These partnerships ranged in type and intensity of service, including providing connections to food banks, linking families to mental health services, and tutoring and other academic intervention services.

The Four Pillars of Community School Implementation and Continuous Improvement

The LACOE model is grounded in the four pillars of community schools established by the Learning Policy Institute (LPI; Maier et al., 2017; Oakes et al., 2017). These four pillars are (a) integrated student supports, (b) expanded and enriched learning time opportunities, (c) active family and community engagement, and (d) collaborative leadership practices. The intent of the framework is to ground activity in research-based practices while simultaneously acknowledging the need for site-specific interventions. LACOE also instituted guidance on the practices and competencies associated with each pillar by specifying implementation practices at the school sites, including establishing an advisory council, conducting semiannual needs assessments, engaging staff in data collection, conducting a community school survey to gauge implementation, and establishing a community school annual plan. These principles for effective community schools were folded into a broader framework of using data for continuous improvement.

Using Data to Drive Program Implementation and Continuous Improvement

Strong data orientation and associated practices allow school system leaders to address some of the staff fears of using data and replace that fear with confidence using data as a tool for making improvements in the classroom, delivering programs, and making decisions (cf. Cannata et al., 2017; Grunow & Hough, 2018; Harvard Strategic Data Partnership, n.d.; Ikemoto & Marsh, 2007). To do this well, school site staff need underlying knowledge on how to understand, interpret, analyze, and use data to identify areas for improvement and adjust their actions accordingly.

Towards this end, one of the primary roles the CSI site coordinators play is to use data to achieve desired educational outcomes and greater equity. Community school staff are expected to use several types of data, including the following: (a) data that are traditionally collected and maintained as part of school improvement efforts, such as attendance data, student contact information, and academic outcomes to support targeted interventions for groups of students and individual students; (b) data that are independently collected on community-school-specific program delivery to families, such as the number of referrals made to external providers or the number of parent workshops; and (c) community data to understand the school community’s needs. Coordinators collect, analyze, and present data to the advisory council, school administrators, and other school stakeholders to lead the semiannual needs assessment process and develop the CSI annual plan.

LACOE Community Schools Office collaborated with faculty and students affiliated with the USC Center on Education Policy, Equity and Governance (CEPEG). The collaboration was aimed at helping CSI choose a set of indicators to track and develop a data system specifically designed to move beyond school-level data and incorporate community-level indicators of wellness. This was important because addressing the needs of the whole child (as community schools aim to do) depends on understanding student needs and connecting families with appropriate services.

The need for a dynamic and local data dashboard

The primary goals of the partnership between LACOE and USC CEPEG were to understand the CSI program and its implementation, review existing data systems, and make recommendations for building a community schools data dashboard. Because school site CSI staff needed access to a variety of school- specific, community-focused, and statewide data, all at different intervals of time, LACOE requested recommendations for what data were most important to monitor on a regular basis to help staff with program implementation and how to develop and maintain a dashboard to support data monitoring.

Access to accurate, quality, and timely data is necessary for effective program implementation and continuous improvement. Data dashboards are tools that include curated data from a variety of sources to help track, visualize, and analyze important information. They are particularly useful for combining data from disparate sources and showing only what a specific audience needs as well as for providing the ability to dig deeper into data sources through various functionalities. Such dashboards can be used by community school site coordinators, school leadership and educators, and school site advisory councils.

Recommendations for Developing a Community Schools Indicator Dashboard

The following recommendations describe several essential steps for developing a dynamic CSI dashboard that provides staff with timely, relevant, accurate, and actionable data. These recommendations speak to the need for both academic and nonacademic outcomes that are connected to LPI’s four pillars of community schools.

Develop data systems that support both real-time action and reflection on program efficacy

Along with encouraging staff to align certain activities and outcomes with the four pillars, we recommend developing a dual framework for success indicators to encourage staff to frame in a consistent way how they use data and for what purpose. Community school staff have at least two primary data needs. The first need is for ongoing, quickly adaptive data on student progress (student contact information, academics, engagement, needed interventions) and family needs (services provided, engagement in workshops, quality partnerships and linkages executed). LACOE CSI staff regularly need to access these data to contact families, identify student and family needs, and track ongoing service delivery. The secondary need is for “summative,” or less frequently updated, data that serve two purposes: (a) provide feedback on whether staff inputs and outputs are leading to desired outcomes (e.g., A–G course completion at the end of the year) and (b) provide information on community needs (rates of uninsured individuals or number of undocumented families in the community).

To support these two uses, we recommend establishing a data dashboard solution that integrates both leading and lagging indicators. Using leading and lagging indicators is a common practice in the business community for differentiating between key performance indicators that inform the strategy needed to move towards a goal and the indicators that allow organizations to look back and evaluate their performance. Leading indicators are key inputs captured in data, usually viewed in short-term intervals, that generate specific outcomes that indicate progress towards a goal. The essential component of a lead indicator is that it has a causal relationship with the outcome of interest (e.g., if a student receives passing grades, they will be on track to graduate). Lagging indicators, on the other hand, retrospectively tell the story of how inputs led to desired outcomes. Some leading and lagging indicators may intersect, but it is important to maintain this framework of thinking in that leading indicators can be updated more quickly, can be used to guide continuous improvement, and can provide the right guideposts at quick intervals so that necessary adjustments to organizational strategy can be made while staff can still achieve desired outcomes. (See Table 1 for further elaboration.)

Using adaptive and frequently updated data is important for community school staff. Although districts may eventually want to develop a dynamic dashboard that enables both types of data to “live” in one central place within different reports, in most districts that custom dashboard does not currently exist and would rely on district staff to connect previously disconnected data systems, such as school districts’ student information systems (SIS), state Department of Education data, and community-level data. The good news is that most data that staff need (detailed in the following sections may already be collected by other agencies (school districts, the state, and university partners, among other sources). That means community school staff can focus their data collection on the narrow set of data that no other agency collects. The challenge is that the data collected by different agencies, typically with different definitions and styles of data collection, will likely require data use agreements and inevitably add a significant data-capacity need to the community school implementation team.

Focus on a subset of key academic and community performance leading and lagging indicators

The LACOE CSI initiative had preestablished academic goals for the community schools program. The indicators recommended through this project focus on a subset of important academic and community indicators that measure their progress towards those goals (see Tables 2 and 3).

Academic indicators include measures of progress in and completion of courses required for high school graduation as well as early warning indicators like chronic absenteeism, course grades, and suspension (Bowles & Krivoshey, 2014). Data collection for all these indicators depends on the status of the indicator as either leading or lagging and the extent to which the data are already collected. For all leading and real- time indicators, the primary solution we recommend is to develop a dashboard that pulls from the district SIS to provide community school staff with the frequency of data they need to deliver their interventions and programs. Alternately, to facilitate agreements across a region, a county office of education can establish memorandums of understanding with each school system to grant community school site staff access to the SIS and provide quality training on how to retrieve, analyze, and maintain the established indicators. Community school staff would need to orient themselves around a key set of indicators (detailed in Table 2) and could also supplement with additional data that may be important and specific to their school site.

Community indicators include measures of conditions that residents face in the neighborhoods surrounding the school (e.g., food insecurity, housing instability, or homelessness). These are data points that schools typically do not collect or use. However, these kinds of indicators are often collected by regional partners and can be used in community school implementation. For example, in Los Angeles County, most of the data elements highlighted in Table 3 have been collected at the county level by a university partner, the USC Sol Price Center for Social Innovation; only three community indicators from the full priority list—referral status, mental health, and partnership status—need to be collected, cleaned, and maintained by school site staff. Although this data collection partnership is unique to Los Angeles County, organizations in other localities could replicate it since all the data used to create the indicators are available in census and other noneducation databases.

Develop clear, dynamic, and focused data-driven systems

To achieve the goal of establishing a data dashboard to embed data more effectively into various components of community school implementation, we recommend the following:

• Ensure access. Ensure that site-level staff have access to SIS and student-level data. Community school staff should be embedded into schools in ways that allow comparable access to personal information about students. This will help staff to identify and monitor student needs, target adequate support and resources, and assess the progress and/or efficacy of their efforts.
• Provide data integration. Spreadsheets and data worksheets should be assigned to and maintained by staff members to manage the various types of data they need. Absent a more permanent solution like a custom-built dashboard, we recommend establishing a central place (e.g., a document and resource hub such as a website or a document with all important data sources) with consistent data collection that provides staff with more accessibility as well as guidelines and support.
• Define all data. A technical and more widely accessible definition resource with all data definitions can help staff understand the data they encounter regularly. This can take the form of a data glossary that includes all types of data that staff encounter, technical and more broadly accessible definitions, indications on which data are the most important to track and why, and any details about accessing or analyzing those data.
• Ensure data can be analyzed dynamically by intervention and need. Ensure that staff at all levels can manipulate data by filtering for different student populations, including by subgroups (e.g., race, gender, foster youth, homeless), level of intervention needed (e.g., ability to identify students at risk or at any specific tier of intervention), individual students (e.g., ability to view identifiable data like attendance, grades, and behavior), and service/intervention (e.g., accessed tutoring or mental health service in the past and details on the outcome of that specific service or intervention).
• Provide clear presentation of data and short time-to-insight. In reporting data, provide clear and visualized data that make transparent the trends and implications, are easily digested and shared, and are ready to share with various audiences. Time-to-insight is the expectation that it will require the least amount of time possible for multiple audiences to understand the bottom line of the information being shared as well as have access to the details that enable clear understanding. Data should be clearly labeled in easy-to-understand ways using language that is commonly shared among staff. Ideally, this should include a graph style that allows staff to understand primary trends in the data and color coding (e.g., stoplight colors) to suggest what is desirable and undesirable in the data.
• Provide the ability to track unmet needs. Include a space for tracking unmet needs in the primary data collection that is used to track referrals and partnerships. This includes a designated place on the dashboard to input needs raised by families (e.g., housing if at-risk of homelessness) that the program may not be able to meet with existing referrals. This is important to understand any emerging trends for family and community needs.

Establish a community schools data dashboard advisory team

Creating and sustaining a dynamic dashboard that community school site-level staff and leadership regularly use will require a significant level of maintenance. For a district to build in structural feedback from various stakeholders as well as input on system revisions, we recommend establishing a Community School Evaluation or Data Advisory Team. This team should comprise staff who regularly use the dashboard or are key recipients of the information it provides, including school leadership, community school site staff, educators, and advisory council members (parents/caregivers). The purpose of the team is multifaceted and should be designed by the community school advisory team itself, focusing on providing iterative feedback on the design, data inclusion, presentation, and effectiveness of the dashboard. The team’s purpose should also include broader reflections on the status of program implementation at the school site, including identifying promising practices through the data as well as areas that need improvement and practices that should be replaced with different approaches.

Build capacity for protecting privacy of student data

Protecting student information and ensuring privacy when reporting school information or data sharing have been both a long-time necessary practice and a challenge for school systems. Under the Family Educational Rights and Privacy Act (FERPA), tracking, reporting, and sharing information require schools to maintain information privacy for students and families. In most cases, schools must have written permission from the parent or guardian or eligible student to release any personally identifiable information from a student’s education record to external parties (U.S. Department of Education, 2022). Community schools using data for continuous improvement can do so in ways that are consistent with FERPA and existing state privacy laws. The federal government has developed a useful resource for community schools given their mission to help students access support services. This resource addresses questions such as what types of data might require consent and where FERPA rules offer waivers for consent (U.S. Department of Education, 2016). Although state laws vary, centralized systems such as statewide longitudinal data systems are encouraged as valuable components of any local data plan because they ensure privacy through uniform protections. At the local level, it may behoove school systems to establish criteria for which local officials (which may include teachers and clerical staff) have by law “legitimate educational interest” in the data and to develop processes for making sure access is limited to these staff.

Expected Challenges

The challenges of creating effective data systems for community schools are similar to those already present for traditional schools. For districts or counties operating at scale, this includes gathering data from disparate student data systems, often with varying definitions and measurement approaches to the same indicator (e.g., chronic absenteeism measured as distinct numbers of days or as classes missed across school districts or school sites). The saying “data don’t drive” applies to community schools as well. In community schools, as in traditional schools, staff within and across schools and levels of the district or county will be more or less welcoming of the use of data sharing and progress monitoring. There are challenges unique to community schools as well, including how to use both academic or school-based data (i.e., data typically collected by schools) and nonacademic data that originate from varying community sources. Interpreting and explaining these data for the purposes of data-based decision-making may require additional training for staff, including those who already feel competent with using data from existing systems and practices. The Cradle-to-Career data system is being developed with the express purpose of helping to support this kind of data access and use, and we recommend that districts/counties include design specifically for integration of new data provided by this system as well as mechanisms for building local capacity to use data effectively.

Conclusion

Effective use of data will be key to the success of expanding community schools, either as single-school endeavors or within school systems like the LACOE CSI model. The first step is to establish academic and nonacademic community data indicators to guide understanding, community need, and implementation of the community school model at the school site. Districts and counties can and should play a lead role in developing dashboards to house these data and expanding training to help staff effectively use academic and community-level data. Recommendations to develop more robust and responsive data systems can be applied to other communities that are considering expanding their community school approach.

References

Bowles, S., & Krivoshey, A. (2014, August). College persistence indicators research review. American Institutes for Research. https://www.air.org/sites/default/files/downloads/report/College-Persistence-Indicators-August-2014.pdf

California Department of Education. (2021, October 5). Budget act for 2021–22: Information. https://www.cde.ca.gov/fg/fr/eb/yr21ltr0811.asp

Camera, L. (2021, August 25). ‘Community schools’ see revival in time of heightened need. U.S. News. https://www.usnews.com/news/education-news/articles/2021-08-25/community-schools-see-revival-in-time-of-heightened-need

Castrechini, S., and London, R. (2012, February). Positive student outcomes in community schools. Center for American Progress. https://files.eric.ed.gov/fulltext/ED535614.pdf

Cannata, M., Redding, C., & Rubin, M. (2017). Continuous improvement in action: Educators’ data use for school improvement. Vanderbilt Scaling Up Center. https://my.vanderbilt.edu/marisacannata/files/2013/10/PDSA-in-Action-102417.pdf

Grunow, A., & Hough, H. (2018, September). Continuous improvement: Building system capacity to learn [Brief]. Policy Analysis for California Education. https://gettingdowntofacts.com/sites/default/files/2018-09/GDTFII_Brief_CI.pdf

Harvard Strategic Data Project. (n.d.). Strategic use of data rubric. Center for Education Policy Research at Harvard University. Retrieved September 31, 2021, from https://sdp.cepr.harvard.edu/strategic-use-data-rubric

Ikemoto, G., & Marsh, J. (2007). Cutting through the “data driven” mantra: Different conceptions of data- driven decision making. Teachers College Record, 2015, 105–131. https://journals.sagepub.com/doi/pdf/10.1177/016146810710901310

Jacobson, R. (2016, November 18). Community schools: A place-based approach to education and neighborhood change [Discussion paper]. The Brookings Institution. https://www.brookings.edu/articles/community-schools-a-place-based-approach-to-education-and-neighborhood-change

Maier, A., Daniel, J., Oakes, J., & Lam, L. (2017, June 5). Community schools as an effective school improvement strategy: A review of the evidence. Learning Policy Institute. https://learningpolicyinstitute.org/sites/default/files/product-files/Community_Schools_Effective_REPORT.pdf

McDaniels, A. (2018, August 22). Building community schools systems: Removing barriers to success in U.S. public schools [Report]. Center for American Progress. https://www.americanprogress.org/article/building-community-schools-systems

Maier, A., Daniel, J., Oakes, J., & Lam, L. (2017, June 5). Community schools as an effective school improvement strategy: A review of the evidence. Learning Policy Institute. https://learningpolicyinstitute.org/sites/default/files/product- files/Community_Schools_Effective_REPORT.pdf

McDaniels, A. (2018, August 22). Building community schools systems: Removing barriers to success in U.S. public schools [Report]. Center for American Progress. https://www.americanprogress.org/article/building-community-schools-systems

Oakes, J., Maier, A., & Daniel, J. (2017, June 5). Community schools: An evidence-based strategy for equitable school improvement. Learning Policy Institute. https://learningpolicyinstitute.org/product/community-schools-equitable-improvement-brief

Starr, J. S. (2018). 2 challenges for community schools. Phi Delta Kappan, 99(6), 70–71. https://kappanonline.org/starr-challenges-community-schools

U.S. Department of Education. (2016, March). Data sharing tool kit for communities: How to leverage community relationships while protecting student privacy. https://www2.ed.gov/programs/promiseneighborhoods/datasharingtool.pdf

U.S. Department of Education. (2021, July 14). Frequently asked questions: Using American Rescue Plan funding to support full-service community schools & related strategies. https://oese.ed.gov/files/2021/07/21-0138-ARP-Community-Schools-OMB-and-OS-Approved- 071421-1.pdf

U.S. Department of Education. (2022, January 31). Protecting student privacy: Privacy and data sharing. https://www2.ed.gov/programs/promiseneighborhoods/datasharingtool.pdf

Voight, A., & Hanson, T. (2017). How are middle school climate and academic performance related across schools and over time? [REL 2017–212]. Regional Educational Laboratory West. https://files.eric.ed.gov/fulltext/ED572366.pdf