Improving Accountability through Expanded Measures of School Performance

California is one of the few states that implemented a school-based accountability system prior to No Child Left Behind. The Public School Accountability Act of 1999 created the Academic Performance Index (API) that includes ELA, mathematics, science, history/social studies, and writing, as well as high school graduation. This puts California ahead of many other states that are now trying to move beyond the NCLB measures. Like these other states, California is seeking to improve the API by including measures of student growth, readiness for college and careers and other important academic and social goals.

A recent RAND study outlined a set of principles to inform public education agencies about tradeoffs associated with the use of new measures. RAND found 20 states that published ratings of schools in 2008–2009 or 2009–2010 based on measures beyond those required by NCLB. These measures fall into four categories:

• student test performance in additional subjects
• growth in student performance over time
• indices that reflect student achievement along the entire spectrum of performance, from low to high
• college readiness

The authors identified three other types of measures that are rapidly becoming more common among states:

• indicators of a safe and supportive school environment
• indicators of risk for students not graduating on time
• results of “interim” academic assessments

An expanded set of measures might allow stakeholders to draw more valid inferences regarding school performance that better reflect the multiple goals of schooling, and could create incentives for educators to focus on processes or outcomes that have been underemphasized in the past.  With this in mind, we provide the following recommendations for state and local education agencies to consider.

• Develop customized local indicator systems to supplement federal or state requirements. Districts might consider collecting data about schools’ course offerings and student participation in certain curricula, non-tested skills such as civic engagement, innovative programs, extracurricular offerings, leading indicators such as attendance or on-time grade promotion, or other aspects of the education system that the district believes are worth making public If incentives are attached to performance, they should be carefully aligned with goals and should: (1) de-emphasize cut scores in favor of measures that capture performance throughout the distribution such as through a test score index; (2) balance individual- and group-level incentives in ways that support desirable outcomes such as a collaborative environment within the schools; and (3) move beyond educator incentives to include student incentives, keeping in mind that incentives for behaviors linked to positive outcomes, such as reading books, have shown some promise while incentives for test scores have not.
   
• Provide supports for teaching that are aligned with the system’s goals.  Teacher preparation and professional development, curriculum, instruction, and assessment should all point in the same direction. Of fundamental importance are resources and supports such as sample lesson plans and videos that are embedded in teachers’ day-to-day work and that help teachers engage in effective instruction that is aligned with the system’s goals.
   
• Conduct local validation studies.  Key validation questions include whether the measurement system provides district leaders with information they can use to improve their school support efforts, whether principals and teachers find the data useful for day-to-day decision making and for broader strategic planning, and whether any unintended shifts in curriculum, instruction, or resource allocation occur in response to the implementation of the system.
   
• Expand statewide indicator systems with input from local education agencies to promote comparability and cost sharing. An appropriate expansion of statewide systems of measures could shift costs from the local to the state level and provide better comparative data for districts. It is important that any new state requirements be responsive to local needs, so district and state leaders should work together to design systems that offer an appropriate balance between comparability and local flexibility, and that include measures of the processes and outcomes that are most highly valued at the local and state levels.

The impending adoption of new assessments aligned to the Common Core State Standards offers an opportunity to rethink not only how we measure student achievement, but the full range of information that can be collected to inform the decisions of families, educators, and policymakers. 

The full study [ungated] can be found in Schwartz, H. Hamilton, L.S., Stecher, B.M., & Steele, J.L. (2011).  Expanded measures of school performance.  Santa Monica, CA:  RAND. Additional information is also in the journal article [gated] Hamilton, L.S., Schwartz, H., Stecher, B.S., & Steele, J., (2013) "Improving Accountability through Expanded Measures of Performance", Journal of Educational Administration, Vol. 51 Iss: 4.