The Efficacy of Private Sector Providers in Improving Public Educational Outcomes
School districts are spending millions on tutoring outside regular school day hours to increase the achievement of economically and academically disadvantaged students. The No Child Left Behind Act (NCLB) sought to introduce greater choice, flexibility and accountability in public education by allowing parents of children in persistently low-performing schools to choose providers of out-of-school-time tutoring (known as Supplemental Educational Services or SES) for their children. Importantly, the law also required state and local educational agencies to assess provider effectiveness in increasing student achievement and to use this information to withdraw approval from ineffective SES providers. The benefits of choice in a competitive market can only be realized, however, if the purchasers (i.e., parents) also have sufficient information to make good choices of providers for their children.
This study presents findings from our evaluation of SES providers in a large, urban school district, Chicago Public Schools, that has accounted for a significant share of students receiving SES under NCLB. Our work with Chicago Public Schools began in 2009, and for each of three school years, we generated estimates of the performance of SES providers in improving students’ math and reading achievement. Our statistical models for producing this information aimed to put providers on a “level playing field” by accounting for the characteristics of the students they served (including their past achievement test scores) and the schools those students attended. Chicago Public Schools circulated this information on provider performance to principals of schools where students were eligible to receive SES and to parents choosing SES providers for their children. In the tutoring provider market there, we saw the hourly rates of tutoring providers decline over time (as providers tried to stay competitive with the district-run program that charged the lowest hourly rates) and a strong, positive link between provider effectiveness and the number of students they served each year.
Our findings indicate that attending SES can increase student achievement (as reflected in their standardized test scores), particularly for those who receive a greater number of hours of tutoring (e.g., 40 or more). The size of these effects is equivalent to about one third of the annual gains made by students in these schools. Our research also identified some basic attributes of successful approaches to the organization and management of these programs by districts (related to monitoring and oversight, coordination between providers and day school teachers, and efforts to reduce the costs of service delivery) that might be more widely adopted among other school districts arranging for these services, either under NCLB or following waivers from NCLB that give states and districts more flexibility in developing programs to reduce the achievement gap.
In addition, the estimation strategies we developed could be readily adopted by school districts to use in generating their own information on the effectiveness of providers of out-of-school-time tutoring interventions. And more school districts could follow the approach of Chicago Public Schools in making sure that students and parents get information on provider performance that they need to make better informed choices about supplemental educational services for their children. As eight school districts in California were recently granted a waiver from NCLB requirements to set up their own accountability system outside of the state, they could draw on the findings of this study in guiding program development and the design of their accountability systems. In fact, we are currently completing these same types of analyses on program impacts and provider effects for Los Angeles Unified School District (LAUSD).
The full study (gated) can be found in Carolyn J. Heinrich and Hiren Nisar, The Efficacy of Private Sector Providers in Improving Public Educational Outcomes, American Educational Research Journal, October 2013, vol. 50, no. 5, 856-894.