Measuring Student Academic Growth

More than a decade ago, states began using “student proficiency” classifications as a primary method of holding schools accountable for providing high-quality instruction. In recent years, many states have moved beyond considering student proficiency at a single point in time and started using “growth models” to show whether students are making progress towards greater academic proficiency from grade to grade. There are several models currently in use by various states to capture this growth. Although the validity and fairness of these growth models have been evaluated for the general population, the impact of the models for English learner (EL) students had not been previously studied.

In a recent article, Joni M. Lakin (University of Alabama) and John W. Young (Achieve the Core) compared three widely used growth models when applied to a longitudinal set of student achievement data provided by a large school district in California. The authors compared growth model predictions of students’ future proficiency from earlier grades (3^rd–5^th) to their actual proficiency level at 7^th grade. The authors found that all three models were more likely to make prediction errors for EL students than for non-EL students, leading to inaccurate classifications of which EL students (who were non proficient in grades 3–5) would be proficient by 7^th grade. The most common error was to predict that EL students would remain non-proficient, when in fact they were proficient by 7^th grade. The authors found that 16–28% of future-proficient EL students were missed by the model compared to 13–17% of non-EL students. One possible explanation is that, although EL students score poorly in early grades (not surprising given their language support needs), their growth trajectory is steeper than non-EL students. Therefore, assuming that all students grow at the same pace, as existing models do, creates more error for EL students and may disadvantage the students as well as schools that serve a large proportion of EL students.

Policy Implications

Model errors have important policy implications for how growth models can be effectively used to hold schools accountable for effective instruction. If a model fails to recognize students who will be proficient in the future, this may lead educators to implement unnecessary interventions or may result in schools and teachers not receiving credit for effective instruction in those early grades. Comparisons of model behavior for important student subgroups (in addition to the general student population) should inform the selection of models for accountability policies.

At the state level, California has not yet adopted an accountability policy based on models of student growth. However, growth models (indicating whether students are “on-track to college- and career-readiness”) are part of the assessment plans of the Smarter Balanced Assessment Consortia, of which California is a governing state and which may replace the existing school accountability system. Some districts and other organizations are also using growth models for other purposes. Most notably, the LA Times used one variation of a growth model, the value-added model (VAM), to report teacher effectiveness ratings, a practice that is hotly contested in the educational measurement field due to questions of the validity of these models for identifying effectiveness at the teacher level. Although the study described here did not directly test VAMs, evidence of significant differences in model performance across student groups strongly calls into question the validity of any model that makes assumptions about student growth trajectories without consideration of the types of students a teacher serves.

The full study is in Lakin, Joni M. and John W. Young, Evaluating Growth for ELL Students: Implications for Accountability Policies, Educational Measurement: Issues and Practice, Volume 32, Issue 3, pages 11–26, Fall 2013.