The Scarlet Letter as an "F" Not an "A"

The L.A. Times publication in August of a value-added analysis of teacher effectiveness, based on student scores on math and ELA tests, has sparked national debate about the ethics of publicly ranking individual teachers. Educators and researchers questioned the usefulness of student scores on standardized tests as teacher effectiveness measures, the merits of teacher performance-pay systems, and whether journalist-researchers should be allowed to conduct studies that university Human Subjects boards would flatly reject.

That’s a lot to chew on. Now that New York City, the largest school district in the nation, has decided to publicly release a ranking of its 12,000 teachers (twice the number of teachers ranked in the L.A. Times study), it's bound to happen at other school districts across the country.

A few questions come to mind as the movement to brand teachers with a Scarlett Letter gains momentum.

First, how do we acknowledge and address all the ‘noise’ in statistical attempts to measure student achievement and tie it to teacher performance? Standardized test scores are not perfect; they don’t capture all the influences on student learning. Let’s talk about influences other than teacher performance that account for student test scores—and let’s acknowledge that even Value-Added Analyses don’t fully control for out-of-classroom variables affecting student test scores.

Second, how do we feel about tying the educational goals for our schools and children so closely to what standardized tests can measure? Stated as cliché: is the tail wagging the dog?

On the one hand, proponents argue that the tests adequately measure the kind of student learning our states have determined is important. On the other hand, critics point to the fact that these tests can be validated only to measure the ability of students to take the tests—not necessarily to be literate in English or use math successfully, and certainly not to become creative, critical, productive thinkers or responsible, ethical citizens in this complicated world. Where do you stand on this issue?

Third, will these kinds of public rankings accomplish their intended goals? Several researchers have argued that the value of performance-based pay systems is that good teachers will be motivated to stay in the profession and bad teachers will be pushed to either improve or leave teaching. I see published teacher rankings as similar, though perhaps in converse: if merit pay plans are supposed to reward teachers who do well, these rankings spotlight teachers who don’t. But is this really going to happen? Also, if these ranking projects are intended to identify ineffective teachers and offer useful information for parents and other stakeholders interested in school improvement, then can we be certain that these rankings are accurate enough to warrant the high-stakes consequences of publishing them?

And, finally, how might we educational researchers develop a better instrument (or cluster of instruments) that captures and measures the teacher performance we believe leads to the kind of student outcomes our states desire?

Leaving aside input-based models of teacher effectiveness (such as level of professional education, years teaching, and teacher test scores—none of which has been found to correlate strongly with student success), let’s focus on outcome-based indicators of teacher performance. Who among us has some good ideas about integrating multiple measures (including supervisor evaluations, observations, school climate information, qualitative student data) effectively and cost-efficiently to identify what kinds of teacher characteristics, approaches, and pedagogies lead to well-rounded and robust student achievement—and don’t require us to resort to public shaming in local newspapers?

Before we jump on this new Scarlet Lettering bandwagon, I hope educational researchers, policy makers, and journalists can slow down, thoughtfully discuss and debate the related issues, and be sure to include respect for teachers and teaching in the process of identifying and scoring what it takes to support rigorous learning for our children.