An IRT Mixture Model for Rating Scale Confusion Associated with Negatively Worded Items in Measures of Social-Emotional Learning

This article illustrates the application of mixture IRT models to evaluate respondent confusion due to the negative wording of certain items on a social-emotional learning (SEL) assessment. Using actual student self-report ratings on four social-emotional learning scales collected from students in Grades 3–12 from CORE Districts in the state of California, it also evaluates the consequences of the potential confusion in biasing student- and school-level scores, as well as the estimated correlational relationships between SEL constructs and student-level variables. Models of both full and partial confusion are examined. The results suggest that (1) rating scale confusion due to negatively worded items does appear to be present; (2) the confusion is most prevalent at lower grade levels (Grades 3–5); and (3) the occurrence of confusion is positively related to both reading proficiency and ELL status, as anticipated, and consequently biases estimates of SEL correlations with these student-level variables. For these reasons, future iterations of the SEL measures should use only positively oriented items.

This article was originally published in Applied Measurement in Education by Routledge and Taylor & Francis Group.