Impact of a Classroom-Based Guidance Program on Student Performance in Community College Math Classes

With the passage of the Student Success Act of 2012, leaders of higher educational institutions in California are grappling with how to comply with its new requirements. One of the main recommendations of the Student Success Task Force was to restructure the way student support services are delivered to increase the quality of assistance students receive early in their college careers. Our randomized-controlled evaluation of a guidance program implemented at South Texas College (STC) in McAllen, Texas, may hold some lessons for policy-makers and practitioners in California, and for their counterparts around the country.

For many students in community college, passing through remedial and gatekeeper math courses is one of the highest hurdles preventing students from advancing towards a college credential. Recent efforts to improve the pass rates of students in remedial math have taken many forms, including structural changes in the sequence itself, reformed instructional strategies and providing extra support. Administrators at South Texas College (STC) wanted an effective, yet scalable and affordable intervention that would address students’ struggles in these entry-level math courses. The “Beacon Mentoring” program was a home-grown intervention that, at its core, took existing staff at STC, trained them in the basic elements of advising, and then sent these “Beacon Volunteers” to make short (10 minutes) presentations to students in their math classes, about three times during the semester. The presentations focused on information about support services that were available on campus, including a tutoring center, the financial aid office, and registration and enrollment deadlines. The Beacon Volunteers also gave out their contact information and encouraged students to get in touch for more information.

The evaluation relied on random assignment to determine which classes would be assigned a Volunteer (treatment group) and which would not (control group). In the Fall of 2007, over 2,100 students registered as they would normally for the targeted math courses.  In January of 2008, entire class sections of these math courses were randomly assigned to either the treatment group or the control group. Our study used enrollment, transcript and other administrative data to evaluate the impact of this light-touch intervention on subsequent student outcomes.

We found that being in the treatment group increased the probability that students visited the tutoring center by about 30 percent (19% of control vs. 26% of treatment group students visited the tutoring center). The program also led to a 20 percent reduction in student withdrawals from math classes, which did not translate into statistically significant increased pass rates (or fail rates) for students overall. However, for the (nearly) half of STC students who attended school part-time, the intervention resulted in a 10 percent increase in math class pass rates. These part-time students tend to be especially time-constrained and spend less time on campus, perhaps making information delivered to them in the classroom particularly valuable.

There are potential lessons for both researchers and practitioners to draw from this study of a light-touch intervention. First, on a technical note, whole-class randomization is less burdensome than individual-level random assignment for the institution and can deliver sound treatment and control groups—potentially making experimental evaluation of college programs more feasible. Second, more research is needed on the effectiveness of the prolific tutoring centers on college campuses. Third, the brevity of the in-classroom sessions combined with use of existing campus staff makes this approach both affordable and scalable. And finally, sending advisors to deliver information directly to students where they are mostly likely to be found—in the classroom—prior to their needing help, seems to be an effective way to deliver this type of information.

The full study (gated) can be found in Kristin F. Butcher and Mary G. Visher, “The Impact of a Classroom-based Guidance Program on Student Performance in Community College Math Classes,” in Education Evaluation and Policy Analysis, September 2013, vol. 35, no 3, pp. 298-323. An ungated report on the original research project can be found here.