Corrected: An earlier version of this story contained incorrect information on the Advancement Via Individual Determination, or AVID, program’s participation in the federal Investing in Innovation grant program. AVID applied for but did not win a grant through the i3 program.
Includes updates and/or revisions.
Individual interventions intended to improve academic skills, such as the popular Advancement Via Individual Determination, or AVID, program, may not secure a student’s path to graduation and college without a schoolwide structure to support it, according to a study from the Consortium on Chicago School Research.
In a report set for release in the fall and previewed at the American Educational Research Association convention in New Orleans in April, researchers analyzed how AVID, a study-skills intervention for middle-achieving students, played out in 14 Chicago high schools. They found AVID participants in 9th grade gained little advantage that year over peers not taking part in the program, and remained off track for graduation and college.
The study highlights a potential pitfall for the dozens of student-based interventions aiming to scale up nationwide through private support and programs like the federal Investing in Innovation, or i3, program: As programs move out of the schools for which they were originally developed, their success becomes increasingly dependent on individual schools’ context and capacity.
“We’re not really trying to say, does AVID work or doesn’t it, but what has been its impact in the Chicago context,” said Jenny Nagaoka, the Chicago Consortium’s associate director and postsecondary-studies manager, and a study co-author. “It’s not a transformative experience for the AVID student; it’s not doing enough to change the trajectory of these students for graduation.”
‘The Real Payoffs’
After adjusting for academic and socioeconomic differences between Chicago students who took part in the AVID program and those who had never had access to it, researchers found AVID seemed to have little impact on students’ GPAs.
SOURCE: Consortium on Chicago School Research
Ms. Nagaoka and her CCSR colleagues Melissa R. Roderick, a senior director, and Melanie LaForce, a research analyst, studied 14,031 students who attended 9th grade from 2004-05 to 2007-08, including 2,521 AVID students. The students came from 14 schools with stable 9th grade AVID programs with student data from both before and after the programs’ implementation. The schools also had to follow the basic AVID model, rather than tweaking it to start in 10th grade or after school.
Unlike other studies of the program, the CCSR researchers tried to control for the fact that AVID chooses students who are already highly motivated to achieve.
Ms. Nagaoka used a method known as “propensity matching.” She paired students who attended after AVID was implemented with 7,357 9th graders who attended the same schools in 2002 and 2003, before AVID, and compared the performance of students with access to the program with that of nonparticipating students who had similar academic and socioeconomic profiles.
Ms. Nagaoka found that students participating in AVID had average weighted grade point averages of 2.32 in English and 1.9 in mathematics on a 4-point scale. Those averages were slightly better than nonparticipating students’ GPAs of 2.06 in English and 1.75 in math, but not good enough for the participants to be considered on track for graduation. No significant effects were seen on students’ gains on state tests in reading, math, or science.
Robert P. Gira, the executive vice president of the San Diego-based AVID, said the Chicago study was too short-term to be conclusive, because student academic gains from AVID build over a student’s high school career. “We expect 9th graders to be making some progress, but the real payoffs start to happen two to three years later,” Mr. Gira said.
Ms. Nagaoka said the research team is also conducting long-term studies of AVID in Chicago schools.
Doug Rohrer, a psychology professor at the University of South Florida, in Tampa, found the CCSR study more rigorous than prior AVID research.
In a September 2010 analysis, the U.S. Department of Education’s What Works Clearinghouse found only one of 66 AVID studies met its quality standards. Based on that study, the clearinghouse found AVID had “no discernible effects on adolescent literacy.”
Mr. Rohrer said school leaders should consider their entire school improvement approach in selecting an intervention for their own campus.
“The critical question in my mind,” Mr. Rohrer continued, “is whether AVID is better than requiring students to go to another class, such as an extra dose of math or writing. Learning how to take notes is a fine strategy, but it might not help you in Algebra 2 if you haven’t learned Algebra 1.”
The school context in which an AVID program is implemented may well be the crux of the problem for AVID implementation, both Ms. Nagaoka and Mr. Gira agreed.
Chicago Context
Used in more than 4,500 school nationwide, the program was developed in San Diego in 1980, when low-income, minority, and English-language-learner students were first bused into the then-white, middle-income Clairemont High School. It encourages middle-achieving students—defined as those with a B, C, or D average—to enroll in advanced academic classes. Students take a daily elective that teaches study skills, organization, and critical thinking, and provides tutoring.
Chicago started AVID in eight schools in the 2003-04 school year and has expanded it to 59 high schools, as part of a larger effort to boost college access.
Unlike those early AVID schools, Ms. Nagaoka said, the Chicago AVID schools are generally low-performing, without a long-established infrastructure of AP courses or an existing core of high-achieving peers.“If you look within these schools, the ‘middle achieving’ kids are middle-achieving on a national scale,” she said, “but within these schools, these are actually the highest-achieving students.”
The consortium, based at the University of Chicago, also surveyed students about their study behaviors, such as whether they set aside time for homework or studying or study even when the topic is uninteresting. The AVID students were found to have slightly better study habits than non-AVID peers, but during classroom observations, Ms. Nagaoka said, “it seems their classroom experiences are very similar to those of their classmates.”
