Corrected: An earlier version of this story gave an incorrect name for Sandra L. Christenson’s university affiliation. She is an education psychology professor at the University of Minnesota in Minneapolis.
While more states and districts are developing “early warning systems” to target students most at risk of dropping out, many of them may still not be reaching students early enough, according to the first national study to look at the data-based identification-and-intervention practice.
A study released last week by Civic Enterprises, a Washington-based policy firm, and the Everyone Graduates Center at Johns Hopkins University in Baltimore found that at least 16 states now produce early-warning systems that flag students who are not “on track” to graduate from high school, while at least 18 others have plans to implement such systems. Only four states so far—Delaware, New Jersey, Oklahoma, and Virginia—provide early-warning feedback to educators on a weekly or daily basis. (At least one more state—Louisiana—launched its early-warning system since the study count was taken.)
“A lot of these school districts and states are awash in data,” said John M. Bridgeland, the chief executive officer of Civic Enterprises and a co-author of the study. “The big problem has been using that data in a way that’s useful to teachers, administrators, and community-based nonprofits to target in on kids.
“There are a lot of [early-warning systems] in schools across America, but a lot of them are put in place too late, in 9th grade,” Mr. Bridgeland said. “By the time kids are in 9th grade, a lot of them have already made their decisions about whether they are going to drop out.”
The researchers conducted detailed interviews and site visits at 16 “early adopter” districts in seven states. They found that many are already working on the next generation of indicator systems, intended to be more streamlined with information sent to teachers faster.
A growing number of states have early-warning data systems that flag students who are not “on track” to graduate from high school. Other states are in the process of developing such systems.
SOURCE: Data Quality Campaign
“The whole notion of using early-warning signs is critical,” said Sandra L. Christenson, an education psychology professor at the University of Minnesota in Minneapolis, who helped develop the Check and Connect dropout-intervention program used in Minneapolis public schools. “We’re trying to counter this hopelessness that can reside in the student and families when the student has not been very successful.”
Varied Indicators
States and districts have experimented with both streamlined and complex systems of indicators—at one point, Louisiana’s system tracked and analyzed more than 200 separate indicators. At minimum, researchers found that most existing systems flag what Robert Balfanz, a co-author of the study and the director of the Everyone Graduates Center, calls “the ABCs” of such systems:
• Attendance: Students who have missed either 10 percent of the school days or 20 days total;
• Behavior: Students who receive two or more mild or more-serious behavior citations, which in most schools means detentions or suspensions; and
• Course performance: Students who struggle to keep up in key classes at different grades.
The last indicator varies at critical transition grades. A student who can’t read on grade level by grade 3, when students begin moving beyond basic literacy to read to learn, is four times less likely to graduate by age 19 than a child who reads proficiently. Failing English or mathematics during grades 6-9 increases the chance a student will never catch up. The systems also tend to flag more general academic struggles associated with dropping out, such as a grade point average of less than 2.0, two or more failing grades in 9th grade, or not progressing from 9th to 10th grade on time.
Grades vs. Scores
While many state warning systems do include state assessment scores, Mr. Balfanz, whose center has helped districts and states develop early-warning systems, said school-based grades have proved more popular markers for educators. “Grades are a cumulative thing: Did you attend, did you try, did you get your work in? It includes all of those things,” Mr. Balfanz said. “The evidence shows [annual state] test scores are really not as predictive at the individual kid level.”
Flagging a student based on behavior also can be problematic, the study found, because discipline policies vary so drastically from state to state and even school to school. Mr. Balfanz found that schools with “zero tolerance” policies tend to have much higher overall discipline referrals, making it harder to tease out which students have the most severe underlying problems. But he said the number of discipline referrals will still predict students who are likelier to leave school, because “overreactive school policies” can prompt students to disengage.
The momentum to build and use these early-warning systems has developed incredibly rapidly. While it usually takes a decade or two for interventions identified in research to translate into practice, much of the research on dropout-warning systems has come out in the past five years.
Part of the impetus has come from an increasing federal focus on raising high school graduation rates. In the 2011-12 academic year, states, districts, and schools will for the first time be held accountable for their graduation rates based on a common federal metric in which cohorts of students entering 9th grade are tracked through graduation.
The study, funded by the AT&T Foundation, suggests that local businesses and community groups are also pushing districts to adopt early-warning systems as part of grants or partnerships.
“There’s a sense of urgency in these local communities to get these things moving quickly,” Mr. Bridgeland said. Businesses and foundations, in particular, “were frustrated in grantmaking [by] not getting sufficient feedback from school districts. They felt like they didn’t have the ability to understand the return on investment. ... [They] don’t want to wait for 10 years for a longitudinal study to show their investment is helping kids.”
The vast majority of the state and district early-warning systems have been implemented only in the past year or two—insufficient time to evaluate how well they are working overall—but the few that have been in place longer show promising results. For example, a review by the U.S. Department of Education’s What Works Clearinghouse found that the Check and Connect early-warning program, which targets middle and high school students with learning, emotional, and behavioral disabilities, reduced truancy and helped students stay in school.
The 34,000-student Minneapolis school district now uses the intervention in all seven of its high schools as well as its support program for teenage parents.
Colleen M. Kaibel, the director of the program for the district, said she and the district’s data officials analyze middle school attendance, behavior, and grade data for all incoming 9th graders. Those who were failing math or English, who had two or more suspensions, and who attended class 80 percent or less of the time get flagged. Mentors at the high schools greet those students and help them get oriented in the first days of school. After four weeks, if the flagged students haven’t improved in all indicators, they can be referred for weekly check-in sessions.
“We know those kids are not on track to graduate; they will not accrue enough credits to graduate in four years from a traditional high school, and each year they earn fewer credits, they become more disengaged from school, and it’s easier for them to drop out,” Ms. Christenson said.
At an annual cost of $847 per student, mentors meet with at-risk students weekly, helping them set goals, catch up on classes, and work with their families to attend class regularly. Once the student is passing all classes and has 95 percent or better attendance, he or she is moved to less-frequent monitoring, but the mentor still checks in to ensure the student stays on track.
This year, Minneapolis is analyzing the dropout data from its class of 2010 to identify early-warning signs of those students all the way back to grade 3, in an effort to implement Check and Connect interventions at earlier grades. It also has expanded the program to try to recover students who have already left school, identifying and acting on the indicators that former students as old as 21 might be ready to give school another try.
