San Antonio, Texas
As states and districts work to develop new accountability systems under the Every Student Succeeds Act, six California districts who received federal waivers under the Obama administration are getting the first hints of how more holistic accountability systems might work.
Researchers and district officials discussed the first evaluations of the California Office to Reform Education, or CORE, consortium of districts at the American Association of Educational Research conference here this weekend.
“We’re really thinking about this as an experiment in, how do we hold ourselves accountable in a way that can move us away from the strict focus on compliance that other systems of accountability have moved us into?” said Dave Calhoun, executive director of research, assessment, and evaluation at Fresno’s unified school district, which belongs to CORE. “We’re very excited to see what accountability looks like when it’s focused on support instead of sanctions—but this is still a work in progress.”
Fresno, Long Beach, Los Angeles, Oakland, Santa Ana, and San Francisco base 60 percent of their school accountability on students’ academic status and growth, as well as graduation rates at the high school level and an indicator of “high school readiness” for middle schools. However, 40 percent of schools’ accountability is based on social and emotional indicators, including rates of chronic absenteeism, suspension, and expulsions, and surveys of school culture and climate and students’ reports of their own social and emotional growth.
The districts developed surveys for the student self-reports to be “measurable in less than 20 minutes, meaningful to academic and life outcomes, and malleable through school-based interventions,” according to Martin West of Harvard University’s graduate school of education. West and his colleagues analyzed administrative data from more than 250,000 CORE district students in grades 3 through 12, from 2013-14 to 2015-16.
The researchers found students’ reported self-management skills and growth mindset were the best predictors of students’ later reading and math performance; a higher sense of self-efficacy was associated with higher test scores for white and Asian students, but not for black or Hispanic students. Moreover, West said, “Girls consistently rate higher in self management and social awareness, have a sharp decline in self-efficacy from grades 3 to 12, both absolute terms and relative to boys—but that is not translated to reduced test scores compared to boys.”
However, West cautioned that unlike academic skills, in which students typically grow by a steady amount each year, he found the social-emotional skills all showed a sharp decline for students at the beginning of middle school, followed by partial recovery by the middle of high school. That variation may make it more difficult to evaluate how students can be expected to develop in these skills over time.
In a separate study, Heather Hough, Demetra Kalogrides, and Susanna Loeb of Stanford found 5 percent of the differences in schools’ math growth in elementary school and 6 percent of the differences in math growth in middle schools, as well as 11 percent of the differences in high schools’ graduation rates, could be explained by differences in their school climate and student-reported social skills. That was the case even after controlling for school demographics and quality indicators, like teacher quality. Combining the results of the student social-skills surveys and school climate surveys accounted for 21 percent of the difference in math scores for the lowest-performing 5 percent of low-performing schools.
Interestingly, Hough found teachers and parents typically agreed on the climate of a school, but students’ responses often differed.
“Students experience the school very differently depending on how they are treated and who they associate with. The adults experienced not necessarily the right view of school climate, but the same view of school climate,” Hough said. “We have to dig into what this means. If parents think the school culture or climate is bad but students don’t, what does that mean for how we improve the school?”
Partnering for Growth
The CORE districts together represent about 20 percent of the state’s students, or more than 1 million K-12 students.
“We found over and over again this echoed story about why the decision happened” to create a new accountability system, said Taylor Allbright, also of USC Rossier. “Educators struggled under NCLB, believing in the importance of [social-emotional learning] but lacking evidence or policy support. CORE districts created multiple measures, and believe the accountability system will lead to equity.”
With that in mind, the CORE districts matched their lowest-performing 5 percent of schools with high-performing schools that had had high growth with similar students. Allbright and Julie Marsh of the University of Southern California Rossier School of Education, and Heather Hough of Stanford University interviewed principals at 15 of the matched schools, conducted more than 40 hours of classroom observations, and reviewed administrative data, meeting minutes, and reports. They found the districts generally bought into the new accountability system, but tended to tweak it significantly for their own schools, and the partnered schools were often confused about how to work with each other.
“Achieving reciprocity of district and school peer-to-peer interactions was challenging,” said Marsh, co-director of Policy Analysis for California Education and associate education professor at USC-Rossier. “Some were questioning what they could learn from other schools and districts considering their different performance and contexts.”
The schools are still working on those partnerships, but the future of their accountability systems is still uncertain; the waiver authority ended with the passage of ESSA.
Related: