As discussions about school accountability begin to focus more intently on factors beyond standardized-test scores, educators and policymakers nationwide are closely watching a group of California school districts—collectively known as the CORE districts—as they rethink how they evaluate school effectiveness.
In 2013, six districts received the only local-level waiver from the U.S. Department of Education when it bypassed California’s state education department to excuse those school systems from some requirements of the outdated No Child Left Behind Act.
In return for the federal flexibility, the districts—including Los Angeles Unified, San Francisco, and Fresno, some of the state’s largest—agreed to create a first-of-its-kind local accountability system that relies on a broad range of indicators in addition to traditional test scores to monitor schools.
Those indicators include suspension rates; school-climate survey responses from parents; and measures of traits related to students’ social development and engagement, like self-management and social awareness.
That work foreshadowed an inclusion of nonacademic factors as a part of school accountability in the Every Student Succeeds Act, a bill to reauthorize the Elementary and Secondary Education Act that was signed into law by President Barack Obama in December. Under ESSA, state accountability systems will be required to include at least one nonacademic indicator. The legislation lists educator engagement, student engagement, and school climate measures as examples, and it leaves the door open for others.
“We need to make sure that our schools’ quality is measured in a way that is much more reflective of the hard work that’s been done,” said Antwan Wilson, the superintendent of the Oakland district, which is also under the CORE waiver, along with Long Beach and Santa Ana. (Three other districts—Garden Grove, Sacramento, and Sanger Unified—are a part of CORE, but do not operate under the waiver.)
Broader Accountability
The accountability system, known as the school quality index, will be maintained by an organization that is governed by the districts’ leaders known as the California Office for Reforming Education, or CORE. (The Sacramento and Sanger Unified districts were originally approved to be a part of the waiver group, but later withdrew.)
Efforts to broaden accountability to include nonacademic factors come with some big questions.
At a time when some influential researchers have cautioned against using measures of noncognitive traits in school accountability, CORE officials are navigating uncharted territory in their ongoing work to determine what student traits should be used in their accountability system, how to measure those traits, and what to do with the results.
Their work could be informative to states as they seek to select what “other indicator” they will use to measure their schools as the federal government steps back from its previously prescriptive model for accountability and devolves more authority to the states in designing their systems.
The process of creating the CORE accountability system has also created excitement among education leaders who see it as a chance to move away from traditional accountability measures, which they say rely too heavily on standardized-test scores and tend to foster a culture of blame among educators rather than an atmosphere of support.
CORE pairs schools that fall to the bottom of the quality index and also rank lowest in academic achievement, known as priority schools, with demographically similar high-achieving schools. The aim is that principals and teachers trade techniques and strategies to improve.
The system also gives every school a “dashboard” of indicators it can monitor throughout the school year.
“The history of accountability systems has been about, ‘What is the thing I need to do to avoid the punishment?’ ” said Noah Bookman, CORE’s chief accountability officer. “We are trying to move the conversation to: ‘What am I learning? What strengths do I need to leverage? What are the challenges I need to address?’ and ‘What do I need to do that?’ ”
How It Works
Key to the CORE districts’ waiver was a promise to reduce the number of students required for a school to be held accountable for a given subpopulation—students from racial- and ethnic-minority groups, English-language learners, low-income students, and special education students—from 100 to 20. That means schools with smaller numbers of students from those populations will be held accountable for their performance for the first time under the new index.
The school-quality-improvement index bases 60 percent of a school’s score on academic factors. Under that category, schools will be judged by their proficiency rates and growth in proficiency rates on the Smarter Balanced test, aligned to the Common Core State Standards.
That yardstick includes the scores of all students combined, as well as the scores of each subpopulation: English-language learners, low-income students, students with disabilities, and students in the lowest-performing racial or ethnic subpopulation at that school.
High schools’ academic-category scores will also include how their four-year, five-year, and six-year cohort graduation rates compare with those of other high schools in CORE districts.
And middle schools’ academic scores will include a “high school readiness” rate, which is the number of 8th graders who meet criteria researchers have linked to a higher likelihood of high school graduation: a GPA of 2.5 or better, an attendance rate of at least 96 percent, no Ds or Fs in English/language arts or math in 8th grade, and no suspensions.
The remaining 40 percent of a school’s score will be a combination of factors related to school climate and students’ noncognitive skills, such as self-management. Current school-level report cards crafted under the index include three factors in this category: rates of chronic absenteeism, suspension and expulsion rates, and the rate at which English-language learners are redesignated as fluent. Beginning in 2016, two other factors will be added to that nonacademic domain: measures of students’ noncognitive skills and the results of student, staff, and parent surveys about school climate and safety.
Social-Emotional Factors
After consulting a growing body of research that links such skills to a higher likelihood of college and career success, the CORE districts identified four student traits to measure and track: social awareness; self-management; self-efficacy, which is the level of confidence an individual has in his or her ability to succeed and to “control ... their own motivation, behavior, and environment;" and growth mindset, which is defined as an understanding that academic skill is not an inherent, fixed trait but one that can grow through effort.
The districts plan to assess those traits through surveys that ask students questions like how frequently they come to class prepared and how much they agree or disagree with statements such as: “Challenging myself won’t make me any smarter” and “I can earn an A in all of my classes.”
CORE districts piloted a longer menu of survey questions with 450,000 students. After refining that list, CORE will conduct surveys at every school this year.
Leaders of the districts say including such factors in the index will help track the effectiveness of social-emotional-learning programs and similar interventions already in place in many of their schools and help them call out good work that may previously have gone unrecognized.
Oakland, for example, is one of eight urban districts piloting a comprehensive social-emotional-learning program in consultation with the Collaborative for Academic, Social, and Emotional Learning, a Chicago-based organization that promotes research, policy, and practice related to such strategies. Other districts have made efforts on a school-by-school basis.
Among those schools is Beachy Avenue Elementary, which is part of the Los Angeles Unified School District. It uses the Second Step social-emotional-learning program to teach regular lessons in subjects such as empathy and anger management alongside its core subjects.
In one such lesson this fall, third-year teacher Cocoro Morimoto showed a video of two young boys describing their interactions at a sleepover—one had a great time, and the other lamented that his friend left his room a mess—and asked her 4th grade class to identify and relate to each boy’s perspective. In the future, she asked, how could the boys handle their interactions differently?
“Empathy!” several students yelled as they remembered the concept all at once.
Such lessons have a significant impact on student’s academic success, Morimoto said in an interview. Before she used the curriculum, she lost valuable instructional time addressing student conflicts that erupted during recess.
“I think they feel more empowered when they are solving problems on their own,” she said, adding that social confidence helps boost students’ confidence with challenging academic content. “I am giving them words for their feelings.”
Even without such measures, Beachy Principal Stephen Bluestein said he knows the social-emotional lessons are working. Since adopting the approach, the school’s office referrals have dropped and individual students’ academic performance has improved, he said, although the school still performs below district averages academically.
“It has indirect and direct positive effects that reflect in reading, writing, and arithmetic,” he said.
Researcher Cautions
But will the CORE districts’ noncognitive measures detect the changed attitudes of students?
Researchers of that topic have warned that those measures should not be used for school accountability because they are subject to biases and flawed responses. For example, in some schools in other states, students who’ve been taught about issues like self-control rate themselves lower than their peers because they have a greater awareness of what those concepts mean.
Currently, “perfectly unbiased, unfakeable, and error-free measures are an ideal, not a reality,” researchers Angela Duckworth and David Yeager said in a May essay published in Educational Researcher that detailed an array of flaws with current measures.
Influential Stanford University psychology Professor Carol Dweck, who popularized the idea of growth mindset, said she doesn’t support the inclusion of that measure in the CORE districts’ system.
“To use the mindset measures directly for accountability is, I think, asking for problems,” Dweck said. Such practices may lead to shallow interventions that don’t actually improve student learning, she said, and they could indirectly encourage teachers “to be teaching their students how to check off the right box on a questionnaire.”
“I’m deeply pleased that they value the hard work that we’ve done, but then, that said, I have these concerns,” Dweck said.
Researchers with Transforming Education, a Boston-based organization that consulted with CORE on its social-emotional measures, expressed some concerns about bias in student responses, Bookman, the CORE accountability chief, said. But preliminary results show that strong student scores in noncognitive areas correlate with stronger academic performance and observable traits, like regularly turning in homework on time, which lends validity to the results, he added.
Sara Bartolino Krachman, the executive director of Transforming Education, said the questions used on student surveys passed several levels of pilot tests without showing signs of the biases researchers expected.
And CORE Executive Director Rick Miller said traits like self-efficacy and growth mindset are far too important for student success to delay measuring them.
Some of the same researchers who’ve criticized gauging such traits for accountability are actively working to develop more consistent, reliable measurements. Harvard Graduate School of Education Professor Martin West, for example, is working in Boston-area charter schools to devise new tools to measure student traits. Miller said CORE is open to changing its measures over time as new, more-sophisticated tools are introduced.
And, as a new federal accountability law leads to the end of the district-level waiver, the CORE districts intend to continue developing their model for comparing schools, whether or not it is tied to high-stakes federal accountability, he said.
