A portion of the Every Student Succeeds Act that requires states to incorporate nonacademic factors into their accountability systems could help promote a broader vision of school success that extends beyond traditional measures, such as standardized-test scores, educators and policy watchers say.
The newest version of the Elementary and Secondary Education Act, signed into law last month, requires states to use at least one such indicator—like measures of student engagement or access to advanced coursework—in tracking schools’ success.
If states select the right indicators and the right methods of measuring and tracking them, the changes could provide schools with a more nuanced view of student success and equity, supporters said.
“Since we know that what gets measured gets delivered, … if we broaden accountability systems beyond test scores and schools are required to look at these other things, then the public and parents will start looking at them, and there’s a better chance that schools will start providing these kinds of things,” said Megan Wolfe, advocacy manager for ASCD, an organization that has pushed for so-called “multimetric accountability.”
But as states put a largely untested policy idea into practice on such a large scale, implementation is everything. If states select indicators that can’t be accurately measured or influenced by schools, or if they fail to provide schools with the resources they need to carry out new mandates, the indicator requirement could lead to unintended consequences or pushback from educators, K-12 groups and researchers have warned.
And educator groups said there’s a potential land rush building behind the scenes among advocacy groups that are eager to help reshape educational accountability in their image.
‘Meaningful Differentiation’
Specifically, the new law requires states to use at least one “indicator of school quality or student success” that “allows for meaningful differentiation in school performance” and “is valid, reliable, comparable, and statewide,” alongside academic data in their accountability systems. Schools must also be able to disaggregate data related to that indicator to show how it affects students in different subpopulations: those from all racial and ethnic groups, students with disabilities, children from low-income families, and English-language learners.
The law lists a few examples of possible indicators that states may consider—student engagement, educator engagement, student access to and completion of advanced coursework, postsecondary readiness, and school climate and safety—but it also leaves the door open to other indicators of each state’s choice.
Some states and districts already track those sorts of nonacademic factors on “data dashboards” used to improve school quality, but they are rarely tied to accountability systems that could potentially penalize schools that don’t perform well.
That’s in part because many of those factors are difficult to measure in a reliable and objective manner.
For example, teachers and administrators may disagree on what factors schools should consider when gauging postsecondary readiness. It may be difficult to disaggregate some factors, like school safety and teacher engagement, by student subpopulation. And surveys that measure such factors as student engagement could be “faked” if teachers coach students on their responses, researchers warn, adding that attaching such measures to high-stakes accountability only increases the likelihood of manipulation.
“The basic principle is that the minute an indicator has a stake applied to it, it becomes a problematic indicator,” said David Osher, a vice president of the American Institutes for Research, who studies social-emotional learning and school climate issues.
Quality of Measurement
Concerns about quality of measurement are particularly relevant if states consider incorporating factors like students’ noncognitive traits and social-emotional skills into their school quality scales. Researchers like Angela Duckworth, a University of Pennsylvania psychology professor who popularized the idea of “grit,” have warned that existing measures of such factors are prone to flaws and biases and that they should not be used for high-stakes accountability.
Osher said he’s more confident in measures of factors like school climate because, when asking students whether they feel safe, supported, and challenged at school, their perception is reality. In social-emotional learning, on the other hand, student surveys may be less reliable because students may have inaccurate perceptions about their own skills, he said.
Osher helped districts including Chicago, then under the direction of former U.S. Education Secretary Arne Duncan, and Cleveland develop “conditions for learning” surveys that track those conditions in schools, though the districts haven’t used them for accountability.
Whatever measures states use, they should be mindful that it takes time for schools to change learning environments and to devise strategies to nurture students’ relational and personal skills, Osher said. Accountability systems that are too punitive will encourage a condition of “churn,” in which schools quickly abandon strategies and programs before they have the chance to take root, he said.
“I think the important thing is: How do we help people do this well in a way that produces the types of experiences for students in classrooms and in schools that enhances their learning and development?” Osher said.
Even those who are enthusiastic about the new law acknowledge that some possible “other indicators” are easier to measure accurately than others.
Members of AASA, the School Superintendents Association, are interested in many of the factors tracked by California’s CORE districts, a group of districts that developed a multimetric accountability system that includes discipline data, social-emotional surveys, and school climate measures, said Noelle Ellerson, the AASA’s associate executive director.
But not all indicators are created equal, and the success of states’ new accountability systems will depend on what they choose, Ellerson said in a forum at the Thomas B. Fordham Institute last month.
Avoiding Unintended Consequences
State school chiefs are in the initial phases of reviewing the nonacademic data they already collect and monitoring how the U.S. Department of Education drafts regulations under the new law as they determine how to broaden their accountability systems, said Kirsten Taylor Carr, the program director for accountability at the Council of Chief State School Officers. Then they will likely explore a number of indicators, deciding which can provide a better view of how different student groups are faring in their schools, she said.
“The underlying goal is ensuring equity,” Carr said. “No matter what the measure is, we’ll always be thinking about does this help us achieve that goal of equity.”
ASCD has stopped short of saying which nonacademic indicators it recommends. It has convened an advisory group of assessment and accountability experts, state decisionmakers, local practitioners, and national organizations to make recommendations, David Griffith, the senior director of government relations, said.
Frustrations with the way the No Child Left Behind law, the previous version of the Elementary and Secondary Education Act, measured school success are “very instructive here,” Griffith said.
“We are even more so, very on guard as to creating unintended consequences,” he said. “And there’s good data being collected, but when you start to attach consequences to it, that can change the reporting of it.”
For example, schools could skew data on attendance by tracking it at days and times when more students are typically in school, or they could make it look as if they’ve lowered discipline rates by replacing formal suspensions with more informal classroom removals that wouldn’t be tracked.
Still others have worried that attaching stakes to high suspension rates will cause some schools to ignore poor behavior they would otherwise discipline, creating a distraction-filled learning environment for other students.
But that cautious tone hasn’t dampened the enthusiasm of “whole child” advocates who’ve been pushing for more expansive data collection and tracking for years.
Hours after the Every Student Succeeds Act was signed into law, the Dignity in Schools Campaign, a coalition of civil rights and student groups, issued a press release urging states to select school climate and safety as their “other indicator” under the law.
