Pandemic disruptions have both increased the need for schools to find faster and better ways to support students’ learning and reduced their capacity to do so.
Continuous improvement—a process by which groups of people or institutions set up feedback loops to identify problems, quickly test solutions, and evaluate their effectiveness—has long been used in manufacturing and health care but more recently gained traction in education when many states adopted the process as part of their plans to implement the Every Student Succeeds Act.
Education Week talked with Robert Balfanz, a research professor at the Center for the Social Organization of Schools at Johns Hopkins University School of Education and a co-author (with associate professor Martha Abele Mac Iver) of the new book Continuous Improvement in High Schools, to look at how the process may help schools adapt more rapidly to the challenges of the last two years.
This interview has been edited for length and clarity.
Why should educators focus on variation when they are trying to solve problems of practice?
One of these really challenging problems [in translating education research to practice] is we tend to reflectively go to the average or the general finding, right? Average test scores or average daily attendance is that, or overall graduation rates are this. Those are all numbers made up of different components, and oftentimes, most of the story is in the components, not the average.
One of the classic examples was, until maybe a decade ago, the only way people looked at attendance was average daily attendance ... how many of our total kids were in the building on a given day. And, you know, except in rare circumstances, almost every school was, on that measure, falling up in the low 90s. [Principals said] I’m sure I could do better, but my literacy rates are in the 30s, and so clearly, I have to focus on literacy and not so much on attendance. But the truth was if you broke down that average, ... you can have average daily attendance in the low 90s and still have 20 percent of your kids missing a month or more school, like different kids on different days. ... Then if you look, that highly overlaps with the kids that are struggling in literacy in this school. So commonly, they would invest energy in a new literacy program and they would get very modest results. And they were like, oh, is that really worth all that effort? ... And often, they were not checking and so didn’t know that half the kids they aimed that intervention for didn’t come to school regularly enough to get it.
The pandemic has caused a lot of disruptions in the data schools use to track early-warning indicators and measure progress. Can education leaders still reliably use data to guide their schools’ improvement?
Yeah, it’s really done a number on many of our traditional metrics. It’s going to be a while before there’s a baseline of achievement data that people can have confidence in, and attendance became very problematic because, how do you measure virtual attendance? ... So one thing it’s done is make people think about what are we really trying to measure with something like attendance? I think it’s a combination of things like engagement, being present, and were you able to actually get your work and assignments done, regardless of where and when you did it. “Attendance” is a proxy for a lot of things that in a traditional setting are easy to measure this way. ... But there are other, very local ways you can get a sense of engagement like [student surveys and assignments completed]. If you don’t have normal attendance data, what data can you collect relatively easily that will still address the underlying issue?
Most schools’ systems of support use a structure in which a general education works for the large majority of students, with smaller groups of students needing increasingly intensive interventions. But right now, there are large swaths of students who have fallen behind academically in multiple subjects. How does that change how educators should think about interventions?
Yes, I saw some recent data from California that suggested it was the kids in the middle of the bell curve that are struggling much more, ... and that increases the total number of kids struggling. Another benefit of a continuous-improvement approach [is] it lets you be adaptive to the situation. So, say I depend on this bell curve for a tiered support system, but ... we can’t individually intervene our way out of half or more of our kids [academically struggling]. So that tells us we need to look at classroom-level activities with help with school-level or grade-level interventions. Instead of just grinding out a strategy that’s not going to work, [the continuous-improvement process] gives you a framework to shift your strategy and figure out what will work now.
The “Plan-Do-Study-Act” cycle includes time for educators to identify potential causes of a problem, research, and experiment with potential solutions. How many different solutions can a school typically test in a given year?
That does get to some of the real-world limitations with this. In general, in our experiences, schools try out two different ideas at once, to find if A or B worked better. And they could maybe do three cycles a year.
In theory, [school improvement] networks should help you because if we all collectively retry and the network [schools] are similar enough, then what works for you should largely work for me. I would say again, in the real world, that probably works half the time, because schools are just hard-wired at some levels that ... they can’t take it at face value that if something worked over there, I can adopt it and have confidence it’ll work. ... There’s some resistance still built into saying, no, we all can try this. They got the good results, so let me just give it a go.
Through necessity, a lot of schools have been experimenting with structural and format changes in the last few years, particularly in high schools. Are there any innovations that you think are the likeliest to stick in high schools?
I think what’s interesting is that during COVID, at the high school level in particular, students spend a lot more of their time with asynchronous learning. They typically had Zoom classrooms only for a couple hours a day, while elementary kids would have all-day Zoom. In high school, they were given assignments to do the rest of the time. And, you know, for some kids that failed spectacularly—but for other kids, they ended up liking it better to be able to do their work in their time and also, say, working to help their family.
And in many ways that was much more a collegelike experience. It has struck the schools as well, since, as you know, there’s been this big push for decades to really make sure the high schools are preparing you for college. ... And perhaps the best way to do that is to actually make high school closer to college.
So I know a number of our schools are now experimenting with how could we keep elements of that, where kids are in classrooms some of the time, but other times they could work in groups, they can work with a supportive adult, and it captures a little bit of that more collegelike experience while they’re still in high school.
