U.S. students have seen unprecedented declines in math and reading performance since the pandemic on the National Assessment of Educational Progress.
Peggy Carr, the commissioner for the National Center for Education Statistics, which administers the assessment known as the Nation’s Report Card, said the NAEP system itself is changing to better understand how the pandemic has affected students and how they can regain their academic momentum.
“Everything has been in turmoil since COVID—it’s just complete chaos for almost any industry you can think of—and education really took the brunt of it,” she said. “People don’t realize what it means when your education system globally is not in place and it impacts everything else, so we have been busy as soon as we could get in the field collecting data.”
In addition to the 2024 NAEP, now in the field, NCES is launching the first post-pandemic cohorts for its long-term studies of early childhood (via the Early Childhood Longitudinal Study-Kindergarten) and the high school longitudinal study. Both collections track groups of students over years, gathering data on their academic progress and social development, as well as background information from parents and teachers. Carr spoke with Education Week about what her agency is learning from post-pandemic assessments.
This interview has been edited for length and clarity.
What surprised you in the NAEP results since the pandemic?
Carr: Many things that have emerged as impacts of the whole COVID situation we could have predicted: [poorer] mental health, bullying is up, all those things. But I don’t think I would’ve predicted that sustained chronic absenteeismis something we now need to start worrying about. [The U.S. Education Department defines chronic absenteeism as missing 15 or more school days in a year; NAEP also measures absenteeism by the number of students who have missed three days or more in the month before testing.] We have been asking about [chronic absenteeism] prior to the pandemic, so we have a gauge of what it was like prior to the pandemic, and now we can see what it’s like three years out, and it’s up.
And ironically, I wouldn’t have predicted this either, but we’ve found regardless of the depth of the decline [in student performance on NAEP]—whether it be a 16-point drop or a 5-point drop—it looks like somewhere between one and a half and two points of the drop [in average NAEP performance] can be associated with an increase in chronic absenteeism.
Every state saw at least some declines in the NAEP during COVID. But, why do you think some states declined so much more than others?
Carr: There’re so many other factors that seem to have influenced how states did … and not just whether they were in or out of school—like whether or not they had technology in place and the infrastructure to support instruction when things hit the fan.
We did an analysis of schools [in a forthcoming study with the American Institutes of Research] that we classified into different groups based upon what their health environment looked like when COVID hit—you know, death rates, teenage pregnancy, any health-related statistics that help you understand what kind of struggles a particular community is going through. We classified them based upon a healthier environment for the schools, or a less healthy environment for the schools.
And what we found out is that schools that already had these determinants that made their environment, their social and health environment, more challenging showed more declines—steeper declines—than those schools that were in environments that were physically healthier. In fact, those that had less-healthy environments showed more double-digit declines.
There has been a big increase in the number of students performing below the ‘basic’ achievement level in the on NAEP, a group we historically haven’t known as much about. What are you finding out about these students?
Carr: Actually, prior to COVID, we were seeing the bottom drop and the students at the top were holding their own or improving. There was this bifurcation with more students at the bottom falling steeper, with lower scores. And then after COVID, it was clearly exacerbated. So the problem that we already had now is a steeper, more troublesome problem.
The average score for Black students, for example, now is in that below-basic category. Right? We know what they don’t know and what they can’t do. But we don’t have a lot of [test] items to help us understand what they are able to do to be able to have something that’s more actionable.
How do you get a better picture of those lowest-performing students?
Carr: Back in the mid-to-late 2000s, No Child Left Behind required that Puerto Rico take the [NAEP] assessment too, in math, but we were struggling to get them on the scale because they were so low, there weren’t enough [test] items down there at the bottom. We created back in that time period, what they call KSA items—knowledge, skill, and ability items—in which we took out the [test] item distractors that really weren’t related to math. We took those out and those items were more accessible to the students in Puerto Rico.
But you know what, it wasn’t just Puerto Rico; it was Detroit and other low-performing districts that benefited from us putting those kinds of items on the assessment. So we’re using that as a model moving forward today to better populate the bottom of the scale, because there are so many kids down there.
The pandemic was the first time in NAEP’s history that it couldn’t be administered because of the shutdowns. How has that changed NAEP?
Carr: There is a concern that if something like this happens again, would we be able to test students from wherever we are? We actually already were planning online testing, but after COVID, it was clear we needed to speed it up. So what will happen in ’24 is we’ll have all of the content on devices, but we’re transitioning to the cloud as well so the content can be streamed down and streamed up [so that students could participate in the NAEP via whatever device schools have available, rather than using tablets NAEP brings in person]. That will make us far more flexible, if something like this happens again.
We’re going to transition from our [Surface tablet] devices, to use devices that are in the schools [like] Chromebooks, which can be found in 70 percent or more of schools. We’re going to do a bridge study in which comparable samples of students will take the assessment on Chromebooks, and others will take it with Surface Pros, which is what we’re doing now, and any differences, we’ll be able to statistically remove from our trends moving forward.
