“The nation’s report card” was released today, reporting the 2022 results for reading and math in grades 4 and 8. While I could ramble on about the National Assessment of Educational Progress’ latest outcome, I was far more interested in hearing what Marty West made of them. Academic dean for Harvard’s Graduate School of Education, research fellow with the National Bureau of Economic Research, editor-in-chief of Education Next, member of the Massachusetts board of education, and member of the National Assessment Governing Board (which oversees NAEP), West has as good a sense of these results as anyone in the nation. Here’s what he had to say.
—Rick
Rick: What’s the bottom line here? Are these results better or worse than expected?
Marty: The bottom line is that 4th and 8th grade reading and math NAEP scores are down from 2019 levels nationwide. They are down substantially. And they are down nearly everywhere: Every state (and the District of Columbia) saw scores drop by a statistically significant amount on at least one of the four tests administered this spring. The same is true for each of the 26 large urban school districts that participate in NAEP. And in those states or districts whose NAEP scores in a particular grade and subject were officially unchanged from 2019, scores were typically lower than they had been that year—just not by enough to achieve statistical significance. It is fair to say based on these results that there are very few school systems nationwide where students didn’t lose considerable ground over the course of the pandemic.
We certainly expected 4th grade scores to be down based on the long-term trend NAEP results for 9-year-olds we released in September. This release revealed that the declines were larger and more pervasive in 8th grade math, where scores were down in 49 of 50 states (all but Utah) and in DC. That was the bad news.
As for good news, there were certainly some systems that seem to have weathered the pandemic better than others. At the state level, Alabama, Alaska, Hawaii, Illinois, Louisiana, and Iowa stand out as places where scores fell less on average across grades and subject areas than others. Among very large districts, Dallas, Miami-Dade, New York City, and especially Los Angeles fared surprisingly well. Catholic schools nationwide saw student scores hold steady on three of the four tests, though it is possible that this reflects shifts in enrollment into those schools in response to public school closures. And the schools operated by the Department of Defense Education Activity actually registered gains since 2019 in 8th grade reading and matched their prior performance on the other three tests. I expect we will see many efforts in the coming weeks to explain those outliers.
Rick: We’ve seen a number of troubling test results over the past couple months: September’s long-term trend NAEP results were dismal, ACT recently reported its worst results in 30 years, state assessments have been grim. What do these NAEP numbers tell us that we didn’t already know?
Marty: You’re right that we have multiple sources of evidence that achievement fell during the pandemic, but they all have limitations. Not all students take college-entrance exams like the ACT, which makes it tricky to interpret changes in scores over time—especially during a pandemic. State tests differ from one state to the next, which makes it hard to compare both achievement and changes in achievement over time. Only NAEP can speak with authority about what has happened to student achievement nationwide and how that varies across states.
Rick: Parents and educators sometimes worry that there’s too much testing and may wonder why we need NAEP in addition to all those other assessments. Why should readers pay attention to NAEP, in particular? For those who aren’t familiar with it, what’s distinctive about it?
Marty: The NAEP program provides data that parents, educators, and policymakers need to put results from other tests and other measures of student performance into context. It is the only test that is routinely taken by a nationally representative sample of students and by representative samples of students in each state. It is this feature that makes it possible to understand how student achievement differs across all 50 states and, through the creative work by researchers at The Educational Opportunity Project at Stanford University, across school districts nationwide.
A second feature of NAEP is that it is a no-stakes test. Except perhaps for governors or chief state school officers, no one’s evaluation or performance rating hinges on the results. In fact, NAEP results aren’t even available for individual schools and students. This means that we don’t have to worry as much that score gains could reflect the use of crude test-prep strategies or an over-emphasis on the test content as opposed to other education goals—a common concern when it comes to state test results.
One final unique feature of the test that parents may appreciate is that no one student takes the entire exam. We instead use an approach called matrix sampling, where several students each take one part of it. This is designed to minimize the burden placed on schools and students selected to participate.
Rick: Can looking at the results tell us more than we already know about which kids have been hit hardest by the pandemic? Are there any surprises here?
Marty: One troubling pattern in the long-term trend NAEP results released for nine-year-olds in September was that lower-performing students seemed to have lost the most ground. That is, the achievement of students at the 10th and 25th percentiles of the national distribution fell considerably more than that of students at the 75th and 90th percentiles. The gaps between higher- and lower-performing students on various NAEP tests had already been growing throughout the decade leading up to the pandemic, and the pandemic seems to have accelerated that trend.
Looking at the results out today, we see this same pattern in both math and reading at grade 4—which is obviously cause for concern. The Black-white and Hispanic-white achievement gaps also grew in 4th grade math, which is a reversal of a long-term trend toward greater racial equity in NAEP results.
What is more surprising is the fact that 8th grade math scores fell dramatically—and by roughly the same amount—for both higher- and lower-performing students. This is the main reason why the change in average scores was largest on this test.
Rick: OK, but how much faith should we put in these results? A number of superintendents and a few state chiefs have told me that, during the pandemic, kids got out of the routines of test prep and test-taking—and that’s what we’re seeing. Do you think that’s a significant issue?
Marty: The fact that NAEP is a no-stakes test means that it is not a main focus of test-prep efforts in schools, even in ordinary times. The test prep that I worry most about when interpreting scores is activity focused on the unique features of a specific testing program—for example, coaching students on topics that are always included or questions that tend to be asked the same way every year. This is generally not possible on the NAEP, and educators don’t have an incentive to try to prepare students for the test. If students are out of practice when it comes to answering the types of questions posed by NAEP, I’d think that would also affect their performance in their classes and ability to make progress over time. For that reason, I’m not sure that I could separate a decline in NAEP performance from learning loss generally.
That said, the theory you cite is testable in the sense that, if it is correct, we’d expect a rapid recovery from this year’s losses by the time the main NAEP, which measures reading and math performance every two years, is next given in 2024. If that happens—and I hope it does—renewed familiarity and comfort with test-taking could be part of the explanation. But I’m far from ready to dismiss this year’s results based on that concern.
Rick: You co-direct the annual Education Next survey on education. One of the things the survey has historically looked at is public views toward testing. Where is public sentiment on testing right now, and do you think there’s anything in these results, and in other grim assessment outcomes, that’s likely to move the public’s views?
Marty: What’s most striking to me is the public’s continued strong support for the federal requirement that states test students annually in grades 3-8 over the course of the pandemic. Roughly three of four adults favor that policy, which suggests that there hasn’t been a groundswell of anti-testing sentiment among the mass public. That pattern doesn’t mean that the public favors all uses of tests, of course, but Americans do still seem to see tests as a valuable source of information. I don’t see anything in these latest results that would change that.
In fact, my hope is that these results can play a role in shaping parents’ views of where their children are academically, as we emerge from the pandemic. When we at Education Next surveyed parents in May 2022, the parents of only 9 percent of students said they were not confident their child will “catch up” from Covid-related learning loss within a year or two; the parents of the rest either are confident the child will catch up or perceive no learning loss in the first place. Today’s results suggest that parents are far too optimistic—and I hope the results spur some parents to take a closer look at their children’s academic progress over the course of the pandemic and what may be needed for them to catch up.
This interview has been edited and condensed for clarity.
