Across the board, struggling American students are falling behind, while top performers are rising higher on the test dubbed the “Nation’s Report Card.”
A nationally representative group of nearly 585,000 4th- and 8th-graders took the National Assessment of Educational Progress in 2017, the first time the tests were administered digitally. The results, released Tuesday, show no change at all for 4th grade in either subject or for 8th graders in math since the tests were last given in 2015. Eighth graders on average made only a 1-point gain in reading, to 267 on the NAEP’s 500-point scale.
That meager gain in reading was driven entirely by the top 25 percent of students. During the last decade, 8th grade reading was the only test in which the average score for both high and low performers rose. By contrast, in math, the percentage of students performing below basic (30 percent) and those performing at the advanced level (10 percent) both increased significantly since 2007. The same pattern emerged in 4th grade math and reading.
Those changes were all statistically significant, and they point to what NCES Associate Commissioner for Assessment Peggy Carr called a “bifurcation” of student performance. American students’ performance on international assessments such as the Progress in International Reading Literacy Study and the Trends in International Math and Science Study show the same spreading gaps.
“That’s disappointing and concerning—not that high achievers are going up, but that low-achieving students are declining,” said Scott Norton, deputy executive director of the Council of Chief State School Officers. “I think regardless of the reasons for that gap increase ... all of our members are concerned about that and want to look at it more closely.”
Achievement gaps remained stubbornly wide for particular student groups, too. In grade 4 math, the average achievement fell by four scale points for students with disabilities, two points for urban students, and one point for students in poverty. No other grade or subject showed changes for individual student groups.
Over the past decade, the results for students with special needs have been grimmer. Students with disabilities nationwide had an average scale score of 214 out of 500 in 4th grade reading in 2017, right at the cutoff for NAEP’s “basic” level of performance. That’s the lowest average performance for this group since 2003. At 8th grade, students with disabilities had an average score of 247, about the same as a decade ago, and down from a high of 250 in 2011. Similarly, 8th grade English-language learners have not improved significantly in reading since 2003.
Few Changes for States
The results are coming under intense scrutiny from states, both because of the change in testing format from paper and pencil to computers and because they are being released at a time when many states are transitioning to new assessment systems under the Every Student Succeeds Act.
"[NAEP] is our only really solid national trendline that we have nowadays,” Norton said, “so tracking that is very important.”
Average scores for most states remained unchanged from 2015 in either grade or subject, and more states saw declines than improvements. Florida was the only state to improve on average in both 4th and 8th grade math and 8th grade reading from 2015.
Puerto Rico saw gains in math at grade 4, while schools run by the Department of Defense also saw reading improvements at 8th grade. In reading, no states improved at 4th grade, but nine states—California, Florida, Georgia, Hawaii, Indiana, Massachusetts, Mississippi, New Jersey, and Washington—and the Defense Department schools improved at 8th grade.
Of the 27 large urban districts that also participated in a targeted NAEP study in 2017, none declined in reading, and a few—San Diego’s 4th graders and 8th graders in Albuquerque and Boston— improved over their 2015 scores.
“Looking more closely at the results indicates that instructional changes and professional learning support are paying off in some districts. However, more professional support is needed for teachers in order to see more broad-based benefits nationwide,” said Matt Larson, the president of the National Council of Teachers of Mathematics, in a statement on the results. “We need to ensure that we are providing each and every student with access to high-quality mathematics curriculum, instruction, and expectations.”
Digital Transition Causes Waves
About 80 percent of students in 2017 took the NAEP on digital devices, while 20 percent used traditional pencil-and-paper tests. The National Center for Education Statistics compared students’ answers to the same questions in digital and print tests both in 2017 and 2015 to link the two formats and account for so-called “mode effects” in the 2017 results.
State schools chiefs, including John White of Louisiana, voiced concern that states whose students had less experience taking computer-based tests would be at a disadvantage; Louisiana was among the states with the biggest declines in 4th grade math and reading. Both White and the Council of Chief State School Officers sent NCES requests for more information on states’ mode effects, and CCSSO’s Norton said states are reviewing those data.
A technical analysis by the Johns Hopkins Institute for Education Policy found that states that had not previously used online testing with their students showed larger average declines between 2015 and 2017 in grade 4 reading and math and grade 8 reading than states that had used online testing before. The study estimated these differences could account for 15 percent of differences in test score changes for 4th grade reading, and 11 percent of changes in 4th grade math, though the report noted those findings were preliminary.
Both NCES Commissioner James Woodward and Associate Commissioner Peggy Carr said they were confident in the results, which had been adjusted for the national mode effect. “We’re going to learn a lot more about what students know and can do—not just students’ answers, but how they arrived at those answers—through the digital platform.”
Other national and international tests have also seen differences in student performance when they moved from print to digital formats.
NCES has been studying the transition to digital testing for more than 15 years, according to Sean P. “Jack” Buckley, a former NCES commissioner and now senior vice president for research and evaluation at the American Institutes for Research. “I know it’s something NCES agonized over. ... It’s not overblown; it’s a real area of concern,” he said, noting that the NAEP’s writing assessment found the difference in students’ digital writing performance warranted starting a new trend line for the study.
Yet the Council of Chief State School Officers also noted that most states use computer-based assessments for their own tests, and students’ familiarity with digital devices in and outside of school has dramatically increased, even in the last five years. “At some point paper and pencil become archaic,” Buckley said. “There’s an ability to test constructs [digitally] that are more relevant to what students are actually doing. Sometimes you just can’t avoid the transition.”