Includes updates and/or revisions.
When 4th grade mathematics scores on the National Assessment of Educational Progress seemed to stall between 2007 and 2009 after years of steadily climbing, some experts pronounced the results disappointing.
But a report released March 17 suggests that the lack of continued progress may have been a necessary correction after a long and possibly unrealistic trajectory, rather than a cause for discouragement.
“The main NAEP has always been an outlier in terms of how much progress it’s measuring,” said Tom Loveless, the author of the report and a senior fellow at the Brookings Institution, a Washington think tank. “So, in a sense, it’s coming back to earth now.”
The review of NAEP trends is among several analyses included in the latest annual report on American education produced by Brookings’ Brown Center on Education. A second study, looking at 20 years of state testing data on more than 1,000 California schools, finds that very few of them ever “turn around” to become a whole lot better in terms of students’ academic performance, nor did they get a whole lot worse. A third report, also drawing on data from California, concludes that public schools that convert to charter status tend to look more like regular schools in their demographics and teaching staffs than do charter schools started from scratch.
Related story:
To put the NAEP results from last year in context, Mr. Loveless compared 4th graders’ academic growth since 1990 with the long-running trend lines for 4th graders on two other tests: the Trends in International Mathematics and Science Study, or TIMSS, and the Long-Term Trend NAEP. The latter, also known as the NAEP-LTT, is a separate test given to the same age group of students every few years.
Of the three tests, Mr. Loveless writes, the main NAEP test has consistently registered the biggest gains from one testing year to the next. If the same trend were to show up, for instance, in the TIMSS results for 2011, U.S. 4th graders would be scoring on par with Hong Kong and Singapore, which were both the world’s top-performing nations in 2007. Likewise, by 2053, 4th graders would know as much math as high school seniors did in 1990, according to the report.
“Perhaps the skyrocketing gains had to stall on this test, and elementary math teachers did not suddenly become horrible in 2007,” the report says. “If you go by the main NAEP, don’t forget, they had been miracle workers the previous 17 years.”
‘Boosting the Bottom’
Mark S. Schneider, a former commissioner of education statistics at the U.S. Department of Education and one of the experts that Mr. Loveless chides in his report for expressing disappointment at the last round of NAEP math scores, said the new analysis doesn’t change his perspective on the stagnating scores.
“Yes, it is true that we did well in the last 20 years since 1990,” said Mr. Schneider, who is now a vice president at the American Institutes for Research, a nonprofit organization based in Washington. “What I would hope is that the U.S. would continue to make improvements in math so that we end up where we should be, as one of the highest-performing countries in the world.”
4th Grade/Age 9 Math Achievement
SOURCE: Brown Center on Education
In his report, Mr. Loveless suggests the test trends differ because the three tests he examined measure different content. While TIMSS and the NAEP-LTT have more questions on percentages, fractions, decimals, and computation, for instance, the main NAEP test includes more questions requiring students to complete number patterns.
A more pervasive trend in student achievement on the long-term NAEP, Mr. Loveless concludes, may be the shrinking of the gap between the lowest- and highest-scoring students that occurred after 1998—for both 9- and 13-year-olds in reading and math.
“Something was going on with NAEP,” Mr. Loveless said in an interview, “and I think it’s the accountability movement that started in the late ’90s and focused on boosting the bottom. All boats were rising, but the bottom was rising faster.”
Bolstering that idea, Mr. Loveless notes that NAEP scores for private schools over the same period did not show a similar contraction between the top-achieving 10 percent of students and the bottom 10 percent.
Mr. Loveless said his second study highlights the challenge that U.S. Secretary of Education Arne Duncan faces in his efforts to turn around the bottom 5 percent of the nation’s failing schools.
Of the more than 1,000 California middle and elementary schools that Mr. Loveless studied, most scored at more or less the same achievement levels in 2009 as they did in 1989, a period of time when the state was undertaking a variety of education initiatives aimed at boosting students’ test scores. Those included new forms of accountability and state testing, new methods of financing schools, charter schools, and state takeovers of failing schools.
Of the 115 schools scoring at the 10th percentile or below in 1989, for example, only four were scoring at the state average by 2009, the report says.
More top-to-bottom movement occurred over the same period, Mr. Loveless says, among professional teams in baseball, basketball, and football. Of the seven professional basketball teams that were in the lowest quartile in 1989, for instance, only two were still hugging the bottom 20 years later.
Also, he says, while the socioeconomic status of a school’s student population was linked to its achievement, demographic changes did not completely explain the academic-performance changes in his sample of schools.
“If failing schools are ever to be turned around, much more must be learned about how schools age as institutions—how they got to where they are and the factors influencing where they are going,” he writes in the report.
A Different Breed
For his analysis of charter schools, Mr. Loveless examined data from 1986 and 2004 for California, where an estimated 16 percent of charter schools were converted from regular public schools. As was the case with the state’s regular public schools, student performance at the conversion charters did not differ much in 2004 from performance in 1986, when many of the schools were regular public schools, according to the report.
Compared with the charters in that state that had started from scratch, however, the conversion schools are two or three times as large and more likely to be located in urban communities, and they serve a larger proportion of black and Hispanic students. The teachers in the conversion schools are more experienced and more likely to be certified, according to the report.
“When people talk about charters, they often don’t differentiate between charters and conversions, and there are some real differences there,” Mr. Loveless said in interview.
