State efforts to carry out standards-based education over the past decade have a positive, but modest, relationship with gains in student achievement on the National Assessment of Educational Progress, according to this year’s edition of Quality Counts.
The 10th annual report by Education Week examines the progress that states have made on a core set of policy indicators related to standards-based education over the past decade. It also explores the relationship between changes on those indicators and gains in student learning, as measured by NAEP reading and mathematics tests in grades 4 and 8 from 1996 to 2005.
The report does not examine the more recent impact of the federal No Child Left Behind Act, which President Bush signed into law in 2002.
The results are at once heartening and sobering.
They’re heartening because when looked at over more than a decade, student achievement has gotten better, particularly in mathematics and particularly for low-income and minority students. An increasing number of states also have embraced a standards-based-education framework, with some of the earliest and most ardent adopters of standards-based accountability systems making some of the most progress in student achievement.
Nationally, NAEP scores in 4th grade math have increased by 18.5 points on a 500-point scale, or nearly two grade levels, since 1992, near the start of the standards movement.
For the first time, the Editorial Projects in Education Research Center also has produced detailed state-by-state reports on how states have performed on this year’s indicators and the progress they’ve made over time.
See the state highlights reports, which replace and expand on the state summaries that appeared in previous print editions of Quality Counts.
Even more encouraging are the gains for black and Hispanic 4th graders: 27.7 points and 24.2 points, respectively. One way to think about those gains is that if the scores for white students had not also improved, the advances would have been enough to shrink the black-white achievement gap that existed in 1992 by 80 percent, and the gap between Hispanics and non-Hispanic whites by 94 percent, virtually closing the gap between those groups in 4th grade math.
The scores for low-income students, which NAEP began reporting in 1996, basically mirror the average national increases.
In math, improvement has come at the “basic” and “proficient” levels on NAEP, both for students overall and for black, Hispanic, and low-income youngsters. Those gains are mirrored across most states, according to special analyses of NAEP scale scores conducted for Quality Counts 2006 by the Educational Testing Service, which looked at changes in state performance between 1992 and 2005. Of the 41 states and the District of Columbia with data, all made significant progress in 4th grade math achievement, and all but two states made significant progress in 8th grade math achievement during that period.
It’s hard to be as sanguine about reading. The national average in reading barely budged from 1992 to 2005, inching up just 2 points in both grades 4 and 8. But, even here, somewhat better news lies beneath the surface. The scores for black, Hispanic, and low-income youngsters in 4th grade reading increased at nearly triple the national average, or about two-thirds of a grade level.
Nationally, the achievement gaps narrowed somewhat for all groups in reading and math and at both grade levels—except between black and white students in 8th grade reading. The gap-closing was significant between black and white students in math at both grades, and between Hispanic and non-Hispanic white students in 4th grade math.
Progress is even more mixed at the state level, according to the ETS analysis, although the picture is complicated by the fact that many states either did not take part in the state-level NAEP during the periods examined or did not have enough poor or minority students in the NAEP samples to permit valid comparisons of change over time.
The 2006 report highlights individual states—including Delaware, Massachusetts, New York, North Carolina, and Texas—whose progress stands out over the past decade, with in-depth profiles that explore what might explain such changes.
The more controversial question is whether state policies over the past decade have made any difference in raising student achievement.
Quality Counts at 10: A Decade of Standards-Based Education tracks state policy initiatives over the past decade in four core areas—standards, assessments, accountability, and efforts to improve teacher quality—based on 24 specific indicators.
The report documents a steady increase in the number of states embracing state content standards and tests linked to those standards between 1997 and 2005. Fewer states adopted all of the accountability and teacher-quality indicators tracked for the report during the same period.
To examine the relationship between standards-based education and gains on NAEP, the Editorial Projects in Education Research Center conducted a series of regression analyses for Quality Counts 2006. The independent variable was changes in the strength of states’ standards-based policies between 1997 and 2005. The dependent variable was changes in NAEP math achievement between 1996 and 2005, and in NAEP reading achievement between 1998 and 2005.
The center conducted separate analyses for math and reading in grades 4 and 8. To avoid biasing the results, the analyses controlled for states’ initial NAEP performance at the start of the period, and for the initial strength of their standards-based policies.
One way in which the research center’s analysis differs from other studies is that it considered the overall effect of states’ standards-based policies—across the areas of standards, assessments, accountability, and efforts to improve teacher quality—and looked at which of those policy areas, relatively speaking, independently contributed the most to changes in achievement.
Initially, the analyses found a moderate positive relationship between states’ overall embrace of standards-based education and gains in student math achievement. The researchers observed a slight negative relationship for reading. Further analyses exploring the relative contribution of the four major policy components revealed that the implementation of policies related to teacher quality related negatively to achievement gains in both reading and math.
In a second analysis, the researchers eliminated the teacher-quality policies from the overall measure of standards-based-reform implementation in order to focus specifically on the contributions of policies related to standards, assessments, and accountability. Once teacher quality was taken out of the picture, the relationship between states’ efforts to implement standards-based reforms and gains in student achievement became much stronger.
Improvement for math in grades 4 and 8 became statistically significant, while more modest, but positive, effects emerged for reading.
Preliminary analyses also found no relationship between state resource and equity indicators and student-achievement gains, after controlling for states’ initial NAEP performance.
Such exploratory analyses have their limits.
Gains on NAEP may fall short of measuring changes in student achievement at the state level, in part because NAEP tests are given only to a sample of students in each state, do not carry any stakes for individual students or schools, and may not completely match state content standards and priorities. States also vary in the specific policies they employ, when they put them into effect, and how consistently and coherently they carry them out. From a statistical perspective, the analysis also relies on a small sample, the 50 states. That fact provides another reason for caution in interpreting the results.
Despite those caveats, the time and energy put into standards-based education over the past decade suggest it’s appropriate to examine the effects of state policies as a way to help inform state practices in the future. The results suggest that while standards-based education and its implementation are far from perfect, they can contribute to improvement in American schools.
As part of this 10-year retrospective on standards-based education, Education Week also invited five prominent policy observers to contribute their personal views to Quality Counts on what standards-based policies have accomplished so far, and what the next phase of improvement steps should be.
As is true every year, the 2006 report also tracks student achievement across the 50 states and the District of Columbia, and charts progress on states’ education systems in four areas: standards and accountability, efforts to improve teacher quality, school climate, and school resources and the equity of school finance systems. States averaged a C+ across the graded categories this year.
