A handful of states have increased the rigor of their state assessments since 2007, an analysis released today by the statistical wing of the U.S. Department of Education concludes.
The finding stands in contrast to earlier studies of state proficiency standards—often referred to as cutoff scores—released by the federal agency, which has generally found mixed patterns in the rigor of states’ 4th and 8th grade proficiency standards in reading and mathematics. Its 2009 report, using the same methodology, found dozens of instances between 2005 and 2007 in which states had lowered their expectations for students.
“We’re actually seeing [states] increase the rigor of their cut scores, at least between 2007 and 2009,” Jack Buckley, the commissioner of the National Center for Education Statistics, said during a conference call with reporters. “That doesn’t fit into the narrative of states lowering their bars” in response to the performance pressures of the No Child Left Behind Act.
Using the National Assessment of Educational Progress as a common yardstick, the analysis finds that during the 2007 to 2009 time period, eight states raised the cut score—the level students must reach to be deemed “proficient”—on one or more exams, while two states lowered them. Other states made no changes during that time, or made changes that were too small to be significant.
Since 2007, states have tended to make their tests harder for students to pass, by increasing the proficiency standard.
SOURCE: National Center for Education Statistics
Even with the higher standards, however, states’ expectations continue to fall far below those on the prominent national exam. And the range of state proficiency standards continues to be vast, with as many as 71 points on the NAEP scale separating the states with the weakest and strongest expectations.
“This mapping study shows low expectations are the norm in way too many states in this country,” said Joanne Weiss, the chief of staff to U.S. Secretary of Education Arne Duncan, adding that the practice of having each state set its own proficiency standards is “tremendously problematic.”
“It’s lying to parents, lying to children, lying to teachers and principals about the work they’re doing,” she said.
The analysis also found no clear relationship between the rigor of the standards and performance on NAEP, and that states often reported gains on their own state tests that did not appear on NAEP.
Common Metric Used
The analysis translates each state’s cut scores for four exams onto the NAEP scale, using it as a common metric to compare state proficiency expectations. It is the fourth report released by the NCES to use that methodology.
The analysis looks only at the relative stringency of state proficiency criteria. It does not address the differing content of the assessments or the varying purposes for which the tests are used. Some researchers have questioned the appropriateness of using NAEP as a common yardstick because of such limitations.
The results cover forty-nine states and the District of Columbia. One state, Nebraska, was not included in the results. Until recently, that state used a patchwork of local assessments to meet the NCLB law’s annual testing requirements, rather than statewide exams.
Past NCES studies have, in general, highlighted disparities among state definitions of proficiency, a phenomenon some observers credit for accelerating efforts to create common academic content standards. (“NCES Finds States Lowered ‘Proficiency’ Bar,” November 4, 2009.)
A study of proficiency standards between 2005 and 2007, for instance, found some two dozen instances in which states had lowered their cut scores.
The phenomenon of lowered standards was widely attributed to the pressure of the NCLB law, which requires states to test students annually in reading and math and to increase the percentage of students scoring at the proficient level over a 12-year period.
It emerged as a core criticism of the 9-year-old law by U.S. Secretary of Education Mr. Duncan, who has spent much time this year, so far unsuccessfully, pressuring Congress to revise it.
But the new findings show that between 2007 and 2009, the states of Indiana, Mississippi, Michigan, North Carolina, New Jersey, Oklahoma, South Dakota, and West Virginia raised the bar in one or more grade levels or subjects. Only two states, New Jersey and South Carolina, decreased student expectations.
Ms. Weiss of the Education Department credited the NCES’ reports on state cutoff scores as one possible impetus behind the higher expectations.
“Data, I think we’re seeing, has moved policymakers to change their behavior and has helped policymakers to raise the bar in the last couple of years,” she said.
Mississippi, long criticized for expecting too little of its students, raised the bar for all four subject-grade combinations over that time period. Its reading expectations for both 4th and 8th graders now outpace those of most other states, while its math expectations now fall in the middle of the pack.
In contrast, South Carolina ratcheted down its expectations in all four subject-grade combinations during that time period. In both math and reading at grade 4, its standards now trail those of many other states.
“From what we know talking to South Carolina [officials], they fully intended to set their bar lower,” Mr. Buckley said. “For whatever reason, they felt they had set their standard too high.”
A South Carolina education department official noted that the changes occurred before state commissioner Mick Zais took office in January. Further, an independent body, not the state education department, sets cutoff scores in the state, he said.
“Dr. Zais has consistently called for strong accountability and high proficiency standards for students,” said spokesman Jay W. Ragley. “The NAEP report confirms his concerns that the education establishment worked to lower South Carolina proficiency standards under the guise of developing a new statewide accountability test in 2008 to comply with No Child Left Behind.”
Former South Carolina superintendent Jim Rex could not be reached for comment.
Disparities Remain
Overall, states’ proficiency standards continue to fall well below those of the NAEP.
The largest disparities exist for 4th grade reading, where 35 states’ proficiency criteria fell below the “basic” level on NAEP. In the other three subject-grade combinations—4th grade math and 8th grade math and reading—a majority of states’ proficiency standards fell between the basic and the proficient levels on the national exams.
The cause of that finding is difficult to interpret. It could mean that the National Assessment Governing Board, which oversees NAEP policy, has set a higher bar overall for performance on its grade 4 reading exam, Mr. Buckley said, but that supposition is impossible to confirm barring additional analysis.
The report also analyzes changes in student achievement for those states which did not alter their cutoff scores or tests over the time period studied. It found that, in most cases, gains on state exams were not corroborated by NAEP.
More recently, a handful of other states have moved to change their proficiency expectations.
One state with low proficiency standards according to the analysis, Tennessee, recently implemented a tougher assessment regime, and won plaudits from Mr. Duncan for doing so. New York also recently raised its cutoff scores.
The Education Trust, a Washington-based group that advocates on behalf of poor children, said other states need to follow suit. A spokeswoman praised states for beginning to address disparities in performance between state tests and NAEP, but added, “we’re still nowhere near where we need to be.”
Common Core Link?
Several observers argued that the move to common standards, which 46 states have now adopted, could give additional momentum to the effort for raising standards.
Last week, Education Department officials said they would allow states to waive portions of the NCLB law in exchange for adopting various reforms, including a set of “college and career-ready” academic-content standards.
Mitchell Chester, the commissioner of education for Massachusetts, dismissed concerns voiced by some that the Education Department’s waiver plan would co-opt what has been a state-led effort for common standards.
“It seems to me that there’s lots of precedent for federal policy-building on an aggressive state policy, and this report for me made that concrete,” he said. “It’s really striking that not only was [higher standards] the rhetoric of governors and the chief state school officers several years ago, but that many of them backed it up.”
