Corrected: An early version of this story incorrectly said that a majority of states had made their testing standards less rigorous between 2005 and 2007, according to the National Center for Education Statistics. An NCES report found that among states that made changes to their testing systems, tests were made less rigorous nearly half the time.
With 2014 approaching as the deadline by which states must get all their students up to “proficient” levels on state tests, a study released last week by the U.S. Department of Education’s top statistics agency suggests that some states may have lowered student-proficiency standards on such tests in recent years.
For the 47-state study, researchers under contract to the National Center for Education Statistics used student test scores to figure out where the proficiency levels on various state tests would fall on the National Assessment of Educational Progress.
Their results suggest that between 2005 and 2007, various states made their standards less rigorous in one or more grade levels or subjects in at least 26 instances. In 12 instances, particular states appeared to make their standards more stringent in one or more grade levels or subjects.
Under the federal No Child Left Behind Act, adopted nearly eight years ago, the states must show that their students have attained proficient levels of performance in reading and mathematics by the end of the 2013-14 school year. Most states are far from reaching that goal.
“I think as 2014 looms, ... clearly what a lot of states are doing is changing the bar so that a lot more students will become proficient,” said Mark S. Schneider, who served as the NCES commissioner between 2005 to 2008.
State standards for what constitutes “proficient” performance on 8th grade mathematics exams dropped in 16 states between 2005 and 2007, a federal study finds. The proficiency bar was raised over the same period in four other states.
SOURCE: National Center on Education Statistics
In 4th grade reading, the federal study found that 31 states had set proficiency scores that were lower than the cutoff for “basic” performance on NAEP. At the 8th grade level, 15 states set standards judged to be lower than the basic level on the national assessment for that grade.
But the study uses a methodology that is controversial among some testing experts. They caution that the standardized exams that states use and the more rigorous NAEP—the congressionally mandated program known as “the nation’s report card”—are too different to put on the same scale.
“If two tests don’t measure the same thing, then mapping is misleading,” said Andrew D. Ho, an assistant professor of education at the Harvard Graduate School of Education. “You can map anything onto NAEP.”
The results, nonetheless, are expected to figure in growing efforts to develop common academic standards for what K-12 students should know and be able to do. So far, 48 states are taking part in a push to craft such standards, while federal education officials are making plans to award $350 million in grants to help states plan common assessments. (“Experts, Public to Weigh In on Common Tests,” Oct. 28, 2009.)
In statement issued in response to the study, U.S. Rep. George Miller, D-Calif., the chairman of the House Education and Labor Committee, said, “It is unacceptable that many states have chosen to lower the bar rather than strive for excellence.” He said the time has come for states to adopt a common core of internationally benchmarked standards.
U.S. Secretary of Education Arne Duncan said in a statement that the NCES report “confirms what we’ve known for a long time: States are setting the bar too low.” He said states should be raising their standards, and he referred approvingly to the common-standards effort as a starting point.
Common Metric Sought
The Education Department’s statistics agency has been developing and improving its test-mapping method since 2003. According to Peggy Carr, the associate commissioner of the NCES, the aim was to develop a common metric that state policymakers could use to assess how high their states have set the bar for proficiency.
It’s a challenging task: While the national assessment is intended to measure students’ progress against what they ought to know, state tests may have different goals, such as measuring whether students are demonstrating grade-level work. NAEP also draws on nationally representative samples of students from participating states; state tests, in theory, are designed to assess learning for every child.
The new analysis draws on testing data in reading and mathematics for grades 4 and 8. It does not include Nebraska, Utah, or the District of Columbia, or results for 8th grade math in California.
Both the new report and a 2007 study that grew out of the same project show that states vary widely in what qualifies a student as proficient. A 4th grader judged to be proficient in math in Colorado or Tennessee, for example, could conceivably test at the “basic” level in Massachusetts or Missouri, where the standards were judged by the study to be most rigorous.
“This is great fodder for why we need common standards that are of high quality,” said Amber M. Winkler, the research director for the Thomas B. Fordham Institute, a Washington think tank that supports common academic standards.
In collaboration with the Northwest Evaluation Association, a testing group based in Lake Oswego, Ore., Fordham in 2007 published a report that mapped state NAEP data with state-assessment data from the Northwest group and found variations similar to those in the NCES study across states. (“Report Pans How States Set the Bar,” Oct. 10, 2007.)
“I think the intent of NCLB was to let states make that decision” on where to set the proficiency bar, added Ms. Carr, “but I don’t think anyone conceived that we would be this discrepant or this disparate in our definitions about what proficient meant.”
Changing Benchmarks
The new study is the first by the NCES to analyze whether states had changed their testing standards, either implicitly or explicitly, over time. The report stops short, however, of alleging that states may be changing their standards to meet their NCLB targets for raising students to proficiency in reading and math. Under the federal law, students are tested annually in those subjects in grades 3-8 and once during high school.
“There are different ways states could’ve gotten to this point,” Ms. Carr said. “Some states just kept their assessment exactly the way it was and just ratcheted the cut scores up or down. Others made many changes in their testing systems so, of course, they had to have new cut points on top of their new assessments.”
In all, the report says, between 12 and 18 states, depending on the grade and subject, made changes in their state testing systems between 2005 and 2007.
One was Wyoming, where the proficiency bar appeared to have slipped in both reading and math and at both grade levels studied.
“There are really no surprises in Wyoming for this report,” said William Donkersgoed, the state’s national-assesssment coordinator. “There was a change in the assessment system, and it was like comparing apples and oranges from 2005 to 2007.”
Mr. Donkersgoed said the state’s previous system, like NAEP, was developed so that schools could measure their progress against a high standard, and there were no consequences for schools that fell short. The new system, in contrast, was devised in part by panels of teachers examining what grade-level performance should look like.
Rising testing standards in Montana, on the other hand, did not seem to be linked to any dramatic, or intentional, changes in the state testing system, according to Mike Chapman, the NAEP coordinator for that state. He said state officials surmise that the change resulted from instituting tests in additional grades and offering teachers more professional development—two actions that may have had indirect ramifications for the level of that state’s 4th and 8th grade cutoff scores, which are recalibrated from year to year.
“In most states, the actual cut scores haven’t changed,” said Mr. Ho of Harvard. “What’s happening is that a change on one test is outpacing the change on another.”
The NCES researchers also looked to see whether achievement gains on state tests tracked with improvements in state students’ performance on the national assessment. For states that had not overhauled their testing systems, that was the case 60 percent to 90 percent of the time, said Ms. Carr.
“That’s good,” she added, “because that is what NAEP was intended to do.”
