Cheating on standardized tests is not only common, it can also have long-term effects on students’ academic outcomes.
Like social promotion, cheating to help students pass a high-stakes test can allow those who are struggling to get over procedural hurdles, like moving to the next level in school, but it can hurt them when they try to move to more difficult material later on, according to new research on students in New York and Georgia, two states that were rocked by recent high-profile cheating scandals.
In 2011, news reports exposed widespread anomalies in New York State Regents math and science tests, the standardized tests required at the end of many core-subject classes for students to graduate.
The anomalies were caused by local “rescoring” policies for the exams, in which teachers automatically reviewed the short-answer and essay questions of students whose scores fell near the proficiency cutoffs for core subjects. Back then, the researchers found 6 percent of the scores—and 40 percent of the scores near the cutoffs for the state’s three tiers of high school diplomas—were bumped up over the threshold. In the years after that first study came out, New York changed its grading policies, first to bar rescoring and then to move scoring to central locations outside the schools, where scorers would not know the students whose tests they graded.
By tracking what happened to more than a half-million general education students who took the Regents exams from 2003-04 to 2012-13, before and after the grading policies changed, Stanford University economist Thomas Dee and colleagues at Princeton and Columbia universities and the University of Michigan dug into what made teachers manipulate test scores and what happened to students whose scores were inflated.
A ‘Cultural Norm’
Teachers at schools with high percentages of poor or minority students were more likely than those in other schools to manipulate test scores, as were teachers at schools whose 8th grade test scores ranked below the city’s average.
Mean likelihood of graduation for students with inflated scores:
Source: National Bureau of Economic Research
Researchers found, however, that the rates of cheating did not significantly rise in schools where teachers were eligible to receive bonuses based on their students’ test-score growth. Nor was cheating significantly more prevalent when schools were under more pressure from state or federal accountability systems.
“It seems that the manipulation of test scores may have simply been a widespread ‘cultural norm,’ among New York high schools, in which students were often spared any sanctions involved with failing exams, " the researchers concluded. The suspicious test-score patterns virtually disappeared after the state banned rescoring and moved grading outside of local schools.
For struggling students, the bump went a long way. Of students who barely passed the Regents, scoring 60 to 69 points on core subjects, on average 76 percent graduated from high school. Having a score pushed just over the exams’ passing cutoff increased a student’s odds of graduating by more than 28 percent. The researchers found the black-white graduation gap in New York City would have been nearly a third larger without the cheating, and the gap in graduation rates between high- and low-performing schools would have risen from 18.3 percentage points to 22 percentage points.
Still, getting students over the Regents bar didn’t always help them. A student whose scores were manipulated was statistically 11 percentage points less likely to earn an advanced college-ready diploma than he would have been otherwise.
“On the one hand, if you are on the margin of dropping out in high school, and the score manipulation gives you a mulligan, you’re pretty unambiguously better off over your entire life,” Dee said. “But if you’re a kid who had a low Regents exam score but a high capacity for future achievement, this manipulation could be harmful, because you’re not going to relearn the kind of foundational material you didn’t demonstrate an understanding of in the Regents exam.”
In secondary analyses, Dee and his colleagues found students who had been on a trajectory to earn an advanced diploma before their scores were manipulated were less likely to take and pass advanced-math classes later on in high school.
Cut Off From Remediation
A separate study by Georgia State University economists Tim Sass, Jarod Apperson, and Carycruz Bueno found even stronger academic problems caused by widespread cheating on state standardized tests.
While Georgia’s largest district, Atlanta, was embroiled in a widespread cheating scandal at the same time, Sass, an economics and public policy professor, and his colleagues began to track the consequences of cheating on students in Atlanta and another, undisclosed district.
In both districts, the researchers found schools with higher percentages of black students or students in poverty were significantly more likely to show signs of cheating, such as excessive erasures from wrong to right answers on multiple-choice questions, according to a pro bono report for Atlanta public schools and a separate study of the unnamed district—presented at a conference last month for the National Center for Analysis of Longitudinal Data in Education Research.
In the CALDER study, the researchers tracked the academic progress, behavior, and attendance of more than 50,000 elementary and middle school students in the years following the cheating. In 2010, the first academic year after the cheating was discovered, they found middle school students’ attendance and behavior stayed the same, but for students who had had 10 or more test answers changed, their scores in both English/language arts and math dropped significantly.
Math performance recovered in the second year after the cheating scandal broke, but reading and English scores remained low.
Links to Social Promotion
Sass, a policy-studies professor at Georgia State, suggested that this sort of cheating is similar to grade inflation or social promotion: Students whose scores are improperly changed get passed into progressively harder work they are not prepared for. “One of the main concerns with teacher cheating is that because of artificially inflated test scores, students are not identified to receive remedial services, such as intervention programs, summer school or retention,” the researchers told Atlanta public schools. “If remedial programs increase student achievement, denial of these services could potentially have lasting effects on the student.”
Based on the report, Atlanta launched Target 2021, a $9 million initiative to create individual support plans for some 3,700 students who were affected by the cheating and are still enrolled. Sass and his colleagues found on average, those students’ math and reading scores lagged their peers by a half-year or more.
In the other district, the researchers found cheated students were significantly less likely to be eligible for extra support in math a year later and in reading for two years.
Sass and his colleagues also found some preliminary evidence that students whose scores had been manipulated by 10 or more changed answers became less likely to graduate from high school than those whose test scores were not fudged at all.
There was evidence that teachers in the Georgia study changed multiple-choice answers to boost students’ scores as the school system was operating in a high-stakes accountability environment. But teachers in New York likely thought they were manipulating scores for “altruistic” purposes, Dee said—to help “good” students who were having a “bad test day,” for example. The students who had previously had better scores in non-Regents math and science tests were more likely to be nudged over a test cutoff, as were those with better behavior records.
But that subjective judgment also led to implicit biases, Dee said. Black and Hispanic students made up 70 percent of students scoring near the “bubble” of the Regents test, and so they also made up the majority of those who had their scores manipulated. But teachers still were more likely to give borderline white and Asian students the benefit of the doubt: White and Asian students made up only 4 percent each of students in the test-score bubble, but more than 44 percent of those had their test scores nudged over, compared with 40 percent of black and Hispanic students on the bubble.
Taken together, the studies suggest that cheating can be widespread in more than just multiple-choice tests, and even within environments that are not under heavy accountability pressure.
“Even when the teachers were acting out of what we might think of as a kindness, edging [students] above that test threshold, it harmed them,” Dee said. “It suggests that there were kids who had that bad day who had greatness within them, and that manipulation prevented them from having that greatness realized,” he said.
