An extensive analysis of standardized test results across the country conducted by the Atlanta Journal-Constitution found that about 200 of the 14,700 districts investigated had schools where test scores were suspiciously inflated.
In an article published Sunday, the newspaper analyzed scores for 69,000 schools (a link to the methodology the newspaper used can be found here.) The reporters requested average reading and mathematics results for state exams given in grades 3 through 8 from 50 states and the District of Columbia, as well as the count of students tested for each school, grade and subject in those jurisdictions. Scores that rose or dropped more than a predicted amount were flagged as unusual.
The paper noted that the score variations do not prove cheating occurred. However, it suggested that the results merit further investigation.
The Dayton Daily News, part of the same Cox Newspapers chain that owns the Atlanta Journal-Constitution, wrote its own story in partnership with the Atlanta newspaper, using the same methodology but focusing more closely on Ohio schools.
The Journal-Constitution is no stranger to investigating test scores. Its digging into test scores in the Atlanta school system prompted a state investigation that eventually led last year to one of the largest cheating scandals in the country. The Georgia Bureau of Investigation determined that cheating on the 2009 administration of the state’s Criterion-Referenced Competency Test occurred at 44 of the 56 Atlanta schools it investigated.
In the new article, the newspaper contends the school districts took an “apathetic, if not defiant” stance to the newspaper’s findings. Several denied any problems; others acknowledged that the changes seemed unusual but did not blame testing improprieties.
However, days before the newspaper released its report, many organizations were ready with a response, and others have come forward since publication to say that the newspaper’s methodology was flawed or that cheating will not be condoned.
The Washington-based Council of the Great City Schools, a coalition of the nation’s urban school districts, said “these are serious charges that should be supported by the strongest evidence. That strong evidence has not been provided.”
A statement from the National Education Association said NEA President Dennis Van Roekel “categorically denounces cheating or altering test scores. However, he said individuals accused deserve a thorough investigation, and should not be accused based solely on complicated probability formulas.”
The Journal-Constitution has gathered more responses here.
Gary Miron, a professor of education at Western Michigan University in Kalamazoo who reviewed the data for Ohio at the request of the Dayton Daily News, wrote in a Washington Post blog that the testing irregularities uncovered by the newspaper could be an artifact of high student mobility, not test tampering.
Miron said in an interview that he was given the Ohio data to analyze less than a week before it was published. What the newspapers did “was a reasonable first step, but you have to be cautious,” he said. “You want a high level of certainty, and to not cast suspicion on a variety of school districts.”
This investigation used a fairly wide net to identify schools. Test-cheating investigations typically take into account the number of wrong-to-right erasures on the tests themselves. Even though this one did not, the number of districts flagged for potential problems was small—just 1.3 percent of the total number investigated.