Widespread technical failures and interruptions of recent online testing in a number of states have shaken the confidence of educators and policymakers in high-tech assessment methods and raised serious concerns about schools’ technological readiness for the coming common-core online tests.
The glitches arose as many districts in the 46 states that have signed on to the Common Core State Standards are trying to ramp up their technological infrastructure to prepare for the requirement that students take online assessments starting in 2014-15.
Disruptions of testing were reported across Indiana, Kentucky, Minnesota, and Oklahoma and were linked to the states’ assessment providers: CTB/McGraw-Hill, in Indiana and Oklahoma; ACT Inc., in Kentucky; and the American Institutes for Research, in Minnesota.
Thousands of students experienced slow loading times of test questions, students were closed out of testing in mid-answer, and some were unable to log in to the tests. Hundreds, if not thousands, of tests may be invalidated.
The difficulties prompted all four states’ education departments to extend testing windows, made some state lawmakers and policymakers reconsider the idea of online testing, and sent district officials into a tailspin.
The testing problems were “absolutely horrible, in terms of kids being anxious,” said Eric F. Hileman, the executive director of information technology services for the 43,000-student Oklahoma City schools. Some high school students were taking Oklahoma’s high-stakes tests, which require that students pass four out of seven end-of-instruction tests to graduate.
“It was heartbreaking to watch them,” Mr. Hileman said. “Some of them were almost in tears.”
System Overload
The problems in Oklahoma and Indiana began on April 29.
Several states experienced major mishaps with their statewide assessments recently, breakdowns that caused delays and disruption for teachers and students.
Indiana
April 29: Over 30,000 test sessions were interrupted as students began taking state tests. The state department of education extended the testing window by three days.
April 30: Test interruptions spiked to 8 percent of test-takers, and the department suspended testing for the rest of the day.
May 1: The education department instructed districts to continue testing students, but to reduce the number of tests they plan to give daily by 50 percent.
May 2: The department extended the testing window an additional two days, for a total of five extra testing days.
Kentucky
May 1: School systems were ordered to suspend online end-of-course tests after dropped and slow connections were reported in about 25 districts throughout the state.
May 2: State officials said the problem was caused when its testing vendor, ACT Inc., reported that its system became overloaded. Company officials told the state that the capacity of the system would be increased, but the state department of education also said it would work with local districts to help them “maximize the testing system’s capacity” and avoid other problems. Online testing was scheduled to resume by May 8. State officials say they will provide districts that give online tests paper exams as an alternative.
Minnesota
April 16: Schools reported widespread problems with online testing. Up to 5,000 students’ tests were disrupted.
April 17: Testing resumed.
April 23: Test interruptions resurfaced. About 48 districts reported slow loading times or other problems.
April 24: A handful of districts reported persistent testing interruptions. About 60 students were affected.
May 1: Minnesota added one day to its testing cycle to allow districts to catch up.
Oklahoma
April 29: Students were taking state tests online when problems began at 9 a.m.
April 30: Testing began at 7 a.m.; problems arose again around 10 a.m. About 3,000 out of 300,000 students statewide experienced some test irregularities.
May 1: The state extended its testing window by two days to allow districts to catch up.
In Oklahoma—where roughly 300,000 students were using online tests and about 3,000 experienced problems—it was the end of the testing window for grades 3-8 and the middle of the testing window for high school students, said Tricia Pemberton, a spokeswoman for the state department of education.
At 9 a.m., students’ testing sessions were disrupted by the glitches, and some were unable to restart or to proceed, she said. The problems continued the following day, when the system crashed at 10 a.m.
Indiana experienced similar problems on those days, and districts were instructed later in the week by the state education department to reduce the number of tests being given by half to proceed with the state assessments.
Ms. Pemberton said the simple explanation from Monterey, Calif.-based CTB/McGraw-Hill was that computer servers could not handle the testing load.
In a statement, CTB/McGraw-Hill officials said that earlier practice simulations “did not fully anticipate the patterns of live student testing, and as a result, our system configuration experienced service interruptions that impacted the testing process.”
Kentucky officials were forced to suspend online end-of-course exams last week after problems, including slow or dropped connections, were reported in about 25 of the state’s school districts, said department of education spokeswoman Nancy Rodriguez. About 60 percent of Kentucky’s district’s deliver those state-mandated exams online, while the rest use paper-and-pencil assessments.
State officials issued a statement saying that the test vendor, ACT Inc., indicated the problem occurred when its system became overloaded, and that capacity for it had been subsequently increased. Online testing was scheduled to resume early this week, and state officials said they would provide districts with guidance on how to “maximize the testing system’s capacity” to avoid additional breakdowns.
In Minnesota, problems with online testing began on April 16 and were experienced in multiple districts across the state, affecting up to 5,000 students, said Charlene Briner, a spokeswoman for the state education department.
A week later, on April 23, 48 districts reported disruptions in online testing, and the following day, a handful of districts experienced further problems, she said.
Jon Cohen, an executive vice president of the AIR, the Washington-based not-for-profit research and assessment organization that provided the online tests in Minnesota, said the tests are designed to allow students to pause and log back in later if they’re experiencing technical problems.
According to AIR data, he said, about 3,000 students out of 15,000 being tested at the time had slower load times of more than 30 seconds. He said AIR servers were overloaded not by the number of test-takers, but by the large amount of diagnostic data the organization was collecting.
About 95 percent of students in Minnesota take the math portion of the state tests online, but only about 30 percent take the reading portion online, Ms. Briner said. That will change under the common-core standards, which Minnesota has adopted for English/language arts but not for math. All students will eventually have to take ELA tests online.
Ms. Briner said that in light of the recent online testing problems, however, the state is evaluating “whether or not a paper option is better for accountability testing.”
“We believe in moving to a next-generation set of assessments,” she said, “but we’re also believers in making sure people have confidence in the accuracy of the information we report.”
On ‘Pause’
Others were also worried about the future, particularly when it comes to common-core testing.
Coincidentally, just before the testing problems arose in Indiana, the state legislature passed and sent to the governor a bill that would “pause” common-core implementation there.
Glenda Ritz, Indiana’s superintendent of public instruction, told Education Week that the state might pull out of the Partnership for Assessment of Readiness for College and Careers, or PARCC, one of two consortia developing common-core tests.
She called the testing problems the state experienced last month “unacceptable.”
Wendy Y. Robinson, the superintendent of the 30,900-student Fort Wayne, Ind., community schools, said she doesn’t know how students, parents, or educators can now have confidence using online testing for the common core.
“Teacher pay, school evaluations, student grades … are all going to be tied to a system that none of us have any faith in anymore,” she said.
In Oklahoma, lawmakers such as Rep. Curtis McDaniel, a Democrat, called for a moratorium on online testing. “If we can’t get this little piece of the puzzle working in the right direction, how are we going to get it right for the whole state or the country?” he said.
“Common core has some good values,” he said, “but we need to re-evaluate what we’re doing.”
But Chad Colby, a spokesman for PARCC, said despite the problems, the advantages of computer-based testing remain, especially when it comes to evaluating student knowledge, offering more interactive testing, and maintaining test security.
He acknowledged, however, that the tests must be reliable.
“The benefits of computer-based assessments for students and teachers vastly outweigh the growing pains and issues in a few states,” he said. PARCC will work to solve any technical problems before the common-core online tests are rolled out, he added.
Anticipating Glitches
Joe Willhoft, the executive director of Smarter Balanced Assessment Consortium, the other group developing common-core tests, said a number of states in his consortium have already been using online testing with all their students for years without major incidents. He said he was confident any kinks could be worked out before common-core tests were launched.
But educators around the country remain concerned about their technical preparedness for common-core online testing, said Keith R. Krueger, the CEO of the Washington-based Consortium for School Networking, or CoSN.
A survey that CoSN released in March found that preparing for the online tests ranked second among the top priorities for the group’s members.
Though districts have primarily been worried about their own infrastructure and testing capacity, now they are realizing that even if they are well prepared, some problems are out of their control, Mr. Krueger said.
“I would think of this as the canary in the coal mine,” he said of the recent testing problems. “These things are not easy to pull off on a statewide basis. We need to do it in a careful way and plan for the unexpected.”
Mr. Cohen of the AIR, which is working with Smarter Balanced to deliver adaptive pilot tests that adjust the difficulty of questions based on how well a student is answering them, said his organization’s assessments are designed in a way that allows for glitches and gets students back on track when they occur.
“The tests need to be designed as online tests and not as paper tests,” he said. “You recognize that the technology is going to fail somewhere, sometime, and you build the test to be robust.”
Douglas Levin, the executive director of the State Educational Technology Directors Association, or SETDA, based in Glen Burnie, Md., added that there can be risks with paper tests, too, such as when floods washed out warehouses of tests in the past year, he said.
Even so, Mr. Levin—like Ms. Ritz in Indiana—said the recent problems with online testing were unacceptable.
While there’s been an emphasis on getting districts technologically ready to administer online testing for the common core, he said, “there is a need for the assessment industry to ensure that it has the capacity to serve these larger numbers of kids with a quality of service that really has to be very high.”
‘Doubt and Uncertainty’
In the meantime, districts were just trying to deal with the logistical and emotional fallout from the online testing problems.
John Althardt, a spokesman for the 30,000-student Indianapolis public schools, said students in 50 buildings experienced testing disruptions, and the district was just focusing on getting through the testing cycle before thinking about how to proceed in the future.
“Some of our folks would say they’re ready to go back and use stone tablets at this point,” Mr. Althardt said.
In Oklahoma, Superintendent Keith Ballard of the 42,000-student Tulsa public schools, said in a statement that the testing problems were “nothing short of disastrous,” adding that the district would be forced to invalidate at least 460 tests.
Ms. Pemberton of the Oklahoma education department said all students will have an opportunity to retake the tests if they want to. Those who scored enough to receive a proficient grade on the tests do not need to retake them, even if they did not finish.
But the logistics of extending the testing window and retesting students are significant.
In the Oklahoma City system, for instance, one middle school had bused its students to a local university because the school lacked enough devices for students to take the online tests.
Education observers will be watching whether the spate of problems helps prepare both districts and testing companies for the online common-core tests.
Mr. Levin of SETDA said the testing problems will provide an additional argument for opponents of the common core.
“There are people who, for all sorts of reasons, are looking for ammunition to spread fear and doubt and uncertainty about the implementation of common core overall,” Mr. Levin said. “I wouldn’t be surprised if this ends up among the arrows in their quiver.”
