Just about everything that could go wrong did go wrong with online state testing this year in Tennessee.
First, login problems prevented some students from accessing the exams. The following day, an apparent cyberattack on the state’s assessment vendor prevented thousands more students from submitting their tests.
Then, thanks to human error at some schools, about 1,400 students ended up taking the wrong version of the TNReady exam. And a rogue dump truck severed one of the state’s main fiber-optic cables, causing temporary connectivity problems during the testing period.
Bringing frustrations throughout the state to a boil: It’s the second time in three years that schools have experienced major online-testing disruptions.
“We’re sick and tired of the failures,” said state Rep. Eddie Smith, a Republican.
Along with fellow lawmakers, Smith responded by quickly passing a new state law that prevents scores on this year’s TNReady exam from being used to punish schools, teachers, or students.
The move highlights the significant policy implications when online testing goes awry. It also puts the U.S. Department of Education in an uncomfortable position. Secretary of Education Betsy DeVos could be forced to decide whether to grant Tennessee a waiver from the testing and accountability requirements of the federal Every Student Succeeds Act, setting a precedent the department would likely prefer to avoid.
Across the country, the transition to administering state standardized tests via computer has at times been rocky. Schools have dealt with cyberattacks and poor connectivity, unfamiliar interfaces and login problems, scoring errors and students getting booted from online-testing platforms. In recent years, however, such troubles have generally been more episodic than systemic. During spring 2018, publicly reported online-testing problems occurred in:
New York
Like Tennessee, the New York state education department contracts with a company called Questar.
On April 11, the company was hit with an apparent cyberattack, causing an “unacceptable failure” in New York, said state board of regents Chancellor Betty A. Rosa and state education Commissioner MaryEllen Elia in a statement.
Across the state, students had difficulty logging in and connecting to the testing platform. According to Questar officials, 4,723 New York students were initially unable to submit their test responses as a result of the troubles.
Minnesota
Minor connectivity issues and login problems delayed testing at three Minnetonka schools late last month.
The state’s assessment vendor is global education company Pearson. In a statement, the company said its support teams responded to “sporadic reports of connectivity issues at schools” and “quickly added additional capacity, which immediately mitigated the issue.”
Mississippi
Another state using Questar, Mississippi also experienced problems with intermittent connectivity and some students being unable to submit online tests this year.
All told, 47 of 144 districts were affected, and a total of 606 students in the state experienced problems during a one-hour disruption, according to a spokeswoman for the state education department.
Ohio
Students across the Buckeye State had trouble logging in to state tests on April 18. The issue was resolved later that day. State education officials declined to detail how many students were affected.
Ohio contracts with the American Institutes for Research to administer its statewide exams.
Tennessee
For the second time in three years, Tennessee experienced widespread troubles with its online TNReady exams. The issues stemmed from a number of separate problems and prompted new state legislation.
—Benjamin Herold
Despite the meltdown in Tennessee and smaller issues in a handful of other states, assessment experts say the nation’s K-12 schools mostly seem to be getting the hang of how to administer online tests to millions of students each year.
“I don’t know that we’ll ever get the problems down to zero,” said Marianne Perie, the director of the Center for Assessment and Accountability Research Design at the University of Kansas.
“But paper-and-pencil testing was never error-free, either,” said Perie, who also chairs the technical-advisory committees around online testing in New York and Tennessee.
A Troubled History
Like many states, Tennessee made the decision in 2014 to switch to online tests.
The move was originally spurred by the state’s membership in the Partnership for the Assessment of Readiness for College and Careers, which was preparing to roll out new assessments aligned to the Common Core State Standards.
The Tennessee legislature quickly backed out of PARCC, however, setting in motion a multiyear chain reaction.
In spring 2014, faced with a narrow window to find a new online-testing vendor, the state education department rushed into a $108 million contract with a company called Measurement Inc.
Despite what department officials now describe as concerns about the company’s lack of experience administering a large-scale statewide online test, the state education department pushed forward with a wholesale switch to the new online assessments in 2016.
The administration of the exams that year quickly went awry.
State education Commissioner Candace McQueen responded by suspending testing in grade’s 3-8 and canceling the state’s contract with Measurement Inc.
At the time, Henry Scherich, the company’s president, told Education Week that Tennessee was giving up too quickly on the online tests.
Scherich declined to comment last week, citing ongoing litigation between his company and the Tennessee education department.
By summer 2016, the state had selected Questar as its new assessment vendor. Back then, state officials touted the company’s experience and track record as reasons for awarding it a $30 million per-year contract.
Limited online testing last spring and fall—part of McQueen’s new phased-in approach—was mostly smooth.
This year was supposed to be the breakthrough. All told, 650,000 students were expected to take the TNReady exam. More than 300,000 of them were expected to do so online.
But problems started almost immediately.
A software flaw caused “unacceptable login delays” for some students on the first day of testing, according to a presentation later delivered by Tennessee education officials to state lawmakers.
On the second day of testing, Questar was flooded with unanticipated traffic that overwhelmed the company’s servers and prevented some students from connecting to the TNReady testing platform.
All told, the problems resulted in 5,066 Tennessee students being initially unable to submit their tests, according to Questar. An undetermined number of additional students also experienced login difficulties, and some may have been kicked off the testing platform after they started taking the exam.
In the presentation to the Tennessee legislature, McQueen described her department as “completely devastated” by the disruptions.
Outsiders said Tennessee’s recurring problems point to some kind of deeper underlying trouble.
“I’m not aware of a state that has had a more troubled transition” to online testing, said Douglas A. Levin of the consulting group EdTech Strategies.
‘The Million-Dollar Question’
Questar officials noted that they have successfully administered roughly 3 million tests across multiple states this spring.
But any problems are unacceptable, Brad Baumgartner, the company’s chief operating officer, said in an interview.
The initial login problems were the result of “a design decision we made a year ago, which in hindsight we would not have made,” Baumgartner said.
And the flood of unanticipated traffic on day two of the exams bore the hallmarks of a type of cybersecurity incident known as a distributed denial-of-service, or DDoS, attack. In addition to Tennessee, students in New York and Mississippi were also affected.
Given past DDoS attacks during online state assessments, Questar likely should have been better prepared, said Levin. It’s particularly concerning that the company was apparently running multiple state assessments through the same “load balancing” server, creating a single point of failure when the traffic surge occurred.
Why weren’t such problems caught before the live administration of the exams?
“That’s the million-dollar question,” Baumgartner said.
Questar’s contract with the Tennessee education department is up for renewal this fall. Sara Gast, the communications director for the department, said officials anticipate “revisiting” the state’s deal with the company.
In the meantime, Tennessee educators and lawmakers are out of patience.
“Student and teacher morale has been hurt, parents are increasingly upset, and the credibility of the entire testing system has been severely damaged,” said Jim Wrye, a spokesman for the Tennessee Education Association, the state’s largest teachers’ union.
As the testing problems mounted, TEA members flooded state lawmakers with complaints.
The state legislature responded, quickly passing measures designed to prevent 2017-18 TNReady results from resulting in adverse consequences for those in the K-12 system.
Now, none of this year’s test scores can be used:
• To assign a letter grade to a school under the state’s accountability system or to designate a school as low performing;
• To move a school into the state-run Achievement School District; and
• To make decisions about hiring, firing, or compensating any teacher.
In addition, Tennessee districts were given discretion to determine the extent to which this year’s test scores should be used to determine students’ final grades.
“What we’re doing is finding the best way to hold everyone harmless, but to not have unintended consequences by doing more than we should,” said Smith, the state representative.
What’s Next?
Even as state testing continues, the Tennessee education department has referred the apparent cyberattack to the Davidson County district attorney and investigators from both the state and an undisclosed third party.
In addition, the department takes questions about the validity of this year’s TNReady results “very seriously,” Gast said.
“We have provided guidance on how a school can nullify the results for a student if they feel the interruptions prevented him or her from demonstrating their full understanding of the standards,” she said.
Department staff members are also helping schools recover interrupted test sessions, and they will be conducting numerous analyses to make sure that any unusual score results are excluded.
As for applying for an ESSA waiver, Gast said the department is “still discussing with the U.S. Department of Education what we may need to submit to them,” but remains confident it will ultimately be in compliance with the federal education law.
The reality is that Tennessee’s online-testing mess has left everyone in a difficult position, said Chad Aldeman, a principal at Bellwether Education Partners, a consulting organization.
“The state has not [made] stability a key priority in their testing vendors,” Aldeman said, and as a result,Tennessee may need to be “bailed out by the feds.”
