Beef up your digital networks. Marshal enough computers for test days. Ensure that all students have user names. Have a backup plan if the system fails.
Those are some of the important steps school technology leaders need to take before their schools use online assessments, experts on such testing say. Without proper planning, they say, districts are likely to encounter all kinds of problems, turning online testing into more of a liability than an asset for improving student achievement.
“You definitely don’t want a glitch during testing,” says Jim Hirsch, the associate superintendent for academic and technology services for the 33,000-student Plano, Texas, school system.
For more stories on this topic see Testing and Accountability.
Nine states now offer computer-based assessments of one sort or another to all students, while another 14 offer computer-based assessments to at least some students, according to the Editorial Projects in Education Research Center. Some of those assessments are not delivered via the Internet.
Among the leaders, Oregon and Virginia are using online assessments for their standardized state tests, at least as an option for schools. In Texas, online assessments are used as an option for students with special needs, for a reading proficiency test, and for some parts of the state’s standardized tests. In addition, online assessment is being pilot-tested in Texas for certain end-of-course exams.
Capacity Crunch
But online assessments can overburden a school’s technology, authorities caution.
For instance, South Dakota discontinued the online version of its state assessment in 2003 even though it had been recognized for its pioneering efforts in online testing. State officials had grown increasingly concerned that imbalances in technology capacity across the state and possible technical glitches would affect the accuracy of the test results. And in May of this year, many public high school students in Virginia had their state-mandated online test-taking interrupted by three major glitches, which hit sporadically across the state. Officials blamed the problems on software or server problems in the online testing system provided by an outside contractor and on Internet congestion caused by the large number of students taking the test at once.
At the local level, online testing demands careful allocation of school technological resources, says Edwin J. Zaiontz, the executive director of information services for the 40,000-student Round Rock school district in Texas.
He learned that lesson recently during a trial of new Texas end-of-course assessments in biology and geometry. Initially, Zaiontz thought that all five of the district’s high schools would be involved and testing about 1,500 students.
Because test security required roughly simultaneous testing of all the students, that posed some problems. If Zaiontz turned the schools’ media centers, which have about 50 computers each, into test centers, the testing would occupy them for 15 school days, he calculated.
1.Make sure that enough network-connected computers are available for the number of students to be tested.
2. Reschedule network maintenance so as not to interfere with the schedule for online testing.
3. Plan for different possible glitches, depending on whether the tests are downloaded from a testing company to student computers or hosted entirely from the company’s Web site.
4. Verify that all students have valid user-identification names and passwords for the testing.
5. Use network-management systems to prevent test-takers from using online chats, e-mail, or discussion boards.
6. Check whether other software on computers may be incompatible with the software used in testing.
7. Allow ample time for preliminary tasks, such as students’ typing of their names on electronically submitted assignments.
8. Make sure that technical support is available during the administration of the exams, and that backup procedures are in place for emergencies, such as system crashes or loss of power.
9. Be sure that the server and network being used are reliable.
10. Isolate the server containing the exam questions from the Internet to maintain test security.
“Going without the library for three weeks—that wasn’t a good alternative,” he says. “You’re closing the library at a time when students are needing to use it for research or projects.”
But using high school computer labs would bring to a halt courses in computer-assisted drafting, Web design, and journalism “because computers are key to instruction that is occurring,” Zaiontz notes.
As it turned out, only 300 students at one district high school took part in the field test, which the school accomplished by using the library plus one computer lab for three days.
What might have been a third option—using sets of laptop computers on carts with wireless connections to the Internet—was ruled out for the trial. NCS Pearson Inc., which the state hired to run the online test, considered wireless less reliable, Zaiontz says, and students’ performance could be adversely affected by laptops’ small screens, keyboards, and trackballs.
Still, as online testing becomes more common and as students become more familiar with laptops, the smaller computers might be the best way to go, he says, because they would allow students to take tests in their classrooms.
Adaptive Assessments
Other types of online assessment pose fewer potential problems because they do not count for such high stakes, some experts point out.
For example, Agile Mind Inc., based in Grapevine, Texas, offers complete online curricula in high school math that have multiple types of assessment embedded within lessons and available in large banks of test items for benchmarking.
Other companies, such as Study Island, headquartered in Dallas, develop online tests based on state standards for informal drilling of students to prepare for state tests. School districts can schedule students to take those tests during classes, in study halls, or at home.
One test provider that has acquired a strong following in school districts is the nonprofit Northwest Evaluation Association, based in Lake Oswego, Ore., which provides standards-based tests for kindergarten through grade 10.
Raymond Yeagley, the NWEA’s vice president of product and business development, says the organization does not serve up those tests to students on a secure Web site, as some companies do. Instead, a district downloads the tests and loads them on its own computer server. Students can log in over their schools’ networks and answer the test items that teachers have selected.
The NWEA tests are adaptive, meaning that a student gets easier or harder test items depending on how well he or she is doing. An adaptive test requires much more computer processing than a test that is not adaptive.
“Because of the adaptive nature of the test, bandwidth could become a very serious problem for us—if there is a delay of more than a second, it could have a bad impact on the validity of the test,” Yeagley says.
Future Planning
Sign up for the Accountability Alert newsletter and stay abreast of the most recent news coverage and ideas on accountability issues.
Michael K. Russell, an associate professor of education at Boston College who conducts research on testing, says that in the next few years, school leaders can expect to see improved forms of adaptive tests that will make them more diagnostic, giving teachers better clues to why students choose particular right or wrong answers.
Technology leaders should also be aware of the growing interest in the use of online assessments that are adapted for students with disabilities, whom federal mandates entitle to accommodations on state tests.
Various adaptations are “much more possible, but not in place yet,” Russell says, citing examples such as read-aloud tests, enlarged text or magnification, masking parts of test items that could be distracting to some students, glossaries to define terms, and switches and other writing devices for students with physical disabilities.
In the future, he says, schools will probably need to provide such accommodations not just when students are taking their state tests, but also when they are practicing for them, or taking other online assessments.
And that, undoubtedly, will require proper planning.
