New research on technology’s effectiveness in teaching math appears to confirm what many educators have optimistically suspected: Computers can raise student achievement and even improve a school’s climate.
But they have to be placed in the right hands and used in the right ways, says Harold Wenglinsky, an associate research scientist at the Princeton, N.J.based Educational Testing Service who carried out the analysis for Education Week.
In fact, used for the wrong purposes, Wenglinsky says, computers appear to do more harm than good.
“Technology, indeed, can have positive benefits,” he says. “But those benefits depend on how the technology is used.”
His findings are welcome news to school systems under increasing pressure to justify the nation’s substantial investment in education technology—now estimated at more than $5 billion a year.
But they also present a major challenge to policymakers looking for a good return on that investment. Too often, the study finds, the tool is being used for the wrong job.
Until now, most research on technology’s effectiveness has taken the form of small case studies, some of which examined just a classroom or two at a time.
While several of these studies found evidence that technology can improve student achievement, researchers have been wary of presuming that the same results could be replicated in other classrooms where teachers might be less motivated or knowledgeable about computers.
Wenglinsky breaks new ground by analyzing a national database of student test scores, classroom computer use, and other information, including school climate.
His study, “Does It Compute? The Relationship Between Educational Technology and Student Achievement in Mathematics;' is based on the performance data of 4th and 8th graders who took the math section of the 1996 National Assessment of Educational Progress. The study will also be published as an ETS policy information report, and is available at www.ets.org/research/pic.
Administered by the U.S, Department of Education, NAEP-known as “the nation’s report card"has tracked student achievement for nearly three decades. For the first time in 1996, NAEP asked students and teachers additional questions about how they use computers in math.
After factoring out the influence of several other variables that affect achievement, such as students’ socioeconomic status, class size, and teacher qualifications, Wenglinsky found strong links between certain kinds of technology use, higher scores on NAEP, and an improved school climate. In every case, the gains were greater at the middle school level than in elementary school.
Among his findings:
Many education experts see the findings as validating their long-held beliefs about what works and what doesn’t when it comes to school technology.
“We didn’t know this,” Douglas H. Clements, an education professor at the State University of New York at Buffalo, says of the findings. “But we had some hints from more qualitative and smaller studies. So to have a large national study with a lot of the controls that this kind of analysis allows is quite a striking confirmation.”
Critics and some skeptics have long argued that computers add nothing to the education process, and may even be a distraction.
At the same time, technology’s greatest proponents have portrayed computers as a potential savior of American education, with the power to increase student learning and stimulate widespread reforms in teaching practices and the way schools are structured.
Wenglinsky’s research finds elements of truth in both views. What matters most, it suggests, are not the machines and the wiring themselves, but what teachers and students do with them.
“People are getting beyond the idea that this thing is magic, and that, like fire, just by sitting near it, you can get some benefit,” says Christopher J. Dede, a professor of education and information technology at George Mason University in Fairfax, Va.
The benefits seem to increase as the use of the technology becomes more sophisticated.
Eighth graders whose teachers reported using computers primarily for drill-and-practice scored lower on NAEP-by more than half a grade level-than students whose teachers reported other primary uses of computers. Students whose teachers concentrated on simulations and applications, meanwhile, scored higher by two-fifths of a grade level.
A simulation can illustrate relationships for students and allow them to test the effects of changing variables; an application, such as a spreadsheet program, lets students manipulate and analyze data.
Both uses are aimed at developing higher-order thinking skills, which researcher Barbara Means of SRI International describes as being “based on multiple mental steps rather than memory and rote procedure.” Means works at the Center for Technology in Learning at the Menlo Park, Calif.-based research group.
One example of this use of technology is a unit of instruction designed by researchers at the University of Massachusetts Dartmouth that allows students to test the concept of velocity, something most children understand intuitively but often have difficulty learning to quantify.
The program shows the up-and-down movement of an elevator alongside a graph of its changing speed in floors per second. Then, it asks where the elevator would end up based on a new graph, demonstrating that a lower velocity does not mean the elevator is going down, just that its speed is slowing. Finally, it asks students to create a new graph that makes a second elevator wind up at the same floor at the same time, but after traveling at different velocities.
“The computer’s most powerful uses are for making things visual,” says James Kaput, a math professor who led development of the unit. “It can make visual abstract processes that are otherwise ineffable.”
Wenglinsky’s research found no discernible effect of using computers primarily for simulations and applications among 4th graders, probably because there simply aren’t that many teachers who regularly use technology for this purpose with young students.
But Wenglinsky did find that 4th graders whose teachers used instructional computers mostly for math/learning games posted an achievement gain equal to roughly 15 percent of a grade level.
Over the past decade, elementary school teachers have seen an explosion in the number of learning games available on the market, and many of these products have moved beyond computerized versions of worksheets, says Clements of SUNY at Buffalo.
“Students are motivated by interacting with programs when they can influence the outcome of the activity,” he says. “And games have the benefit of providing immediate feedback.”
Clements believes the best feedback is that which is also instructive. He points to a simple program that teaches fractions by asking students to help a turtle eat a berry that sits between the reptile and a wall. When the student enters a fraction, the turtle moves that distance toward the wall, demonstrating the result of changing the numerator and the denominator.
That kind of visual response, Clements believes, is more effective than mere bells and whistles that reward a correct answer.
The same factors that were tied to student achievement gains were also related to a better school climate, Wenglinsky found.
In other words, where teachers used computers for more sophisticated activities than drill-and-practice, school officials were more likely to report higher teacher and student attendance, less tardiness, and better morale.
Researchers who have observed such results firsthand cite several possible reasons. One is that computers provide instantaneous, nonjudgmental feedback, a characteristic that can be especially beneficial to students with low self-esteem. Computers also can perform routine computations, giving students more time to wrestle with larger concepts.
“Technology certainly can be highly motivating,” says Means of SRI International. “Teachers see it in their kids’ interest. They see them come in at lunch and after school to work with it.”
Technology also can liberate teachers from the traditional lecture-style of instruction, by encouraging them to act instead as coaches and facilitators.
“One of the real benefits of different types of technology is the way they influence how teachers and students relate to each other,” Wenglinsky says.
In fact, a growing number of education technology advocates argue that this “constructivist” approach toward learning-in which students work in rich environments of information and experience, often in groups, and build their own understandings about them-taps into the computer’s greatest strengths.
“Kids learn by doing, by presenting, by displaying, by engaging,” says William Fi ke, the educational technology specialist at the Rhode Island department of education. “Learning happens best when the youngsters are doing the heavy lifting.”
Unfortunately, most of the nation’s schools aren’t using computers in ways that Wenglinsky’s finding indicate are linked to better scores. And survey responses from the 1996 NAEP also raise serious questions about technology’s role in closing the achievement gap between disadvantaged students and their peers.
Over the past decade, tremendous resources have been committed to the cause of ensuring that schools in disadvantaged communities have computer equipment and network connections on par with those serving more affluent populations. Survey results show, for example, that black students use computers in learning math somewhat more often than white students.
But the survey paints a very different picture about how those students use their school ‘computers.
At the 8th grade level, about 31 percent of white students used computers mostly for simulations and applications, compared with just 14 percent of black students. At the same time, more than half of America’s black students had teachers who used computers mostly for drill-and-practice, compared with only 30 percent of white students.
In short, black students have closed the digital divide where it matters least-the amount of time on a computer. The gap persists where it matters most—how the computer is used.
“It’s the low-income communities that have invested the most in technology for drill-and-practice,” says Margaret Honey, the deputy director of the New York City-based Center for Children and Technology. “But if this research says those investments don’t have the kind of impact we want them to have, then that’s an important message.
“There’s an expectation among many urban administrators that kids from disadvantaged backgrounds need basic skills and nothing else,” she adds.
Still, even some experts who praise more sophisticated uses of education technology say it may be too soon to throw out drill-and-practice altogether, especially in the early grades.
Using computers to practice basic skills, they say, may have benefits that don’t show up on NAEP scores. Several studies have suggested that computer-based exerci es can raise student motivation levels, especially for children who haven’t found other media engaging. Many education experts also see drill-and-practice as a useful way to build the skills needed to advance to the next level.
“A lot of people do not understand that drill-and-practice does not teach students anything,” says Ted S. Hasselbring, the co-director of Vanderbilt University’s Learning and Technology Center in Nashville, Tenn. “It develops fluency of an existing skill.”
The study offers a more encouraging result in the area of professional development.
Teachers who had received any amount of professional development with computer within the past five years were more likely to use computers in ways that Wenglinsky found to be effective than teachers who had not received such training.
In addition, 8th graders whose teachers had had technology training performed more than a third of a grade level better than those with teachers who lacked such training. The training also was linked to gains at the 4th grade level, although the effect was indirect and the difference in scores was smaller.
“The teacher has always been the key to determining the impact of innovation,” says Elliot Soloway, who holds professorships in the schools of education and engineering at the University of Michigan. “It’s not surprising that technology falls into this camp.”
School systems appear to be waking up to the important role played by teacher training. Survey results from the 1996 NAEP show 81 percent of the nation’s 4th graders had teachers who had received professional development with computers within the past five years, and 76 percent of 8th graders had math teachers who had received such training within the past five years.
But the survey says little about the quality or amount of that training; it could be a one-time workshop on a piece of software or ongoing, one-on-one assistance in the classroom. And given teachers’ frequent complaints about the quality of professional development in general, many education experts believe that most technology training likely leaves plenty to be desired.
Wenglinsky, in fact, says he was struck by the fact that any amount of professional development translated into student achievement gains. “Which leads one to think that more elaborate training might post even greater gains,” he adds.
Despite his findings. Wenglinsky says that much about the computer’s role in raising achievement remains unknown.
Because the study is based on a single year of data, it can’t prove effects over time. It also can’t say for certain whether certain uses of computers are causing higher student achievement, or vice versa.
Means, for instance, has observed that teachers often use games not to teach new math skills, but as a reward for those students who complete their assignments early.
“The major limitation of the study is that the direction of causality is not established,” Means says. “It’s going to be very difficult to tease apart how much that technology adds to the quality of instruction with this study. But a more qualitative description can show classroom interactions that would just not be possible without the technology.”
She also notes that the study concerns only one academic subject.
“The data might look very different with reading comprehension,” Means says. “So I’d be a little bit cautious before drawing many public policy implications.”
Finally, the NAEP survey didn’t allow Wenglinsky to compare the effectiveness of computers with that of other educational tools, some of which might be much less expensive.
While leaving room for further research, Wenglinsky says his study does tell educators something they’ve been waiting a long time to hear.
“What we do know for certain,” he says, “is that when teachers use the computer to teach higher-order thinking skills, students benefit.”
