Struggling algebra students in the Everett, Wash., school district get help from special tutors who diagnose their weaknesses, tailor instruction to their needs, and provide on-the-spot feedback—all with an inhuman degree of patience.
That’s inhuman literally: The tutors are computers. Three years ago, the district started employing Cognitive Tutor, a series of computer programs based on artificial intelligence that were developed by researchers from Carnegie-Mellon University in Pittsburgh. The programs provide an alternative form of math instruction to secondary school students who haven’t succeeded in regular classrooms.
The experience proved so successful that officials in the 20,000-student district have expanded the program. Now, all of Everett’s high school students have a choice in signing up for Algebra 1, Algebra 2, and geometry: a traditional class or one that mixes teacher-led lessons with a sizable dose of machine-based tutoring.
“I was a math teacher for 20 years,” said Mary Ann Stine, the district’s director of curriculum and instruction, “and the question I have to ask is: How come this is so much more successful than I was?”
Used in 1,500 districts nationwide, the Cognitive Tutor programs are the best-known of a small but expanding contingent of “intelligent tutors” that are making their way from the laboratory into college and K-12 classrooms.
Such artificial-intelligence-based programs now are on the market or in development for teaching chemistry, physics, foreign languages, reading, and computer science, among other subjects, and for grading essays.
“What distinguishes intelligent tutors from integrated learning systems or skill-building software is that the tutors sort of both scaffold and support more complex cognitive processes,” said Margaret Honey, the director of the New York City-based Center for Children and Technology. “Well-designed tutors are smart enough to know there’s not a single way to solve a problem, and that’s what makes them ‘intelligent.’ ”
No Rivals for Live Tutors
Since the 1970s, the National Science Foundation, the Pentagon, and the U.S. Department of Education have opened their wallets to seed research and development of intelligent-tutoring systems.
The agencies have put up the money, in part, because studies show that the systems are promising.
The Cognitive Tutor Algebra program, for instance, is one of just two middle school mathematics programs to get a “positive” effectiveness rating in 2004 from the What Works Clearinghouse, a federally funded effort to vet research on educational programs and practices.
Studies suggest that, on average, students who use Cognitive Tutor make learning gains that roughly translate into the equivalent of as much as one letter grade—the difference, in other words, between an A and a B.
The improvements seem greatest, the research also suggests, in the area of complex problem-solving and for students who start out with weaker math or English-language skills. Similar gains are being reported for other kinds of intelligent-tutoring systems as well.
Those kinds of improvements still don’t rival one-on-one lessons from a flesh-and-bones teacher, though. According to an influential 1984 study review, performance improves by the equivalent of about two letter grades when students get individual lessons from human tutors.
But the level of progress also depends on how skilled the tutor is, noted Kenneth R. Koedinger, a professor of human-computer interaction and psychology at Carnegie Mellon and one of the developers of Cognitive Tutor.
“Our goal isn’t to replace teaching,” said Mr. Koedinger, who also co-directs the Pittsburgh Science of Learning Center, a $25 million operation run jointly by the University of Pittsburgh and CMU that uses intelligent-tutoring systems to study learning. “It’s to give teachers more time to do what they do best.”
Most intelligent-tutoring systems are built as supplements to classroom lessons. Even in Cognitive Tutor’s full-course programs in algebra and geometry, students spend 40 percent of their time on computers and 60 percent in classrooms, where teachers use lessons developed by the Pittsburgh researchers.
“The better contrast to use might be a textbook,” said Mr. Koedinger. “With textbooks, students don’t get feedback on solutions.”
Projects in Pipeline
Cognitive Tutor was born out of work begun by John R. Anderson, a prominent cognitive scientist at Carnegie Mellon, in the 1970s. Mr. Anderson wanted to use the system to test a framework for modeling how advanced students process and learn information.
In 1998, though, Mr. Anderson, Mr. Koedinger, and their research partners had a string of products that were ready for mass distribution. They formed Carnegie Learning Inc., a Pittsburgh-based company, to promote Cognitive Tutor nationwide.
Meanwhile, other researchers were also experimenting with the technology and adding twists. Some of the other innovations in the production pipeline include:
Learn more about Project LISTEN and AutoTutor.
• Project LISTEN, a system developed by Carnegie Mellon researcher Jack Mostow that employs speech-synthesizing technology to “listen” to students read and give them feedback. The computer also assigns students new stories based on their progress.
Studies so far show that elementary school students who use the program in addition to their regular reading instruction for as little as 20 minutes a day show learning improvements equivalent to about half to two-thirds of a letter grade better than their regular-classroom counterparts.
• Online intelligent-tutoring systems in chemistry and applied mathematics that allow students to plug in their own homework problems, rather than rely on computer-generated tasks. Developed by Quantum Simulations Inc., of Murrysville, Pa., the chemistry programs, which are already on the market, have been tested with high school and college students.
The studies show that high school students who used the tutor at home were better at balancing chemical equations than were their peers who did not use the technology.
The AutoTutor system features an on-screen talking character that interacts with students. Developed mainly for university-level physics and computer-literacy studies, the program is now being tested with students in grades 8-11 and has been shown to boost learning by the equivalent of roughly one letter grade.
—Image courtesy of Institute for Intelligent Systems
• AutoTutor, a system developed by researchers at the University of Memphis, in Tennessee, and Carnegie Mellon that features an on-screen talking head that conducts a dialogue with students. Developed primarily for physics and computer-literacy studies at the university level, the program is now being tested with students in grades 8-11 as well.
“The goal is to help students reason,” said Arthur C. Graesser, the Memphis researcher heading that project. “When we compare it to just reading texts, we find that the AutoTutor helps by about one letter grade.”
Reading Body Language
An open question is whether the systems work as well in subjects whose subject matter is less specified, or more the focus of contention, than in math and science.
Experts also wonder how widespread such programs can become, given the time and expense they take to develop. Cognitive Tutor, for instance, was developed and refined over more than 20 years. Even now, researchers continue to tweak it.
“For every hour of instruction, there’s hundreds of hours spent in development,” said Mr. Anderson of Carnegie Mellon.
Yet Carnegie Learning has reached a point now where it can market its product for $30 to $60 per student and earn enough to keep devising new products, according to Steven B. Ritter, another of the company’s founders.
Researchers are also discovering, though, that intelligent-tutoring systems are as valuable as research tools as they are as teaching instruments. That’s because the systems can track student behaviors with a precision not available to classroom researchers, who rely on twice-a-year surveys or occasional classroom observations for their data.
With AutoTutor, Mr. Graesser says, researchers have embedded sensing devices to track students’ facial expressions, voice signals, and posture.
“We’re getting good at detecting boredom versus frustration versus confusion,” he said. “The more difficult the AutoTutor is, the better we get students to learn, and the less they like it.” That may not be so different, after all, from their experiences with human teachers.
