Although Carnegie Learning’s Cognitive Tutor Algebra 1 is the only blended learning curriculum Mary Brierley has ever taught, she trusts its quality.
“It’s made me change the way I question students, and it’s improved the way that we teach the course,” said Ms. Brierley, who teaches two block sections of Algebra 1 and one of Algebra 2 at Severna Park High School, located about a half-hour south of Baltimore. “I don’t know how, it just seems to force you to ask the question that makes the student think about how to get to the answer.”
In many respects, the curriculum used in the Maryland teacher’s classroom bears similarities to many blended learning programs, instructional approaches that combine technology-based and traditional classroom lessons.
While testimony such as Ms. Brierley’s has been enough to persuade hundreds of districts to use some form of the Cognitive Tutor program since the late 1990s, the curriculum is unusual in that it also now has the backing of independent research, stemming from a $6 million study financed by the U.S. Department of Education carried out by the Santa Monica, Calif.-based RAND Corp.
The two-year study—which included 25,000 students across middle and high schools—found that in the second year of the curriculum’s implementation, high school students using the curriculum’s combination of self-paced software and class-paced textbooks made statistically significant additional learning gains compared with students using a traditional curriculum. Middle school students were also found to make gains in the second year, though not to a statistically significant degree.
What is less clear is why the program works, which of its components are most effective, and whether its apparent success is a greater victory for the Pittsburgh-based Carnegie Learning’s corporate future, or the entire field of blended learning, which has seldom had a chance to prove itself in research of this magnitude.
“I do think that it has value in the blended learning community as [a measure of] proof that a strategy that incorporates technology in a blended fashion produces positive results,” said John Pane, a senior scientist at RAND and the study’s lead author. “But generalizing from that to other blended learning curricula is a risky move. And while many people may be tempted to do that, the devil is in the details of any blended learning curriculum.”
Software-Textbook Connections
When it comes to Cognitive Tutor, those details are not all that revolutionary, at least within the scope of the field of blended learning, which has gained most of its traction in the past half-dozen years.
Ms. Brierley’s Algebra 1 classroom, and many others that use the program, functions squarely within the commonly used “station rotation” blended learning model, which is seen more often in the elementary and middle grades.
After a brief pencil-and-paper warm-up, her second-period class divides into two groups of about a dozen students each. One group of students turns to a problem from a textbook, with clusters of students working together at desks, while members of the other group migrate to the laptop cart in the classroom’s corner, take a device back to their desk, log in to their Cognitive Tutor software accounts, and tackle problems tailored to each student’s learning progress. After 35 minutes or so, the groups switch tasks.
“It does free [teachers] up to be more of a troubleshooter than anything,” said Ms. Brierley, an 18-year teaching veteran who has spent the last third of her career working with Cognitive Tutor. “It gives [students] an opportunity to be independent and work through things and sometimes work things out in their head without us telling them what they should be doing.”
A study found that the math scores of high school students using Cognitive Tutor improved significantly during the second year that schools implemented the program. That growth was equivalent to moving a student from the 50th percentile to the 58th percentile of performance.
Source: RAND Corp.
But Cognitive Tutor has some notable nuances for a station-rotation model. Among them, both the print text and the software come from the same provider. So while some students may reach concepts in print first, and others first encounter them online, the terminology and theory behind teaching concepts remains constant.
Both branches of the curriculum also stress the manipulation of numbers and variables. The text features perforated tearaway pages so students scribble in or alongside charts and equations rather than on separate scrap paper. (This also means a district implementing the curriculum has the added expense of purchasing new textbooks every year.) The software requires students to set their own bounds for graphs and tables and type key information from paragraph-length word problems into charts before answering a series of questions all based on the same scenario.
Perhaps most importantly, the curriculum has undergone an endless evolution since its launch as Carnegie Learning’s first product in 1998 and is modeled around research on human cognition conducted by Carnegie Mellon University, in Pittsburgh, which at one time owned a stake in the company.
“It is built on a very well-considered theory of cognition and has been assembled over decades into what it is now,” Mr. Pane said. “That’s quite in contrast with the great vast set of education technology/blended learning interventions that are out there that are ad hoc.”
Academic Improvements
It’s worth noting that RAND restricted Carnegie Learning’s employees in their interaction with schools in the study, to ensure schools did not receive an unusual level of assistance in implementing the curriculum. In other words, the study suggests those second-year results stem from a range of real-world conditions, where some schools and some teachers are implementing the program in more effective ways than others.
That’s essential in a district like the 78,000-student Anne Arundel County, where every Algebra 1 classroom has used Cognitive Tutor since 2009, but not every teacher is as enamored with the program as Ms. Brierley.
While the expectation is for students to spend 30 to 40 minutes of class time twice a week working on the curriculum’s computer component, some teachers barely meet the minimum time requirement, said Amy Smith, the district’s coordinator of secondary mathematics. Others, meanwhile, allow more class time and encourage students to use the program during study halls or after school, she said.
Teachers’ responses range from “excellent fidelity,” Ms. Smith said, to “ ‘I’m checking the box—yep, I put them on’ ” the computer.
Inevitably, challenges related to classroom management or instructional-time allotment—especially in shorter-length middle school periods—force teachers to lock up laptop carts more than they’d like.
“It’s not perfect by any means,” said Ms. Smith.
Still, she insists Cognitive Tutor Algebra 1, first used as a supplemental tool by the district, is driving recent improvements on the Maryland High School Assessment for that subject. Between 90 percent and 93 percent of the county’s high school students have passed the graduation-requirement exam during each of the past six years, and during the past five, the proportion scoring “advanced” has risen steadily from 34.2 percent in 2009 to 41.6 percent in 2013. (No Anne Arundel County schools were included in RAND’s research.)
Paper vs. Computer
Julia Freeland, an education fellow at the Cambridge, Mass.-based Clayton Christensen Institute, which has led the effort to catalog and categorize blended learning, agrees that results both from the RAND study and real-world settings, such as in Anne Arundel County, are encouraging. But she said the field is still lacking granular research that shows the reasons that programs like Cognitive Tutor are effective—research that could help educators decide whether implementing the program is worth the investment.
Although the federal Education Department paid for the sizeable study, it may be more difficult to find a funder for follow-up work that examines those kinds of questions.
Given that reality, John Watson, the founder of Evergreen Education Group, a Colorado-based consulting organization that tracks online and blended learning trends, said he hopes the study is considered more broadly than just in terms of Carnegie Learning’s credibility.
“There are literally dozens of [blended] math programs,” he noted. “There’s no way that we’re going to have $6 million studies done about dozens of math programs individually. And so, while this is certainly a good thing for Carnegie and it’s a good thing broadly for the field, I want to make sure that it’s seen as part of the validation of the entire field as much, or more than being seen as suggesting a particular specific product.”
Ms. Smith is confident that the program is helping the 2,700 Algebra 1 students across the Anne Arundel County school system. Yet even six years after it was introduced, there are always new challenges, such as the first year of implementing the Common Core State Standards, which she fears could drag down achievement results.
Perhaps sensing that uneasy transition, Ms. Brierley’s students—most new to blended learning—appear mixed on whether they prefer doing algebra on a screen or in a book.
Celeste Davis, a freshman in Ms. Brierley’s first-period class, actually would like to see less of the laptops.
“I like working out on the paper and being able to figure the problems out,” she said. “I think the computer just doesn’t explain it the way that I read it.”
Antony Rutherford, a freshman in Ms. Brierley’s second-period class, said doing the work on a computer screen can be a bit more cumbersome at times.
“I’m kind of an all-around person,” he said. “I do it a little bit faster on paper. But on the computer, it’s fine.”
