Are curricula that emphasize conceptual understanding the best way to teach mathematics?
In the 14 years since the National Council of Teachers of Mathematics issued its standards for teaching the subject, a debate has raged over whether schools should follow its recommendations and emphasize conceptual understanding as well as performance of skills.
Curricula based on the national standards have been on the textbook market for six years, and studies now are starting to be published that shed light on the issue of whether the new path was a good one to take.
A panel convened last year by the National Research Council is evaluating the research to determine whether it is rigorous enough to draw conclusions about the effectiveness of the new math curricula. The committee is scheduled to release its report this year.
And the Bush administration is going to be evaluating curriculum in its five-year effort to raise the quality of the nation’s math and science education.
But many math education researchers say that studies published in professional journals and a recent book suggest that the new curricula are on the right track.
The research fails, however, to answer definitively the big question in the debate: Are curricula that emphasize conceptual understanding the best way to teach mathematics? Or should schools continue the traditional approach of teaching the rudimentary skills of the discipline before expecting students to apply them in real-life situations?
The research conducted to date isn’t specific enough to identify which kind of curriculum works best in schools.
“A superintendent of a large school district contacted me not long ago to ask if there was research that would help him select an elementary school math
curriculum that would be effective for the types of children served by his district,” Grover J. “Russ” Whitehurst, the director of the federal Institute of Education Sciences, told a U.S. Department of Education “summit” on math education this month.
“I had to tell him that there was no rigorous research on the efficacy of widely available elementary mathematics curricula, and that about all I could offer him was my opinion,” he said.
Supporters of the new curricula, though, say that enough research is showing positive results for schools to go ahead with the new programs while researchers define the ideal conditions for using them.
“I haven’t seen any evidence that they are failures and we should pull them out of schools,” says James Hiebert, a professor of education at the University of Delaware, located in Newark. “They are promising enough that we should pursue the implementation of them so we can collect long-term data.”
But critics maintain that the research hasn’t delivered nearly enough evidence to warrant schools’ switching to approaches that they say gloss over basic mathematical skills.
“I haven’t seen a study that really convinced me,” says Michael McKeowen, a co-founder of Mathematically Correct, the national parent group that is influential in organizing opposition to the new curricula, and a professor of medical science at Brown University.
Critics maintain that the research hasn't delivered nearly enough evidence to warrant schools' switching to approaches that they say gloss over basic mathematical skills.
The studies often compare the results from new curricula with those from control groups whose teachers didn’t receive the same professional development, McKeowen says. Students who have the chance to study under teachers with such preparation, he says, are likely to perform better than those who learn from less qualified teachers.
While advocates on both sides of the debate differ on the best way to improve math education, there is a consensus that U.S. student performance in the subject needs to improve.
Even though 4th and 8th graders’ scores on the National Assessment of Education Progress rose steadily during the 1990s, U.S. students have scored poorly on several international studies.
In 1996, for example, the Third International Mathematics and Science Study found that U.S. 4th graders performed above the national average, that 8th graders were in the middle of the pack, and that high school seniors fell below average.
A repeat of the study given to 8th graders in 1999 found the 8th graders again to be around the international mean. (“U.S. Students’ Scores Drop by 8th Grade,” Dec. 13, 2000.)
That disappointing record adds urgency to the challenge for researchers: What does work best in math classrooms?
Control Groups Difficult
One study examining the Interactive Mathematics Program, one of the high school curricula influenced by the NCTM standards, paints a picture of how math classes are changing in some American high schools—and gives an example of how difficult it is to draw conclusions from one project.
When freshmen started at a suburban Philadelphia high school in 1997, the study notes, their math classes were different from the classes that had preceded them.
Instead of just teaching algebra—the common 9th grade math subject—the new Interactive Mathematics Program, known as IMP, included pieces of geometry and statistics. Students would have waited until 10th grade or later to learn such subjects under the previous curriculum.
The teaching methods used in the new freshman course also differed markedly from the problem-solving with x’s and y’s representing mythical variables that had dominated the high school’s Algebra 1 course before 1997. Now, teachers offered examples of problems from real-life situations. Students might be asked to find the length of shadows or graph the population growth of the West during the 19th-century migrations.
That same class of 9th graders went through two more years with the same curriculum, along the way supplementing their knowledge of algebra, geometry, and statistics with trigonometry, basic calculus, and other areas of the discipline that traditionally were taught in separate high school courses.
By the time the group reached the end of 11th grade, the researcher who tracked them found that they outperformed a similar group of students, two years ahead of them, who had studied under a traditional curriculum. What’s more, the students in the IMP group, as a whole, had taken more math classes and enrolled in more Advanced Placement courses than those ahead of them.
“It’s a piece of evidence pointing toward a positive effect of the curriculum,” says Steven L. Kramer, who conducted the study as his doctoral dissertation at the University of Maryland and is preparing the results for submission to peer-reviewed journals.
What the research literature lacks, say critics of the new curricula, is studies comparing student achievement under two specific curricula.
Kramer adds that it’s hard to identify whether the curriculum itself was instrumental in raising student achievement or whether it was one of several ingredients needed.
The students who learned under the Interactive Mathematics Program also studied in a block schedule, in which they attended math classes for 90-minute periods over a semester. Their predecessors had attended 45-minute classes that lasted over a whole school year. Kramer’s study was unable to conclude whether IMP would have succeeded under the previous schedule.
Also, the school, which Kramer has not named, decided to switch to IMP with the full support of the math faculty. The teachers all were given training in how to teach the new curriculum.
Would the program have succeeded with a reluctant and unprepared staff of teachers? Common sense and other research suggest not.
That’s why Kramer and others suggest that his research needs to be viewed along with other studies with similar results to decide the effectiveness of the curriculum and the best circumstances for using it.
While studies such as Kramer’s are common, critics of the NCTM standards and the curricula based on them say the research doesn’t offer any evidence that the math innovations are working.
In such studies and others like it, the critics say, the teachers and students are aware that they are research subjects—an awareness that gives them an incentive to perform well.
“These are weak designs because the very act of volunteering to use a new curriculum typically carries with it extra motivation to succeed, and thus biases the results towards the new curriculum,” Whitehurst, the Education Department’s research chief, argued at the math summit.
What the research literature lacks, according to Whitehurst and critics of the NCTM standards and the curricula based on them, is studies comparing student achievement under two specific curricula.
“They should exist, but they don’t exist,” says Bastiaan J. Braams, a research associate professor of mathematics at New York University and a critic of the NCTM.
The research on the new curricula also is hampered, Braams says, by the fact that it has been conducted by people involved in designing the programs or who are supportive of them.
“In general, I have a very low opinion of the research,” he says. “I see it as advocacy research.”
‘Promising and Substantial’
But supporters of the NCTM approach contend that both the quality and quantity of research they are producing provide evidence that the changes are having a positive impact.
In Standards-Based School Mathematics Curricula: What Are They? What Do Students Learn?, published this year, researchers published a collection of 13 studies that they say supports the new curricula.
“The evidence at hand is both promising and substantial,” Jeremy Kilpatrick, a professor of mathematics education at the University of Georgia, in Athens, writes in the book’s concluding essay. “It will not quiet the critics ... but it should encourage those who welcome improvements in school mathematics and understand the difficulty of evaluating something as complex as a curriculum.”
Supporters of the new approach contend that both the quality and quantity of research they are producing provide evidence that the changes are having a positive impact.
In three studies of high schools using IMP, for example, Norman L. Webb found that students who studied under the curriculum performed as well on standardized tests and the SAT as students in a control group. He also found that students using IMP outperformed others when presented with tests that required them to use problem-solving rather than basic skills. They also took more math classes than did students studying a traditional approach, he says.
“We were able to show that the curriculum did what it said it was going to do,” says Webb, a senior research scientist at the Wisconsin Center for Education Research at the University of Wisconsin-Madison.
But it’s still difficult for him to declare IMP a victor in a competition between mathematics curricula, because true control groups are hard to maintain. In Webb’s study, he found that students would transfer from IMP to classes using the traditional curriculum, depending on their interests. Teachers who taught the traditional way started to use techniques, such as small-group projects, that are a common approach in IMP. They also altered what they taught to match the tests they knew researchers would be evaluating.
“It’s very complicated to come up with a good evaluation design,” Webb says.
McKeowen, the co-founder of Mathematically Correct, says that such complications make it tricky to reach definitive judgments about the efficacy of the new curricula.
He agrees that any study that tries to create a separate control group is bound to have the types of problems that Webb cites. The situation is unlike that in a drug study, he point outs, where participants do what they’re told because they don’t know whether they’re taking a placebo or the drug that’s being tested.
New Federal Interest
What is clear is that the debate over research into the new curricula is about to heat up.
The Education Department on Feb. 6 launched an initiative to highlight the need for improved mathematics instruction. Raising the quality of research in the field is a primary objective in the five-year effort. At the mathematics summit in Washington, department officials proposed to spend $120 million in the next fiscal year to conduct research to determine best practices for teachers, as well as to compare curriculum against one another. (“Ed. Dept. Proposes $120 Million Math Agenda,” Feb. 12, 2003.)
What will it take, then, to determine the effectiveness of the new curricula?
‘Expecting a firm or solid proof [in math education] is unrealistic. It's unrealistic in a lot of other areas that people accept. We should expect the same thing in education.’
Kramer, the researcher who studied students at the suburban Philadelphia high school, compares his study to a piece of evidence introduced in court. Alone, the evidence isn’t enough to prove guilt or innocence. But viewed along with other evidence, it can contribute to delivering a verdict, he says.
His study of the Interactive Mathematics Program gives hints, he says, about under which conditions students learn most effectively that are, in turn, supported by similar research.
Hiebert, the University of Delaware professor, suggests that educators and mathematicians need to live with the ambiguity of the research, just as people do in other fields. Recommendations on diet and exercise, he says, are always being updated based on new studies. And when new nutrition standards are introduced—such as suggestions to eat five fruits and vegetables every day—they are guidelines rather than hard-and-fast rules guaranteed to produce better health.
“Expecting a firm or solid proof [in math education] is unrealistic,” Hiebert says. “It’s unrealistic in a lot of other areas that people accept. We should expect the same thing in education.”
The Research section is underwritten by a grant from the Spencer Foundation.
