People in California are worried about the mathematical performance of their children. So worried that debates about the “right” way to teach mathematics have become so metaphorically bloody they have been termed the “math wars.” Much of this concern has stemmed from students’ performance on state and national tests, but nobody seems to have stopped at any point to question the value of the type of knowledge assessed on these tests. I would like to halt the debate for a moment in order to pose this question: Is success on a short, procedural test the measure we want to adopt to assess the effectiveness of our students’ learning? In other words, do these tests assess the sort of knowledge use, critical thought, and reasoning that is needed by learners moving into the 21st century?
One of my concerns in this area is that the current debate about standards assumes there to be one form of knowledge that is unproblematically assessed within tests. This is despite the fact that a large body of research from psychological and educational fields shows the existence of different forms of knowledge. There is also increasing evidence that students can be very successful on standard, closed tests with a knowledge that is highly inert and that they are unable to use in more unusual and demanding situations (such as those encountered in the workplace).
Test knowledge, in other words, is often the sort of knowledge that is nontransferable and is useful for little more than taking tests. To demonstrate what I mean, I would like to describe the results of a research project that monitored the learning of students who experienced completely different mathematics teaching approaches over a three-year period. In response to these different approaches, the students developed different forms of knowledge and understanding that had enormous implications for their effectiveness in real-world situations.
Two schools in England were the focus for this research. In one, the teachers taught mathematics using whole-class teaching and textbooks, and the students were tested frequently. The students were taught in tracked groups, standards of discipline were high, and the students worked hard. The second school was chosen because its approach to mathematics teaching was completely different. Students there worked on open-ended projects in heterogeneous groups, teachers used a variety of methods, and discipline was extremely relaxed. Over a three-year period, I monitored groups of students at both schools, from the age of 13 to age 16. I watched more than 100 lessons at each school, interviewed the students, gave out questionnaires, conducted various assessments of the students’ mathematical knowledge, and analyzed their responses to Britain’s national school-leaving examination in mathematics.
At the beginning of the research period, the students at the two schools had experienced the same mathematical approaches and, at that time, they demonstrated the same levels of mathematical attainment on a range of tests. There also were no differences in sex, ethnicity, or social class between the two groups. At the end of the three-year period, the students had developed in very different ways. One of the results of these differences was that students at the second school--what I will call the project school, as opposed to the textbook school--attained significantly higher grades on the national exam. This was not because these students knew more mathematics, but because they had developed a different form of knowledge.
At the textbook school, the students were motivated and worked hard, they learned all the mathematical procedures and rules they were given, and they performed well on short, closed tests. But various forms of evidence showed that these students had developed an inert, procedural knowledge that they were rarely able to use in anything other than textbook and test situations. In applied assessments, many were unable to perceive the relevance of the mathematics they had learned and so could not make use of it. Even when they could see the links between their textbook work and more-applied tasks, they were unable to adapt the procedures they had learned to fit the situations in which they were working.
| Is success on a short, procedural test the measure we want to adopt to assess the effectiveness of our students’ learning? | 
|
The students themselves were aware of this problem, as the following description by one student of her experience of the national exam shows: “Some bits I did recognize, but I didn’t understand how to do them, I didn’t know how to apply the methods properly.”
In real-world situations, these students were disabled in two ways. Not only were they unable to use the math they had learned because they could not adapt it to fit unfamiliar situations, but they also could not see the relevance of this acquired math knowledge from school for situations outside the classroom. “When I’m out of here,” said another student, “the math from school is nothing to do with it, to tell you the truth. Most of the things we’ve learned in school we would never use anywhere.”
Students from this school reported that they could see mathematics all around them, in the workplace and in everyday life, but they could not see any connection between their school math and the math they encountered in real situations. Their traditional, class-taught mathematics instruction had focused on formalized rules and procedures, and this approach had not given them access to depth of mathematical understanding. As a result, they believed that school mathematical procedures were a specialized type of school code--useful only in classrooms. The students thought that success in math involved learning, rehearsing, and memorizing standard rules and procedures. They did not regard mathematics to be a thinking subject. As one girl put it, “In math you have to remember; in other subjects you can think about it.”
The math teaching at this textbook school was not unusual. Teachers there were committed and hard-working, and they taught the students different mathematical procedures in a clear and straightforward way. Their students were relatively capable on narrow mathematical tests, but this capability did not transfer to open, applied, or real-world situations. The form of knowledge they had developed was remarkably ineffective. At the project school, the situation was very different. And the students’ significantly higher grades on the national exit exam were only a small indication of their mathematical competence and confidence.
The project school’s students and teachers were relaxed about work. Students were not introduced to any standard rules or procedures (until a few weeks before the examinations), and they did not work through textbooks of any kind. Despite the fact that these students were not particularly work-oriented, however, they attained higher grades than the hard-working students at the textbook school on a range of different problems and applied assessments. At both schools, students had similar grades on short written tests taken immediately after finishing work. But students at the textbook school soon forgot what they had learned. The project students did not. The important difference between the environments of the two schools that caused this difference in retention was not related to standards of teaching but to different approaches, in particular the requirement that the students at the project-based school work on a variety of mathematical tasks and think for themselves.
When I asked students at the two schools whether mathematics was more about thinking or memorizing, 64 percent of the textbook students chose memorizing, compared with only 35 percent of the project-based students. The students at the project school were less concerned about memorizing rules and procedures, because they knew they could think about different situations and adapt what they had learned to fit new and demanding problems. On the national examination, three times as many students from the heterogeneous groups in the project school as those in the tracked groups in the textbook school attained the highest possible grade. The project approach was also more equitable, with girls and boys attaining the different grades in equal proportions.
It would be easy to dismiss the results of this study because it was focused on only two schools, but the textbook school was not unusual in the way its teachers taught mathematics. And the in-depth nature of the study meant that it was possible to consider and isolate the reasons why students responded to this approach in the way that they did. The differences in the performance of the students at the two schools did not spring from “bad’’ teaching at the textbook school, but from the limitations of drawing upon only one teaching method. To me, it does not make any sense to set any one particular teaching method against another and argue about which one is best. Different teaching methods do different things. We may as well argue that a hammer is better than a drill. Part of the success of the project school came from the range of different methods its teachers employed and the different activities students worked on.
Some proponents of traditional teaching want students to follow the same textbook method all of the time. A few students are successful in such an approach, but the vast majority develop a limited, procedural form of knowledge. This kind of knowledge may result in enhanced performances on some tests, but the aim of schools must surely be to equip students with a capability and intellectual power that will transcend the boundaries of the classroom.
Jo Boaler is an assistant professor of education at Stanford University in Stanford, Calif. Her book Experiencing School Mathematics received the Outstanding Book in Education award in the United Kingdom in 1997. She can be reached by e-mail at joboaler@stanford.edu.
