Under pressure to raise test scores, math teachers at San Marcos High School decided to do something different this school year.
Setting aside their 7-year-old textbooks, teachers filled the void largely with an online math curriculum, called Agile Mind, that comes equipped with an array of assessment tools. The idea was to get a better read on what their students know and what they should target in planning their classroom instruction.
- • Palm Readers:
Teachers in a rural New Mexico district use hand-held computers to assess students’ reading progress and target instruction accordingly. - • Critical Thinking:
Schools in Boston are gaining insight into their students’ thought processes through a locally developed reading-assessment system.
MAY 2
- • New Math:
Teachers in a Texas school have shelved some of their math textbooks and turned to an online curriculum and formative-assessment resource instead.
MAY 9
- • Teacher Training:
A California center is using formative assessment to help beginning teachers reflect on and improve their own work and that of their students.
MAY 16
“A big push is to get away from instruction being textbook-driven, to needs-driven,” said Joy H. Philpott, the academic dean of the 1,900-student school here, some 30 miles south of Austin. School officials had learned about Agile Mind in a sales call from representatives of the company, which is based in Grapevine, Texas, near Fort Worth. The visit came after last spring’s state tests, known as the Texas Assessment of Knowledge and Skills, or TAKS, but before the scores were known.
When those results came back, it became clear that the high school had missed its “adequate yearly progress” target in mathematics under the federal No Child Left Behind Act, one year after missing AYP in reading. Even though the school had missed the math target by only one student, “we took a huge PR hit on that,” Ms. Philpott recalled. “That was horrible.”
In the aftermath, officials thought back to Agile Mind. The company had left behind two trial subscriptions to the curriculum service, which Ms. Philpott had given to Walter W. “Pete” Darnall, the math department head, and another teacher. They had been impressed by the topic overviews, online animations, and other features that illustrate mathematical principles—such as a cartoon skateboarder who demonstrates how measurements of time and distance on a graph can represent velocity.
“Both of us really thought it was a way to capture students’ attention,” Mr. Darnall said.
So the school bought a year’s worth of the Agile Mind service for 1,200 students, in Algebra 1 and 2 and geometry, at a cost of $40,000.
Teachers Impressed
Agile Mind, a curriculum product in high school math and science, offers teachers multiple ways to use “formative assessment,” a term generally used to describe classroom measures to give teachers and students feedback for adjusting instruction as it is taking place during the school year.
Many of those assessments are woven through the curriculum, which the company has verified as meeting all the goals on the TAKS, as well as the Texas Essential Knowledge and Skills, the state’s academic standards. The company also offers a test bank of 2,500 algebra items that are modeled on the TAKS items.
The online Algebra 1 curriculum offered by the company Agile Mind employs interactive graphics to help students understand such concepts as velocity.
—Courtesy of Agile Mind
The interwoven assessments include items for open, or student-constructed, responses; multiple-choice items that can be administered and scored online; and guided assessments and self-tests, complete with hints and feedback, telling students if they got the questions right or wrong.
The online curriculum and the related assessments can be made available to students using any Internet connection, though teachers can limit students’ access to specific assessments and topics. About 80 percent of San Marcos High students have Internet access at home, Mr. Darnall said; teachers pass out printouts of relevant materials if access is a problem.
Local Experts Tapped
The curriculum and assessments were developed with the aid of the nonprofit Dana Center at the University of Texas at Austin, which is well known in the state for its expertise on Texas’ state tests and standards.
“Because it’s linked with the Dana Center, it’s been blessed,” Ms. Philpott said of Agile Mind.
Teachers use the items from the Agile Mind test bank to create mini-tests, quizzes, benchmark tests, or final assessments. The tests can all be administered online, then scored and analyzed, and compared against the scores of other students in the state.
But San Marcos teachers have, this year at least, preferred to print the tests and give them to students to complete with pencils.
One sign of success is that teachers have actually used the Agile Mind curriculum extensively this school year, rarely if ever asking their students to open their math textbooks, several teachers said in recent interviews.
The teachers know that the service tracks and record their usage, as well as that of their students. But Ms. Philpott and Mr. Darnall said they deliberately used a light touch to nudge teachers forward, so that they would be more apt to use the resources enthusiastically.
Even though teachers have considerable discretion in how, and how much, to use Agile Mind, the potential for overuse of the system has been more of a concern than underuse, school and company officials say.
An Experiment
Most math teachers at the school said they used the curriculum as their main instructional resource this school year, introducing new topics with Agile Mind’s topic overviews, which they project onto classroom screens with digital projectors.
Teachers also hand out quizzes that can be whipped together from the item bank—either open-ended problems that provide depth but are time-consuming to grade, or multiple-choice items that can be scored quickly.
As a department, the math teachers decided to use multiple-choice assessments to create the graded tests every six weeks that serve as benchmarks for mastery of critical math topics.
But after the first six-week test, Mr. Darnall and some of the other teachers suspected that students were guessing at answers rather than actually solving the problems. If that were the case, they reasoned, the benchmarks would be useless in deciding what math concepts the teachers should focus on. So Mr. Darnall decided to do an experiment.
For the November test, all the Algebra 1 students were given a dozen multiple-choice items—but without the choices. Instead of blackening bubbles, they had to calculate the answers themselves.
The next day, the teachers handed the students the same test again—with the identical questions and four possible answers for each question.
“The scores the second day, across the board, were worse,” confirming the guessing theory, Mr. Darnall said. In his view, the experiment raised a yellow flag about the use of multiple-choice tests as benchmarks.
Teachers also used the results of their experiment to show students that they had scored significantly better by working out the problems rather than guessing.
It’s far from certain that students will remember that lesson when confronted with the state test, which does offer multiple choices, Ms. Philpott said. She said the results should be back by mid-May.
The teachers, for their part, said they plan to keep using the same dual testing strategy periodically.
Creative Applications
Individual teachers at San Marcos High have come up with their own innovative assessments, too.
First-year algebra teacher Monica J. Sustaita, for example, has used Agile Mind “constructed response” items in her Algebra 2 classes as the subject of poster projects every six weeks.
Her students are given a week to present a single, knotty problem on a large poster, including multiple representations of the solution—as a graph, an algebraic transformation, and verbal description, as well as a description of a real-life application of the formula.
“It ties together the topics we do,” said Ms. Sustaita. “It’s a kind of assessment.”
An added benefit, she said, is that making the poster draws out students’ creative talents. “I have a lot of very artistic people; this gives them a chance to shine,” Ms. Sustaita said.
Yet she also said that she has needed to balance such time-intensive projects with multiple-choice quizzes that are easy to evaluate.
Other teachers have combined the Agile Mind assessment items with other methods; for example, by having students work out problems with their classroom sets of TI-83 calculators. A system called the Texas Instruments Navigator allows teachers to see each calculator screen on their laptop computers, allowing the teachers to get a quick overview of how students are faring.
Some teachers have had criticisms of Agile Mind, Mr. Darnall said. He himself has the same quibble that he has with the schools’ math textbooks: “Sometimes it seems to get too hard, too quick. That is a problem for some of the new teachers.”
As an experienced teacher, he said, he knows to replace some test items with suitable ones from other sources. To help the new teachers, the math department has same-subject teacher groups that meet every other week, in which veteran and new teachers compare notes.
And in the first year at least, the self-tests, which teachers assigned students as a review tool, were a bust; students simply didn’t complete them.
“They don’t have the habit [of doing self-tests]; that may be a San Marcos problem,” Mr. Darnall said, referring to academically average students. His pre-AP students, on the other hand, took similar online self-tests enthusiastically, he said.
Usage Measured
Another problem, potentially, is overuse of the service, even according the company. Mary Jane Smith, the Agile Mind consultant who helped train the San Marcos teachers and visits the school monthly, said she worries that for “a couple of teachers, their usage is way too high.”
“From my history of working with schools, what I want a teacher to do is use it wisely, not clicking through every doggone screen they have,” Ms. Smith said. “I love to see when teachers are being very specific about what they’re doing.”
Experienced teachers, however, generally have tapered off their use of Agile Mind to about half of their class time, Mr. Darnall said. They have augmented the service with other resources and activities, but not with a return to the textbooks.
The teachers will get another chance to work out the kinks, as the school has already decided to renew the Agile Mind service for next year. “Success of a program is hard to gauge with just one year,” Ms. Philpott noted.
Ms. Smith said that San Marcos High made a degree of progress with Agile Mind that she has yet to see in other schools that have used it for years. Leadership is the reason, she said, citing what she sees as a careful rollout that respected the judgments and learning needs of the school’s math teachers. She gives a gold star to Ms. Philpott in particular.
For her part, Ms. Philpott said she wouldn’t relax until the results of the math assessment arrived this month.
“If you call me then, you’ll find me at a bar,” she said, without revealing whether she expected to be celebrating.
