New Center Asks: Does Merit Pay Work?

In the 1980s, school districts dabbled with programs that offered teachers cash inducements, such as bonuses or raises, for doing their jobs well.

But those merit-pay programs were mostly short-lived, hotly debated, and understudied. Even after all this time, no one knows definitively whether children learn more when teachers are paid extra for boosting their students’ achievement.

Now “pay for performance” is in vogue again, with programs operating in Florida, Minnesota, Texas, and Denver, among other places. This time, though, a group of researchers at Vanderbilt University is seizing the chance to put those initiatives under the microscope.

Learn more about the National Center on Performance Incentives.

With a five-year, $10 million grant from the U.S. Department of Education’s Institute of Education Sciences, the newly established National Center on Performance Incentives has put together an ambitious agenda to study such efforts in Nashville, Tenn., across the state of Texas, and in two other locations yet to be named.

Center investigators say their Nashville project is the first randomized experiment in the United States to test the merit-pay idea, although field trials in Chile, Kenya, and Israel have done so, with mixed findings.

In the end, the center’s researchers hope to shed light on whether teachers behave differently when the prospect of bonuses is dangled before them, whether student achievement improves as a result, and whether the existence of such programs will ultimately attract a different mix of teachers into the field.

“We really have no ideology,” said Matthew G. Springer, the center’s director and a research assistant professor of public policy and education at Vanderbilt, which is in Nashville. “We just want to provide an answer and end this two-decade-old debate.”

The gem in the center’s crown of studies is the experiment it has set up in the 74,000-student Nashville public schools. As part of the project, which got under way last fall, mathematics teachers in the city’s 36 middle schools were invited to join a pilot program in which teachers could earn bonuses of up to $15,000 a year, over the next three years, for the gains their students make on state exams.

Gary W. Ritter, a researcher from the University of Arkansas at Little Rock, said his plan to conduct a similar experiment in the Arkansas capital had to be scuttled when the local teachers’ union opposed the idea. “I can’t imagine too many places where the idea is not controversial,” he said. “I’m quite surprised Vanderbilt was able to do it, and do it as well as it did.”

Union on Board

Despite pay-for-performance programs’ resurging popularity, some skeptics question whether money can ever be a major motivator for teachers, many of whom see the profession as more of a calling than a job.

Teachers also worry that such rewards will be doled out unfairly, create division in the teachers’ lounge, or deflect dollars from across-the-board raises.

“When you talk about performance pay in corporate America, people are getting base salaries on top of salaries that are already high,” said Jamye Merritt, the president of the Metropolitan Nashville Education Association. “We’re not against performance pay, but we would also like teachers’ salaries to be higher.” Starting salaries for the district’s teachers are about $33,000 a year.

For years, the Nashville union also opposed efforts by the district school board and local business leaders to pilot merit-pay programs in the city. Yet 70 percent of the union members voted to back the Vanderbilt plan, which was also endorsed by Nashville Mayor Bill Purcell, the school board, and the business community.

What made the difference for the union this time, Ms. Merritt said, was the strong research component in Vanderbilt’s plan.

“I think it will benefit everyone to once and for all know if this will make a difference,” she said. “I personally don’t think it will. Teachers are already working as hard as they can.”

Test Scores Key

The experiment also was appealing because it differed from the merit-pay programs that came and went around the country in the 1980s.

For one thing, the bonuses are more substantial. Also, while the earlier programs tended to rely heavily on supervisors’ judgments to determine who got bonuses, the Nashville program uses a more objective measure: student test scores.

The growth in performance for an individual teacher’s students will be gauged against a historical average, calculated over three years, for the whole district. Such calculations are possible because Tennessee is among the states with data systems in place that allow researchers to measure the “value added” effect of schools and teachers on individual students’ learning growth.

Likewise, the Nashville plan is not what experts call a “rank-order tournament”—in other words, a system in which teachers compete with one another for rewards. All teachers who meet specified performance targets will get bonuses ranging from $5,000 to $15,000 in the fall.

According to Richard J. Murnane, a Harvard University economist who is a co-author of a widely read critique of the 1980s-era merit-pay programs, such differences suggest the new generation of teacher-incentive programs may be worth a second look.

“I’m much more agnostic on these new plans,” he said. “The only caution I would have is that when you look at the evidence on the stability of these rankings of teacher effectiveness as measured by student gains, you see nontrivial differences across subject areas or from year to year.”

That’s a potential problem for teachers who might wonder why they qualify for a bonus one year but not the next—particularly if they don’t think they’ve slacked off.

The investigators at the National Center on Performance Incentives don’t plan to report any results for at least three years for fear of biasing the outcome of the study. What they will ultimately find is a wide-open question.

A 2001 randomized trial of teacher-incentive programs in Israeli high schools found that individual teacher bonuses can be a cost-effective way to spur student-achievement gains. But researchers studying a similar experiment that took place in Kenya in the late 1990s reported more mixed results.

While student test scores rose in the experimental schools in Kenya, the rewards did not affect teacher-absentee rates, which remained high. Rather than show up more often, teachers taught to the test, spending more time coaching students on the subjects to be tested, according to that study.

Previous studies also did not address whether pay-for-performance systems can change the mix of teachers in the labor pool.

“There might be people out there who are not in teaching now but may decide to go into it if there were financial incentives,” said James W. Guthrie, a professor of public policy and education at Vanderbilt and the center’s executive director. “So you might be able to draw more talent.”

Regardless of the outcome, experts agree that the Nashville study, in and of itself, won’t be the final word on whether pay incentives work for teachers.

Mr. Guthrie said researchers are negotiating with two other locations in an effort to randomly test two other types of incentive programs—a rank-order tournament and a program that awards bonuses to groups of educators, such as those in entire schools or at certain grade levels, when students post big learning gains.

Texas Twist

The Texas study, while not randomized, may provide information on an even wider range of incentive programs. Texas has two new pay-incentive programs this school year: the Governor’s Educator Excellence Grant, which offers a total of $10 million to teachers opting to work in disadvantaged schools, and the Texas Educator Excellence Grant, providing $100 million for performance-based rewards in 1,160 schools.

For both programs, though, the money goes directly to the schools, rather than individual educators. Then the schools set the criteria for deciding who gets the bonuses. The awards also must be based on more than just test scores, according to Mr. Springer.

“Teaching excellence also takes into account such factors as mentoring of other teachers, teaching in subject areas that face a shortage, an individual’s level of education and experience, and whether the teachers are working in schools that are considered hard to staff,” he added.

The center plans to complete its reports on the Texas plans in 2009 and 2010.

Fortunately for the investigators, though, neither the Texas programs nor the one in Nashville are being financed by the Teacher Incentive Fund, a pot of money the federal government created last year to seed pay-for-performance programs.

A signature initiative of the Bush administration, the fund was opposed by both national teachers’ unions. Last month, Congress cut spending for the fund from $99 million in the 2006 fiscal year to $200,00 in fiscal 2007—a dramatic illustration of how politically precarious such efforts can be. That sort of action worries Mr. Guthrie, who has been interested in the incentive idea for 20 years. “I’m still very fearful this will be a fad that will wash across the landscape,” he said, “and go away without being carefully researched and will be thought of as a failure.”