As states’ information-collection systems grow more sophisticated, officials are grappling with where to draw the line on how “value added” data on teachers can be used.
Tennessee law permits the use of such data for teacher evaluation as long as the data meet certain technical requirements. The state’s Hamilton County district, with union approval, also offers bonuses based on the data.
California serves as a counterpoint: A 2006 state law establishing a new teacher-identification database prevents such data from being used for teacher pay, evaluation, or personnel decisions. Some stakeholders there say it might not even be permissible to link teacher records to student data.
“I think we’re going to see more and more state legislatures taking on this issue,” said Aimee Guidera, the executive director of the Data Quality Campaign, an Austin, Texas-based group that promotes the use of longitudinal data to improve schools.
“Yes, some of those conversations are going to talk about merit pay,” she said, “but that’s not the point to start with. Any state that only has the conversation about merit pay is denying itself the benefits of having a truly informed education system.”
New York City is piloting informal teacher-effectiveness reports. The data below about a fictitious teacher are part of a report that might be used with the goal of improving instruction.
SOURCE: New York City Department of Education
Since the adoption of the federal No Child Left Behind Act, the task of establishing data systems for tracking students’ year-to-year achievement gains has fallen largely to the states.
Value-added techniques are one way of dissecting such data. In theory, value-added systems can isolate teachers’ contributions to students’ learning by filtering out factors outside school control, such as family-income level.
Proponents of the systems say value-added information holds promise for being integrated as one of several factors in human-capital decisions, such as evaluations.
Many states are laying the groundwork for such systems by assigning each teacher a unique identification number. According to the Data Quality Campaign, a dozen states can now link teachers to specific students’ course assignments and assessment data. Those states are Alabama, Arkansas, Delaware, Florida, Louisiana, Mississippi, New Mexico, Ohio, Tennessee, Utah, West Virginia, and Wyoming.
Few of those states have set teacher policies that incorporate the data, though—in part, observers say, because value-added systems are a political minefield. States that have progressed on value-added policies—such as Louisiana, which uses the data for teacher-preparation program accountability—typically don’t use the systems for individual teacher decisions.
Researchers note that technical factors can affect the systems’ ability to make reliable determinations of teacher performance, an issue that has raised concerns from the national teachers’ unions.
Last year, the unions condemned a draft of an NCLB reauthorization bill that contained a voluntary performance-pay program for teachers based in part on student test scores. The controversy helped stall consideration of the measure.
“The tenor of that discussion clearly indicates that this is going to be, and will be for some time, a very contentious debate going forward,” said David DeSchryver, the legislative counsel at Brustein & Manasevit, a Washington-based law firm that specializes in education policy.
Changing Minds
Proponents of the systems stress their ability to help districts conduct analyses to determine whether a particular form of professional development, an induction arrangement, or specific grouping of students and teachers led to improved teaching effectiveness and student achievement.
States with value-added data on teachers—also known as “teacher-effect” data—are also using it to tackle some of education’s most intractable problems, including the maldistribution of teacher talent.
In Tennessee, Hamilton and Knox counties use the data to gauge whether schools with higher poverty concentrations have fewer effective teachers than more-advantaged schools. The analyses will help districts devise recruitment strategies, as well as professional development for teachers not as effective as their peers, said Eric Hilgendorf, the state’s director of charter schools and choice programs.
Ideally, Ms. Guidera said, these possibilities should be paramount in discussions with teachers about the systems’ potential.
“It is a very big change for educators to see data not as a hammer, but as a flashlight,” Ms. Guidera said. “Teachers need to see the proof points about why they should change what they’re doing, and how it’s going to make their jobs easier and better. Until we can prove that to them, they have every reason to be concerned.”
Attempts to introduce teacher-effect data without a consensus on appropriate use have generated controversy. The New York state legislature last year banned the use of such data for tenure decisions for two years after a dispute between the New York City school district and the United Federation of Teachers.
But last month, the city garnered the union’s support for a pilot program to provide teachers and their principals with value-added information. The data will be used to help teachers improve, but not for human-capital decisions, said Christopher Cerf, the deputy schools chancellor.
The reports are expected to show whether a teacher, on average, is particularly effective with certain groups of students, such as those at the low end of the achievement spectrum or students with disabilities. Then, principals will be better able to tailor professional development and improve student groupings, Mr. Cerf said.
“If you are going to rely on the innovative powers of educators, you must give them data and information in order to make intelligent decisions,” he said.
‘Bad Consequences’
Beyond the use of data as a tool, though, lingers the possible move to formal personnel decisions based on the data.
State law in Tennessee permits evaluations to incorporate teacher-effect data, provided that such evaluations be based on at least three complete years of academic data. Florida law requires that teacher evaluations be based primarily on student-achievement data.
Randi Weingarten, the president of both the UFT and its national parent, the American Federation of Teachers, remains opposed to such uses of the teacher-effect data. She said those policies would jeopardize use of the data as a tool to improve teaching and learning.
Ms. Weingarten said that using the data for formal evaluations “could create tremendous bad consequences to attracting and retaining teachers in hard-to-staff schools. We have a moral, statistical, and educational reason not to use these things for teacher evaluation.”
As the issue arises in other states, observers expect it to generate legislative and even legal battles.
In California, the restriction on the teacher-identification database is so tightly written that the state may not legally be able to tie its teacher records to its longitudinal student-achievement database. The law prohibits the new system, which won’t be operational until 2010, from being used for pay, promotion, punishment, evaluation, or other employment decisions.
“All along, we have stressed that this system in no way should be connecting individual student-test scores to individual teachers,” said Mike Myslinski, a spokesman for the California Teachers Association, a supporter of the language.
If the union’s interpretation is correct, the state would not be able to provide even informal information to help teachers improve through reports such as those being piloted in New York City.
Russlyn Ali, the executive director of Education Trust-West, which lobbies for better education for poor children, said her Oakland, Calif., organization challenges that interpretation of the law. “We are going to need a judicial opinion to decide this, because there is too much confusion and too much vitriol,” said Ms. Ali, whose organization supports the use of value-added data to identify excellent teachers.
Experts also are concerned about the potential for such laws to hinder research.
“It is worrisome,” said Douglas N. Harris, an assistant professor of education policy studies at the University of Wisconsin-Madison who has studied teacher-effect data. “Certainly the states that have created these data systems and linked teachers to children have learned a lot more than other states.”
State Question
As other states’ data capacity improves, the question remains how their legislatures will respond to those issues.
“If the states right now are the ones who seem to own the standards and assessments, it makes some sense that they are making some rules about the data,” Ms. Weingarten of the AFT said.
In Colorado, a bill to establish a unique teacher identifier tied to student records failed in 2006, in part over concerns about its use for high-stakes purposes.
A commission later set up by the legislature discussed at length potential misuses of the data. But the panel decided against recommending a prohibition on certain policies, fearing that step might hamstring districts’ ability to use the information innovatively, said Jacqueline J. Paone, the executive director of the Alliance for Quality Teaching. The organization works to improve teacher quality in the state.
The commission’s final recommendation to the legislature, in June, stressed that the data should not be used for punitive purposes, said Ms. Paone, who participated on the panel. Now, it is in lawmakers’ hands to translate those recommendations into legislative language.
Advocates of value-added teacher policies worry that future state decisions will be driven by a coterie of stakeholders, rather than a transparent process such as Colorado’s.
“The decisions about value-added-data systems are generally hashed out between the state’s teachers’ union and a few state legislators,” said Kate Walsh, the president of the Washington-based National Council on Teacher Quality, a group that usually wants states to take a stronger role in setting teacher-quality policy.
“If the public were given half a chance to learn about the power of value-added-data systems,” she said, “I doubt they’d remain unengaged and tolerate depriving schools and teachers themselves of the data.”
