Amid stepped-up school accountability pressures under the No Child Left Behind Act, many teachers appear to be adjusting how they do their jobs. But principals and district leaders are not necessarily in control of those instructional changes, a new study concludes.
Using data collected through surveys of math teachers, principals, and administrators in three states, a group of researchers at the Santa Monica, Calif.-based RAND Corp. examined educators’ responses to a series of questions about tests, their instruction, and their states’ accountability systems.
The theory of standards-based accountability assumes an aligned system of district and classroom responses to instruct students to a state-prescribed set of skills. Student-achievement results then feed back into the system as a lever to prompt action at the district and local levels.
SOURCE: Strong States, Weak Schools: The Dilemmas of Centralized Accountability
In the years following the passage of the NCLB law, teachers, on average, focused more attention on state content standards and seeking new teaching strategies, the researchers found. But when the data were analyzed for consistency within the same schools and districts, they found that educators’ responses varied more within individual schools than they did between schools or districts.
“Teachers are continuing to exercise a fair amount of autonomy in their classrooms,” said Laura S. Hamilton, a senior behavioral scientist at RAND and the lead author of the study, which was published recently in the volume Strong States, Weak Schools: The Dilemmas of Centralized Accountability. “The influence of principals and superintendents on teacher practices and attitudes appears limited.”
Some analysts say the findings suggest that the efforts of many district and school leaders to align educator professional development with content standards have not consistently translated into the kind of instructional changes that standards-based reforms are intended to inspire. Those include the increased use of test data to shape what happens in the classroom.
“I think a lot of districts are really very concerned about NCLB,” said Kerstin Le Floch, a principal research analyst at the Washington-based American Institutes of Research who has studied the implementation of the 2002 law. “But somehow there’s some disconnect in the linkages between central district offices and schools.”
Variations Abound
Ms. Hamilton’s study drew on surveys of representative samples of educators in California, Georgia, and Pennsylvania on a variety of questions relating to accountability systems; the study surveyed 2,350 elementary and middle school mathematics teachers, 260 principals, and 70 district superintendents.
An earlier study using the same data found that the federal law—which ties students’ test-score changes to school-level consequences, among other provisions—had unleashed overall changes in teacher practices. (“Teachers Say NCLB Has Changed Classroom Practice,” June 20, 2007.)
The new analysis of the results found more variation in the responses among teachers in individual schools than among schools or districts within the same state.
For example, the survey asked educators whether the possibility of a school receiving rewards or penalties was “a very strong motivator.” In California, 97 percent of the variance in responses was found among teachers in the same school; the variance in average responses from teachers in different schools accounted for only 2 percent of the total variance, while the difference in average responses among teachers in different districts accounted for just 1 percent of the variance.
Similarly, in Georgia, 95 percent of the variance in responses to questions about teachers’ use of data for instructional decisionmaking was observed among teachers in the same school; just 5 percent of the variance was observed among teachers in different districts.
The notion that assessment and accountability systems lead to an alignment of learning standards and coordinated instructional efforts among teachers, principals, and superintendents undergirds the standards-based reform movement, which was formally enshrined in federal legislation through the enactment of the NCLB law.
Half a Reform?
The new studies indicate that for many schools that goal of a standards-aligned system is still a work in progress, some observers said.
“Before standards-based reform, teachers could pretty much teach what they wanted,” said Jack Jennings, the president of the Washington-based Center on Education Policy, a nonpartisan policy-research organization.
“Now, there is greater uniformity in teaching content, but teachers are reacting differently to that. They haven’t all bought into the idea, and until they do, it’s always going to be a half-reform.”
Some teachers may still resist standards-based change, especially those who feel that their teaching practices are being set by accountability tests rather than a strong, aligned curriculum, Mr. Jennings speculated.
Ms. Hamilton’s study found teachers at odds over certain features of what some see as top-down accountability programs. While teachers appreciated the guidance provided by state standards, it found, they also felt constrained to eliminate coverage of topics not measured by state tests.
Still, if the study reveals some degree of push back by teachers as accountability programs influence classroom practices, it does not indicate massive resistance by teachers to standards-based accountability, said Bruce Fuller, a professor of education and public policy at the University of California, Berkeley, and the editor of the Strong States, Weak Schools volume. Instead, Mr. Fuller contended, it shows the need for a stronger focus to prepare school and district leaders to serve as coordinators of school improvement efforts.
“It is not inevitable that teachers respond to accountability in eclectic ways,” he said. “If there is a strong principal, superintendent, or teacher leadership, [educators] will rally around the tools of accountability.”
Lost in Translation
Increasingly, principals are expected to serve as instructional leaders in their schools, helping translate standards-based changes into practice and coalesce teachers into high-functioning teams, said Gerald N. Tirozzi, the executive director of the Reston, Va.-based National Association of Secondary School Principals.
But not all principals have benefited from the preparation and professional development needed to play that role effectively, he suggested.
“Historically, the schools of education have provided a plethora of courses dealing with school administration, management functions, budget preparation—all of which are important,” Mr. Tirozzi said. “It doesn’t leave much time, if any, for teaching principals how to evaluate teachers, supervise curriculum, or lead learning communities.”
Mr. Jennings made a similar point about teacher preparation. When it comes to helping teachers use student-achievement data to inform instruction, he said, “teacher training and retraining programs are still in the Ice Age.”
Other researchers, however, felt that the variation among teachers in the same schools did not necessarily indicate a weakness in the standards-based agenda.
“Given that it’s hard to point to specific teaching practices that we can use to dramatically improve student performance, it seems like standards-based reform might encourage additional experimentation,” said David N. Figlio, a professor of education, social policy, and economics at Northwestern University in Evanston, Ill. “Proponents of standards-based reforms might say putting additional pressure on these educators to try new things is going to be what leads to change.”
Such innovation could be crucial because some evidence indicates that a complex mix of practices is needed to accelerate learning for all students in a school.
In a second study published in the same volume, researchers found that in a select group of schools, the practices apparently associated with overall school achievement were not one and the same with those that were tied to the narrowing of achievement gaps among students of different demographic backgrounds.
“Teachers feel these policies contain a lot of sticks and not a lot of incentives,” Mr. Fuller continued. “Simply trying to control the day-to-day work of teachers is not a very motivating public policy.”
Closing Gaps
Researchers Melissa K. Henne and Heeju Jang analyzed teacher- and principal-survey responses in 257 California elementary schools with similar student-demographic characteristics.
They sought associations between practices linked to accountability, and changes in mean achievement levels and achievement gaps between white and Latino students on the state’s Academic Performance Index.
“It appears to be the case that the factors correlated with a school’s API score differ from those factors that correlate with a school closing its achievement gap,” said Ms. Henne, a graduate student at UC-Berkeley.
For example, increases in API scores in schools and among both white and Latino students appeared to correlate with principals who reported that districts gave schools clear strategies on how to use curriculum. But that element did not seem to relate to the closing of the achievement gap between those two groups of students.
Among the variety of examined practices, the study found narrower achievement gaps to be associated with teachers who reported that their schools prioritized accountability targets.
Neither study examines individual schools through qualitative, in-depth observation, and the researchers were hesitant to extrapolate beyond their findings.
Ms. Henne noted that her study focused on a relatively narrow type of school, while Ms. Hamilton pointed to the limited ability of survey data to capture how accountability programs influence teachers’ day-to-day practices.
But other studies could offer crucial information for lawmakers to consider as they work through proposals for reauthorizing the NCLB law and states’ standards-based accountability systems, the researchers suggested.
The Center on Education Policy is conducting observations in schools in Illinois and Rhode Island to gain further insights into the implementation of accountability programs and hopes to release those findings later this year, Mr. Jennings said.
Sticks vs. Carrots
In the meantime, the RAND study suggests lawmakers should engage teachers in the development of accountability policies that will provide them with tools to improve teaching to standards, Ms. Hamilton said.
Mr. Fuller said lawmakers should not try to put additional prescriptions on teachers’ behavior.
“We want teachers to pull together and review data that results from tests and take more professional development. But the tendency of states and the federal government is to clamp down and regulate teachers and teaching more intensively,” he said.
Mr. Figlio surmised that the continuation of standards-based accountability policies could eventually bring more uniformity to educators’ practices.
“My hunch is that in 20 to 30 years we’ll see a lot less variation than we do today.”
