First in an occasional series.
For the past five years, Ron K. Freeman has enjoyed the enviable position of being the principal of one of the highest-scoring middle schools in all of Kentucky.
Invariably, J. Graham Brown Middle School shows up near the top of the list when the scores come out for the state tests.
Yet in 1996, state officials declared that the Louisville school was “in crisis” because its scores declined on the statewide accountability index. Suddenly, Mr. Freeman found himself in the awkward position of informing parents that, by law, they could transfer their children to a “successful” school someplace else.
- • A New Accountability Player: The Local Newspaper
June 17, 1998
- • Charter Schools Struggle With Accountability
June 10, 1998
- • In Age of Accountability, Principals Feel the Heat
May 20, 1998
- • A Question of Value
May 13, 1998
- • Failing Schools Challenge Accountability Goals
March 25, 1998
- • Tenured Principals: An Endangered Species
March 4, 1998
- • The Push for Accountability Gathers Steam
Feb. 11, 1998
“The thing of it is,” he said recently, “the ‘successful’ schools all scored lower than we scored. It’s kind of a weird situation.”
But the kind of situation more and more educators are facing as states and districts rush to hold schools, principals, teachers, and students more accountable for their performance. Through test scores, performance evaluations, “endorsed” diplomas, and a host of systems that reward success and punish failure, policymakers are demanding results as they seek to assure parents and taxpayers that their children are getting a good education and their money isn’t being wasted.
At least 32 states and 34 big-city districts now have accountability systems based, in part, on test scores. And the numbers are climbing fast.
“We have been, frankly, inundated with calls from states that are looking at their accountability laws and want to strengthen them,” said Christine Johnson, who works on accountability issues for the Education Commission of the States.
But as policymakers wade into the thicket, many are finding that it’s filled with thorns.
“I don’t know of a single accountability plan implemented in a district or state that I would say is well thought through,” said Anthony L. Bryk, a professor of education and sociology at the University of Chicago.
In part, that’s because states like Kentucky must tangle with such prickly technical and political issues as: Should they hold schools to an absolute standard, or adjust their expectations to account for differences in income or existing achievement levels among the student population?
How fast should scores improve? What if too many students fail? Can states fulfill the policing and helping roles simultaneously? How can states and districts avoid trampling each other as they rush to adopt get-tough policies?
Accountability efforts also are encountering fierce resistance, sometimes from unexpected quarters. Parents in one of Michigan’s most affluent districts rebelled last year against a new high school proficiency test. Many refused to let their children take it, saying it offered no benefits—only the possibility of embarrassment—for college-bound students.
A ‘Minefield’
“The fact is, accountability and performance are easy to talk about, but they’re very, very difficult to implement,” Gov. George V. Voinovich of Ohio said at a recent meeting on accountability sponsored by the ECS. “When you start setting standards, and you start giving tests, it’s not easy stuff. It’s like going through a minefield.”
Critics of what are known as high-stakes tests warn that they can distort and narrow instruction, encouraging teachers to focus solely on what is tested and obscuring richer ways of judging schools. They say there is only limited evidence that such systems will actually raise achievement in the long run.
As the stakes rise, so does the likelihood of litigation. Fourteen parents have sued the Johnston County, N.C., district on behalf of children who were held back a grade for low scores on the state’s math and reading tests. The plaintiffs argue that tests intended to rate districts and schools are not valid for measuring the performance of individual students. (“High Stakes: Test Truths or Consequences,” Oct. 22, 1997.)
Texas officials also face legal challenges to their state’s high school exit examination, based on the different passing rates of minority and nonminority teenagers.
‘Getting Their Money’s Worth’
Accountability is the third side of an education triangle that also includes standards and assessments. Now that many states have adopted high standards and tests to measure students’ progress toward those benchmarks, they have turned their attention to making sure that performance matters.
“To have standards and a test without a system of accountability makes no sense,” said Kerry Mazzoni, a California Democratic assemblywoman who chairs the education committee in the legislature’s lower house.
At the same time, lawmakers, business leaders, and the public have placed a strong emphasis on the bottom line--most often measured by test scores.
“Policymakers have put a lot of money into education reform,” said George Madaus, a professor of education and public policy at Boston College. “So they want to tie results--outcomes--to dollars to show that taxpayers are getting their money’s worth.”
Easier said than done, cautions Frank Newman, the president of the Denver-based ECS.
“It is a huge change to move from a system where we assume that good things are happening to a system where we insist that good things are happening,” he said. “The reason the tension is so great is because we want accountability in a system that has been socialized the other way.”
An Imperfect Match
Indeed, while everyone wants accountability, it’s not always clear who should be accountable for what and to whom, how they should be judged, and who should make such decisions.
Many new get-tough measures have been added to those already in place, and in many cases states and districts have not coordinated their efforts. The resulting overlap is creating conflict and confusion for educators, students, and parents.
The fact is, accountability and performance are easy to talk about, but they're very, very difficult to implement."
Last June, DuSable High School in Chicago was targeted for “reconstitution” by Paul G. Vallas, the district’s chief executive officer. The decision to overhaul the school was based on a history of dismal test scores and high dropout rates. Employees had to reapply for their jobs and the principal was told he could be replaced.
In September, the Illinois education department also placed the 1,000-student school on its “academic early-warning list,” based on scores from a different set of tests.
“I feel a lot of pressure,” said Charles E. Mingo, DuSable’s principal. But, he added, “you tend to pay attention to those people who pay you. The state is down the line. Vallas is right here.”
Though he says he supports high standards, the embattled principal questioned whether his high-poverty school should be compared with others around the state, as if they were all on a level playing field.
“If I’m the poorest school in the nation, then compare me with like schools,” Mr. Mingo argued in a recent interview. “Don’t compare me with schools that have selective admissions and higher socioeconomic status.”
Complex Formulas
Many policymakers elsewhere have tried to address such concerns.
In their efforts to make accountability systems fairer, states and districts have resorted to complicated formulas that try to account for differences in student demographics or judge each school’s progress against a baseline.
But in doing so, they risk making the systems so complex that parents and teachers cannot easily understand them.
Kentucky sets biennial targets for each school. Schools that show at least a one-point improvement over their expected threshold receive substantial financial rewards.
Those that score below the expected threshold but above the baseline must write improvement plans. And those that score below the baseline, like Louisville’s Brown Middle School, are designated “in crisis” and receive extra assistance from the state.
But Mr. Freeman, the principal at Brown, said it is hard for top-performing schools to maintain their scores and show improvement, especially since the scores are based on testing different youngsters instead of following the same students over time.
“A lot of people are still thinking that if you could just find the right adjustment factors, then you could hold schools accountable directly on the numbers,” said Paul T. Hill, the director of the Center on Reinventing Public Education at the University of Washington. “But that’s a snare and a delusion. People who’ve tried to do that have found that the numbers are uninterpretable except by the experts, and therefore you can’t act on them.”
He advocates the use of clear and simple measures. But even then, he adds, policymakers shouldn’t base rewards and penalties solely on data: “You have to use human judgments.”
Mr. Freeman will learn in the fall whether Brown Middle School can come off the crisis list. Though the experience has been harrowing, the principal said, it hasn’t been all bad. “I guess what we have to do is look at our core content that we’re going to be assessed on ... and we have to make sure that teachers are teaching those concepts.”
Cost Factors
As they struggle to hold schools accountable, policymakers also are learning that such efforts come at a price.
“One thing that I’m absolutely sure of is that sanctions are not cheap,” said Susan H. Fuhrman, the co-director of the Philadelphia-based Consortium for Policy Research in Education, a federally funded research group.
Kentucky spends about $110,000 a year to assign experienced educators to work closely with schools that are designated ''in crisis.’'
And, like many other states and districts, administrators there wound up with a lot more of those schools than they expected.
The reward program, too, has been costly. Since March 1995, Kentucky has distributed more than $54 million directly to teachers and administrators in schools that qualified for rewards based on their performance.
Most states and districts have focused less on rewards and more on sanctions, in part because there’s little research to show whether rewards motivate educators to produce more.
A study last year by the Southern Regional Education Board found that only three of its 15 member states did not have sanctions on the books. But only six of the 15 provided financial rewards either to schools or individual teachers.
“I think the issue of whether or not financial rewards create change in the classroom is a very interesting one,” said Ms. Mazzoni, the California lawmaker. “I’m not convinced that they do.”
A Shift in the Debate?
Politicians appear more enthusiastic about cracking down on students who fail to perform.
At least 16 states now require students to demonstrate that they have met academic standards in order to graduate or proceed to the next grade.
The reason the tension is so great is because we want accountability in a system that has been socialized the other way."
In 1996, President Clinton joined the chorus of governors and superintendents calling for an end to so-called social promotion. Last summer, Chicago required more than 40,000 students with low test scores--about one in 10 of its total enrollment of 424,000--to attend summer school.
More than 11,000 students were forced to repeat a grade after failing a retest at summer’s end, and 15,000 8th graders were assigned to “transition centers” for intensive tutoring.
Other big-city districts, including Boston and Philadelphia, are considering such measures.
Advocates of such approaches argue that educators do students no favors when they promote them without the necessary skills. And they say it’s unfair to penalize schools unless students also have a stake in their performance.
But basing promotion or graduation decisions on a single test score may not hold up in court, experts caution.
Others see the renewed emphasis on high stakes for students as a way for educators to duck their own responsibilities.
“There’s been a very subtle shifting of this debate from consequences for adults and their organizations to consequences for kids,” observed Ted Kolderie, a senior associate at the Center for Policy Studies in St. Paul, Minn. “You don’t have to be around this debate very long for some teacher to tell you, ‘I am not responsible for what the students learn,’ because they believe that deeply.”
Consumer Choice
Mr. Kolderie argues that one of the best ways to increase accountability in education is through charter schools.
Such schools, he says, are accountable in two directions: upward to the agencies that grant them their charters and outward to the parents and students who choose the publicly funded but largely deregulated schools.
Parental and student choice would also be the dominant form of accountability in voucher programs that provide public money for students to attend private schools, proponents argue.
“Outside of education, that’s the main form of accountability in a free society,” said Chester E. Finn Jr., the president of the Thomas B. Fordham Foundation in Washington. “Does anybody go to your movie, or eat at your restaurant, or go to your store?
“I think it’s worth considering in education whether we want to use just a bureaucratic system, or a marketplace system, or some mixture of the two.”
