‘Growth Models’ Gaining in Accountability Debate

In the debate over the future of the No Child Left Behind Act, policymakers, educators, and researchers seem to agree on one thing: The federal law’s accountability system should be rewritten so it rewards or sanctions schools on the basis of students’ academic growth.

Congress, they say, should scrap the current model, which judges schools on the number of their students who are deemed proficient. That method, its critics say, overlooks schools whose students are showing growth toward proficiency, and sometimes fails to single out high-achieving schools in which a significant proportion of students aren’t making progress.

The U.S. Department of Education recently reaffirmed the Bush administration’s commitment to so-called growth models. Earlier this month, the department announced it would approve all states’ growth models that meet its criteria for participating in a 2-year-old pilot project.

“Everything I’ve done says growth model, growth model, growth model,” said David N. Figlio, a professor of economics at the University of Florida, in Gainesville, who has studied several states’ accountability systems. “A growth model will encourage schools to make genuine improvements, as opposed to cosmetic improvements.”

Compared with the NCLB law’s current “status model”—which makes accountability decisions by tracking the test scores of one year’s group of students against the previous year’s—growth models, by tracking individual students’ progress, can provide a more accurate picture of whether a school is succeeding in helping its students.

Pilot States

In November 2005, the U.S. Department of Education announced a pilot program for up to 10 states to use growth-based accountability models. All states are now eligible to apply for the program.

(Requires Adobe Acrobat Reader).

SOURCE: EPE Research Center, 2007

“The growth model shows you exactly what’s happening in that school,” said Robin R. Taylor, the associate secretary for assessment and accountability in the Delaware Department of Education.

But despite widespread support for the concept, the path to implementing growth models under a revised No Child Left Behind law won’t be easy, researchers and policymakers say.

Not all states have the data systems they would need to analyze test scores for evaluating schools using the proposed accountability model. Experts question whether growth models that required student gains large enough to reach the law’s goal of universal proficiency would deliver results much different from those of the current accountability system. They also disagree on how to design growth models to determine the success of schools with already-high achievement levels and thus low rates of academic growth.

“I don’t think you can do growth without recognizing status,” said Margaret Goertz, a co-director of the Consortium for Policy Research in Education, based at the University of Pennsylvania, in Philadelphia. “There’s got to be some sort of mix.”

The No Child Left Behind law, which marks its sixth anniversary next month, is the first federal effort to hold all public K-12 schools accountable for the performance of their students. The law was due for reauthorization in 2007, but Congress has not advanced beyond holding hearings and discussing draft plans for revising it. The effort is slated to resume in 2008, although some observers predict Congress won’t complete its work until after the next president takes office.

A revision of the Elementary and Secondary Education Act, first passed in 1965, the NCLB law requires states to test their students annually in grades 3-8 and at least once in high school in reading and mathematics. It also mandates that states determine whether schools and districts are increasing the number of students who are proficient in those subjects.

To avoid getting caught in the law’s web of sanctions, schools must be on pace to meet the goal of universal proficiency by the end of the 2013-14 school year. Schools must meet their annual goals for all students and those same goals for specified student subgroups, including members of racial and ethnic minorities, economically disadvantaged students, English-language learners, and children with disabilities.

If schools receiving federal Title I aid for disadvantaged students don’t meet their goals for two years in a row, they must offer their students the chance to transfer to other public schools. After a school falls short of its goals for five years, authorities must completely restructure it.

The law requires schools and districts to compare the test scores of each grade from one school year to the next. For example, the proficiency levels of last year’s 3rd graders will be compared with this year’s to determine a school’s accountability status for the 2008-09 school year. The accountability method is commonly called a status model because it tracks the state of each grade from year to year and doesn’t measure individual students’ growth in achievement.

The weakness of the model is that it doesn’t reward schools that start at low levels of performance. They can show significant growth in student achievement but still be declared “in need of improvement” under the NCLB law. Conversely, schools that start with high student achievement can backslide and not be targeted for interventions, so long as the total number of students scoring as proficient doesn’t drop below the schools’ goals.

Unlike the current accountability system, growth models recognize schools’ success in improving student achievement at all performance levels, Mr. Figlio said. Recent research suggests that the status models reward schools for focusing on students near the proficiency level and slighting those either way above or below that level. (“Acceleration Under Review,” Aug. 1, 2007)

“The big value of growth models is that educators focus on what their goal is: that is, to grow kids [academically],” said Chrys Dougherty, the research director of the National Center on Educational Accountability, an Austin, Texas-based nonprofit group that supports data-based efforts to improve schools.

Congress had little choice but to rely on the status model when it wrote the NCLB legislation in 2001, NCLB legislation in 2001, the law’s supporters say. At the time, few states tested students every year—something necessary for tracking student growth. And even fewer had the sophisticated databases needed to calculate accountability scores based on such growth.

Perhaps six states could have used a growth model under the bipartisan measure at the time President Bush signed it into law in January 2002, Mr. Dougherty estimates.

Today, all states test annually under the NCLB law. But just 34 of them have the three pieces of data that can track every student’s score from one year to the next, according to the Data Quality Campaign, a coalition of education groups led by the National Center on Educational Accountability. Those ingredients are: a unique number to identify each student, the assurance that proficiency levels are consistent across grade levels, and information about why specific students aren’t assessed.

When the Data Quality Campaign first conducted the survey in 2005, 21 states had the data they needed for growth models.

“States are moving very quickly toward” being ready to use growth models, said Brian Gong, the executive director of the National Center for the Improvement of Educational Assessment, a Dover, N.H.-based group that consults with states on testing and accountability policies. “It will take many states several years to work that out.”

Although policymakers, researchers, and interest groups are rallying around growth models, they don’t necessarily agree on how to design them.

The most significant question to be settled: How much growth is enough?

“When most people think about growth models, they think about a year’s growth in a year’s time or a little more than a year’s growth in a year’s time,” said Ms. Goertz, who was on a panel of researchers and advocates who reviewed applications in the Education Department’s pilot project.

While such learning growth may constitute progress, it’s not fast enough to meet the NCLB law’s goal of proficiency for all U.S. students.

In designing the pilot project, Secretary of Education Margaret Spellings required that students post achievement gains that suggest they would be proficient by the end of the 2013-14 school year.

“That’s a very steep trajectory,” said Mr. Gong. “It’s very hard for schools to reach.”

The number of schools meeting their goals hasn’t increased significantly in the nine states that are currently using growth models.

The criteria that the Education Department requires states to meet may make it difficult to dramatically expand the number of states using the growth models. And the steep increases in student achievement required under the growth models may mean that schools’ results may not differ from the current accountability system.

But states are anxious to try growth models to inform Congress’ debate over the future of the law, said one official who works closely with states.

“We have a large number of states trying to improve their approaches” to accountability, said Scott R. Palmer, the director of the education practice of Washington office of the Holland & Knight law firm.

“It’s good that the department would … encourage states to move to these models,” said Mr. Palmer, who advises the Council of Chief State School Officers and several states.

By changing the way states report whether schools have made adequate yearly progress, or AYP—the gauge of success under the law—states could give a better picture of schools’ progress toward the law’s goals. One alternative would be to give schools letter grades based on their progress toward meeting the proficiency goal.

That would eliminate the stigma of failing to achieve AYP and give schools an incentive to improve or maintain their grades.

In his research on Florida’s accountability system, Mr. Figlio found that schools that received an F and were far from moving up a grade rarely made significant progress. Those close to improving their grades had a reachable goal and were more likely to produce student significant achievement gains, he added.

“If schools think there’s no chance, why should they work toward it?” Mr. Figlio said.

Another benefit of a grading system, Mr. Gong said, is that it would recognize all schools that are making progress, not just those that are on track to meet an ambitious standard.

“It would … recognize that some growth levels would be admirable and hard to do even if they don’t get [all] students to proficiency,” he said.

For all the focus on growth models, some scholars suggest that No Child Left Behind also should recognize the overall achievement of a school. Just as the status model overlooks schools with significant achievement gains, a growth model could conceivably determine that a school with 100 percent proficiency is in “need of improvement.”

Trying to balance the grades for status and growth is often difficult, Mr. Gong of the National Center for the Improvement of Educational Assessment said. Last month, the New York City school system released grades based on a combination of schools’ overall achievement and growth of achievement for all its 1,466 schools. The city’s elite schools complained about getting grades of B or C, while some low-performing schools received higher grades.

“That’s always been the case when growth and status are combined,” Mr. Gong said. The best solution is to give separate grades, he added.

But the public isn’t savvy enough to see through such a nuanced system, said Mr. Figlio. In England, the education department gives each school a grade for growth and status, but people overlook the scores for growth, he said.

“What happens is people pay attention to the status score,” said Mr. Figlio, who recently lived in England while on sabbatical from the University of Florida.

Under Florida’s accountability system, schools are rated on the growth of three separate groupings of students: the total enrollment; students starting below proficiency; and those who score at the state’s lowest performance levels.

“The kids who are historically left behind are counted three times,” Mr. Figlio said. “The kids who are historically advantaged are counted once.”