Data Show Schools Making Progress on Federal Goals

More schools nationwide will meet their annual achievement targets under federal law this year than last, if initial trends hold up. But parents in many states won’t know how their children’s schools did until well into the academic year.

By Sept. 1, just over half the states had released at least preliminary lists of the number of schools that had made adequate yearly progress under the No Child Left Behind Act, based on 2003-04 test data.In general, the percent of schools that met all their targets either held steady or increased compared with the previous school year—sometimes substantially, according to an analysis conducted by Education Week.

Alaska, California, Delaware, Georgia, Idaho, Kentucky, Maryland, North Carolina, North Dakota, Pennsylvania, Tennessee, Virginia, and West Virginia, for example, saw increases of 10 percentage points or more in the proportion of schools meeting all the federal benchmarks.

That’s positive news for President Bush—whose signature education initiative has been a source of controversy—as he heads into the fall campaign.

“I would say it gives Bush bragging rights that student achievement is starting to go up, and he can attribute it to No Child Left Behind,” said Jack Jennings, the director of the Center on Education Policy, a Washington think tank.

Yet while state press releases have largely attributed the gains to hard work and better test scores, at least part of the reason stems from changes in state accountability plans and the additional flexibility granted by the federal government.

Critics had forecast that a tidal wave of schools would fail to make adequate yearly progress, or AYP, in part because schools must meet multiple targets both for their total student populations and for subgroups of students who are poor, show limited skills in English, have disabilities, or come from racial- or ethnic-minority backgrounds.

But in the past year, the U.S. Department of Education has relaxed a number of rules on how AYP is calculated and approved numerous amendments to the state accountability plans required under the law. Those modifications appear to have undercut the grim predictions, at least temporarily. (“States Dicker Over Changes to AYP Plans,” July 14, 2004.)

“My opinion is that the act is working, in the sense that teachers are trying very hard to raise test scores, especially for groups that have been neglected,” said Mr. Jennings, a former Democratic congressional aide on education.

He warned, however, that it’s not valid to compare results from last year with this year because of the rule changes. Although some of the improvement is clearly the result of those relaxed rules, he and others said, it’s not possible yet to ascertain how much.

Joel Packer, who tracks NCLB policy for the National Education Association, was more skeptical. “At the end of the day, what is this really measuring when you have so many different statistical permutations and variations?” he said. “To me, I think it really discounts the argument that because the number of schools making AYP has gone up, the law is working.”

In contrast, Ross Wiener, the policy director of the Washington-based Education Trust, said the test data suggest “you can begin to have some confidence that these gains aren’t only about changing accountability systems.”

In general, he noted, elementary schools made more headway than middle or high schools. Only 23 percent of Oregon high schools, for example, made adequate progress, compared with about three-fourths of elementary and middle schools. In Minnesota, where middle and high schools were not rated on their test performance until this year, the proportion of schools that did not make adequate progress climbed sharply, from 7 percent to 24 percent.

Meanwhile, in such states as Delaware and North Carolina, Mr. Wiener said, “you see marked improvement in student achievement and real progress in narrowing gaps between groups.”

Confidence Intervals

In North Carolina, 70 percent of schools met all federal AYP goals in 2003-04, up from 47 percent in 2002-03. That difference can be traced in part to the federal government’s giving the state permission to use a “con fidence interval,” similar to the margin of error applied to polling data, in determining which schools met their targets.

Even without the confidence interval, the percent of schools making AYP would have jumped to 56 percent, according to Louis M. Fabrizio, the director of accountability services for the North Carolina education department.

Similarly, in Pennsylvania, the proportion of schools making adequate progress surged from 62 percent last year to 81 percent in 2004. While 16 percent of schools met their targets this year because of the addition of a confidence interval, said Carina Wong, the state director for assessment and accountability, state test data also showed solid gains—ranging from 4 percent to 6 percent—in the proportion of 5th and 8th graders scoring proficient or advanced in mathematics and reading, as well as solid gains among subgroups.

“I think a lot of the local reporters want to say it’s because you made these changes,” said Ms. Wong. Instead, “it’s because teachers really believe they’re working harder, and people are taking accountability much more seriously,” she said of the gains.

In Tennessee, where 81 percent of schools met all their federal targets this year, compared with 56 percent in 2003, state officials cited two main reasons for the improvement. Along with using a confidence interval this year, said Connie Smith, Tennessee’s director of accountability, the state provided technical assistance to schools that missed their benchmarks last year.

Participation Rates

Beyond the use of confidence intervals, the federal government granted states more flexibility in some areas, including in testing students with limited English proficiency or disabilities and in calculating test-participation rates.

The law requires schools to test at least 95 percent of students in the tested grades and in each subgroup in order to make adequate progress. Many schools failed to make AYP last year simply because they did not test enough students. But permission from the federal government to average test-participation rates over multiple years, combined with greater vigilance on the part of school leaders, seems to have helped.

Only 78 percent of Georgia schools met the test-participation requirement in 2003. This year, 96 percent met the criteria. In California, where 65 percent of schools made adequate pro gress in 2004, up from 54 percent last year, a key factor appeared to be a dramatic increase in the number of students participating in the state’s high school exit exam. Last year, 35 percent of all large high schools failed to make AYP based solely on participation rates. In 2004, only 14 percent fell short simply because of inadequate participation.

Even so, state Superintendent of Public Instruction Jack O’Connell said in announcing the improvement, “I continue to believe that our state accountability model provides a clearer and more accurate picture of how schools are doing.”

Many states made a point of noting the number of schools that fell short of the federal benchmarks because they missed only one of as many as 37 potential targets. In Virginia, for example, 170 of the 507 schools that did not make AYP missed only one target.

In addition to the flexibility around participation rates, U.S. Secretary of Education Rod Paige told states in February they could exclude reading and math results for limited-English-proficient students new to the country for one year in making their AYP judgments. Also, they could count students in the lep subgroup for up to two years after they stopped receiving language services.

Those lep changes were helpful, said Ronald Peiffer, an assistant state superintendent of education in Maryland. “We have very few instances right now where lep students were the sole reason for schools’ not making AYP,” he said.

Still, he noted that the rules on special education students remain a “significant problem,” with many schools failing to make AYP based on that subgroup alone.

Tardy Releases

Under the No Child Left Behind law, a 2½-year-old revision of the Elementary and Secondary Education Act, schools that fail to make adequate progress for two or more years in a row are identified as “in need of improvement.” Those that receive Title I money targeted at disadvantaged children are subject to a range of penalties, such as permitting students to transfer to another, higher-performing school in the district. Students in Title I schools that fail to make AYP for three consecutive years also are eligible for free tutoring.

Despite overall signs of progress, the number of schools in need of improvement appears to have gone up in many states in 2004. That’s in part a matter of timing. Relatively few Title I schools that had failed to make progress under the previous reauthorization of the federal law carried over that status under the new law. For most schools, this year marked the first time they could have failed to make AYP for two years in a row, and thus be identified for improvement.

The law requires states and districts to identify Title I schools in need of improvement before the start of a school year so that parents of children in those schools can be notified. Even so, many states missed that deadline. Utah, for example, will not release the list of such schools or the AYP results for all schools until Oct. 15.

Judy W. Park, Utah’s director of evaluation and assessment, said by the time the year-round schools had finished the 2003-04 school year, and the state was able to process test data, it was already July. The state also had to make some adjustments to the passing scores on tests this summer, as part of a standards-validation process, “and that has delayed the process a good month,” she said.

Although federal officials aren’t happy about the delay, Ms. Park added, “that’s the best I can do.”

Federal officials acknowledged that some states would have problems meeting the before-school-year deadline. “We work with them to push the issue and will deal with those who do not make it as we have to,” said Jo Ann Webb, a spokeswoman for the Education Department. But she declined to provide specifics.

The law does not specifically require states and districts to release AYP results for all schools before the school year begins—and many states have not done so. States attribute the delays to the amount of time it takes to process, verify, and report test data. But they also say the federal government was slow to approve changes to their accountability plans.

Texas won’t release the AYP status of its schools and districts until Feb. 24, although schools likely to end up as needing improvement will be notified in mid-November and must appeal by Dec. 1. Schools that do not appeal must notify parents and offer the choice of allowing their children to attend other schools starting in January.

Criss Cloudt, the state director of accountability in Texas, blamed the delay on the federal government’s failure to approve changes to the state’s accountability plan until July 29.

Ms. Cloudt said Texas didn’t have time to modify its data-collection system to reflect some changes to its accountability plan, which will have to be handled through the appeals process. “We really felt pretty strongly that schools needed the opportunity to see their data and appeal those results prior to the public release,” she said.

But Mr. Wiener of the Education Trust, a research and advocacy group that works to improve the achievement of minority and disadvantaged students, said that for the AYP information to serve its purpose, “it just needs to be available sooner” in many states.

Moreover, he said, while some states have made all the decisions that go into AYP determinations readily accessible and understandable to the public, others have not.

Districts in Improvement

This also marks the first year that many districts can be identified as needing improvement because they missed their performance targets for two or more years in a row. Initially, state officials had feared that virtually all their districts would end up in that category, as test results were aggregated up to that level.

Those dire projections may not materialize this year, partly because the federal government has permitted states to identify districts as in need of improvement only if they miss their targets at all three levels—elementary, middle, and high school—for two consecutive years.

Forty-three additional districts entered school improvement status in North Carolina this year, up from one last year, said Mr. Fabrizio. But, he noted, the state probably would have had nearly 85 percent or 90 percent of its districts listed that way without the added flexibility.

In Pennsylvania, where 171 (or 34 percent) of the state’s districts are in need of improvement, the figures could have been much worse, said Ms. Wong. Last year, 373 of the state’s 500 districts were on a warning list. “My greatest fear was that we were going to flip many of those over,” she said.

At least one state—Indiana—has shrunk its list of districts in need of improvement through a novel interpretation of the federal law. Last spring, it looked as if 161 of the state’s 297 districts were in danger of making the list.

Since then, Indiana has decided to use only the test results from schools receiving Title I money to determine whether districts fall into that category and are subject to penalties. As a result, it identified only 23 districts in need of improvement this year.

“It was not how we initially interpreted the provision,” said Linda Miller, an assistant state superintendent in the Indiana education department, “but subsequently, it has been our opinion that the law allows us to make that decision.”

Harder Times Ahead

Many state officials warned that the real test could come next year, when states must signi ficantly raise their AYP benchmarks for the first time, and many states will begin testing in more grades. And by 2013-14, all students will have to be proficient in math and reading.

“The steps go up every year as we get to 2013-14, so it’s going to get that much more difficult, said Tom Watkins, Michigan’s state superintendent. “Nobody should rest on their laurels.”

Assistant Editor Erik W. Robelen contributed to this report.