The U.S. Department of Education has pumped more than $5 billion into a supercharged version of the School Improvement Grant program that gave grants of up $2 million to the lowest-performing schools in the country to try out dramatic turnaround strategies (like turning themselves into charters, or getting rid of half their staff).

Now top Republicans want to get rid of the program altogether in a rewrite of the No Child Left Behind Act currently working its way through Congress. Under a draft bill introduced by Sen. Lamar Alexander, R-Tenn., states could direct a portion of their federal funds to turnarounds, if they want to. That would essentially take the feds entirely out of the turnaround equation.

So is that a good idea? Or is SIG working?

A big report released Friday by the Council of the Great City Schools, which represents urban districts, tries to get at that question. The report took a look at the progress of SIG schools from the 2009-10 school year (the year before schools got their grants) to the 2012-13 school year (when the first schools that entered the revamped program exited it).

The bottom line: 70 percent of schools showed some progress. That’s roughly consistent with the Education Department’s own (really flawed) data on the program, which showed that two-thirds of SIG schools got better, while another third actually slid backward.

The CGCS data has some key differences from the department’s, though. For one thing, it shows how schools that actually got the grants performed compared to other schools in the same district that were also eligible for the funding, but didn’t get selected, and schools that weren’t eligible to begin with. That gets at one of a number of key, unanswered questions in the original SIG data.

Overall, SIG schools started off behind the other two groups, but started to close the gaps in the first year of the grants, when presumably a lot of change was happening very quickly. Then things leveled off.

Check out the overall results in math here:

And in reading here:

It’s worth pointing out another key difference here between the department’s data and the council’s. Both looked at annual test data in grades 3 through 8. And the department continued to look at annual tests for high school performance.

The council didn’t look at the annual high school test data. Instead, it looked at whether high schoolers stayed in school and advanced to the next grade. That’s because states don’t all necessarily test high schoolers in the same grade. (Sometimes it’s in 10th, sometimes it’s 11th, and sometimes it’s senior year.) The council found that urban high schools that got SIG money showed improvement when it came to the ability to advance students from grade to grade.

On the other hand, when actual test scores were used, the picture looked somewhat different. SIG elementary schools still had low achievement compared to their non-SIG peers even after years of interventions.

Other interesting findings:

•The council couldn’t find any major differences in terms of student achievement between schools that used the most-popular SIG model, known as “transformation,” (which required teacher performance pay, extended learning time, and an intense focus on data to improve student outcomes), and the second-most-popular model, “turnaround,” (which called for getting rid of half a school’s staff). Both models required schools to get rid of their principals if that person had been on the job more than three years.

•SIG schools were more likely to get better when the emphasis was on improving instruction, not making sure all the i’s were dotted and t’s were crossed in grant compliance. But SIG schools didn’t always partner with folks who could help improve instruction. For instance, the report says, sometimes schools brought in groups that don’t really specialize in instruction, to help with turnaround work, including City Year, Communities in Schools, and the Urban League.

“These are fine groups that are often capable of providing much-needed wraparound and other community supports, but are not always capable of boosting instructional capacity,” the report says. “Sometimes more emphasis was put on these groups than on groups or strategies that could enhance academic results.”

•Losing the grant funding has been a big deal for SIG schools. Plus a lot of the collaboration between the state, the district, and the school that began during the grant implementation started to fizzle.

So what does this mean for the program’s (precarious) future?

SIG funding helped, overall, said Michael Casserly, the executive director of the Council. But it wasn’t the models, for the most part. It was the money.

“We think having a dedicated pool of money that specifically targets the lowest-performing schools and the schools in need of turnaround [is preferrable] to turning this over solely to the states,” he said. In the past, when states had a freer hand over turnaround money, “it wasn’t always clear to us that the states provided clear and consistent enough direction to the turnaround schools to make this a going venture.”

Plus, without a dedicated pot of money for turnarounds, there’s likely to be less data collection to show which strategies are actually effective.

For what it’s worth, U.S. Secretary of Education Arne had nice things to say about the report. It “offers valuable insights on what works,” he said.