KIPP charter middle schools enroll a significantly higher proportion of African-American students than the local school districts they draw from, but 40 percent of the black males they enroll leave between grades 6 and 8, says a new nationwide study by researchers at Western Michigan University.
“The dropout rate for African-American males is really shocking,” said Gary J. Miron, a professor of evaluation, measurement, and research at Western Michigan University, in Kalamazoo, and the lead researcher for the study. “KIPP is doing a great job of educating students who persist, but not all who come.”
With 99 charter schools across the country, most of which serve grades 5 to 8, the Knowledge Is Power Program network has built a national reputation for success in enabling low-income minority students to do well academically. And some studies show that KIPP charter schools have succeeded in significantly narrowing race-based and income-based achievement gaps between students over time. While not disputing that track record, the new study attempts to probe some of the more unexplored factors that might play into KIPP’s success.
It concludes, for instance, that KIPP schools are considerably better funded on a per-pupil basis than their surrounding school districts. The KIPP schools received, on average, $18,500 per pupil in 2007-08, about $6,500 more per student than the average for other schools in the same districts, according to the researchers’ analysis of federal 990 tax forms filed by schools reporting both public and private sources of funding. The study reports that nearly $5,800 of that per-pupil amount is private donations and grants.
Mr. Miron said the “$6,500 cost advantage” raises questions about the sustainability of the KIPP model.
The study also faults KIPP for not serving more students who are still learning English or who have disabilities.
“The limited range of students that KIPP serves, its inability to serve all students who enter, and its dependence on local traditional public schools to receive and serve the droves of students who leave, all speak loudly to the limitations of this model,” the report says.
Luis A. Huerta, an associate professor of public policy and education at Teachers College, praised the study for exploring indicators of KIPP’s operations other than student achievement, which, while important, doesn’t tell the whole story, he said.
“If we can start speaking about these more nuanced layers, and move beyond this discussion of student achievement, we tend to get a real picture,” he said. “Here we have schools receiving upwards to $6,000 or more than traditional schools, and that’s not even accounting for the fact they have fewer services than traditional schools, yet the gains they’ve shown in student achievement are quite modest.” Mr. Huerta is a faculty associate of the National Center for the Study of Privatization in Education at Teachers College, which had a hand in distributing the study but did not take part in the research.
The study came in for criticism from KIPP officials, as well as from two other researchers not involved in it. They questioned its methodology and said that while Mr. Miron is asking the right questions about KIPP schools, he hasn’t provided adequate evidence to answer them.
“We see this report as having significant shortcomings in the methodologies and reject the core conclusions the report is making,” said Steve Mancini, the public-affairs director for the San Francisco-based KIPP network, which was started in 1994.
Methods Differ
The study by the Western Michigan researchers used the federal Common Core of Data as its primary source. The researchers were able to obtain data from 2005-06 to 2008-09 for 60 KIPP schools across the country. The KIPP schools were compared with averages for other, more-traditional schools in the same districts. Besides the 990 forms, the researchers drew financial data on KIPP schools from the same federal database, which had financial data for 25 of those schools.
Robin Lake, the associate director of the Center on Reinventing Public Education, at the University of Washington in Seattle, was one of the scholars who questioned the study led by Mr. Miron.
“It seems he’s trying to explain away the KIPP effect rather than explain it,” she said. “More work needs to be done to get real answers.”
“The main point to make is the kind of data they are looking at is quite different from the kind of data we’ve been looking at,” said Brian P. Gill, a senior fellow for the Princeton, N.J.-based Mathematica Policy Research and a co-author of a comprehensive study of 22 KIPP middle schools released last June. (“KIPP Middle Schools Boost Learning Gains, Study Says,” July 14, 2010.) That study was commissioned by KIPP.
Mr. Gill said that Mathematica based its conclusions, including a finding that attrition of students from KIPP schools is about the same as from neighboring regular public schools, on data from individual students, not on aggregate data sets, as Mr. Miron’s study has done.
The study led by Mr. Miron found that approximately 15 percent of students disappear each year from the KIPP grade cohorts, compared with 3 percent per year in each grade in the local traditional school districts. Mr. Miron said that finding doesn’t contradict the finding by Mathematica that attrition rates are comparable between KIPP schools and local district schools on average, because his research team compared only KIPP “districts”—the cluster of KIPP schools in a particular district—and their surrounding local traditional school districts as a whole, not individual schools with schools.
Mr. Mancini, Ms. Lake, and Mr. Gill share the view that the comparison groups used in the Western Michigan study don’t provide reliable information about student attrition. It’s not appropriate, they contend, to make conclusions about attrition by comparing the proportion of students who leave a KIPP district with the proportion of students who leave the entire surrounding school district, which might have hundreds of schools.
“You want apples-to-apples comparisons. This is like apples to watermelons,” said Ms. Lake.
Mr. Miron said that the Mathematica approach to determining student attrition is “superior” to his. But his study explores an issue that he said Mathematica hadn’t addressed: How does the fact that KIPP schools tend not to replace students that leave, particularly in the upper grades, affect attrition?
“The low-performing students are leaving KIPP schools, but they are still in the public school sector,” Mr. Miron said.
Mr. Gill said Mr. Miron’s study doesn’t account for how grade retention, a hallmark of the KIPP model, may account for some of the shrinkage in cohorts of students moving from 6th to 8th grade.
He said Mr. Miron is on target, though, to ask questions about how KIPP replaces students in its schools. Mathematica has gathered data on that point that it will present this month in New Orleans at the annual meeting of the American Educational Research Association, Mr. Gill said.
Praise for KIPP
The Western Michigan study doesn’t challenge KIPP’s positive student outcomes. It says that the nonprofit network’s claims that its schools improve students’ test scores at a faster rate than regular public schools are backed by “rigorous and well-documented studies,” such as Mathematica’s.
Mr. Miron praises KIPP charter schools for how they have prepared and mentored principals and started new conversations in education circles about the benefits of extended instruction for students from low-income families. The KIPP school day is typically nine hours long, as much as a third longer than for the school day in surrounding school districts.
The new study contains at least one finding that echoes what the Mathematica study concluded: KIPP schools are less likely than local regular public schools to enroll English-language learners or students with disabilities—even though Mr. Miron’s data suggest the KIPP schools may have more financial resources to do the job.
But Mike Wright, who oversees KIPP’s network growth and sustainability, characterized the report’s findings on the financing of KIPP’s schools as misleading.
He focused on the finding that KIPP schools receive nearly $5,800 more per pupil from private donations than do their surrounding school districts. One problem, Mr. Wright said, is that the finding is based on a sample of 11 KIPP districts that isn’t representative of all KIPP schools. (Mr. Miron said he used those 11 KIPP districts because they were the only ones that reported public revenues in the federal data set researched for the study.)
Also, Mr. Wright said of the study’s authors, “they are including everything under the kitchen sink, whether starting a school from scratch or investing in facilities” in the figure for private per-pupil funding. He contends it’s a “misrepresentation” to imply that KIPP schools are overflowing with resources, when, unlike regular public schools, they are often left on their own to pay for buildings.
Lastly, Mr. Wright said, one KIPP school district skewed the finding for private revenues for the sample by mistakenly reporting that the bulk of its $8.1 million in revenue for 2007-08 was mostly from private sources, when it was actually from public sources.
Mr. Wright contends that the average funding advantage from private sources for KIPP schools in comparison with their local school districts is closer to $2,500 per pupil.
Ms. Lake said she has found in her own research on charter school financing that it’s hard to make meaningful conclusions based on the 990 tax forms because of ambiguities over what’s behind the numbers in the categories reported.
Mr. Huerta, however, said Mr. Miron’s methodology is strong, even though there are “complications in trying to dig out some of this information.” He added Mr. Miron is very clear in reporting his study’s limitations.
