Students’ gains in mathematics after three years in a charter school run by the Knowledge Is Power Program, or KIPP, are large enough in about half of schools to significantly narrow race- and income-based achievement gaps among students, according to a study of 22 KIPP middle schools nationwide.
The report, released today, is the first of a series focusing on the nationally known charter school network. Conducted by Mathematica Policy Research, Inc., of Princeton, N.J., the study was planned and commissioned by KIPP with grants from several private foundations. Plans are in the works for future reports to expand the sample to more KIPP charter schools, use randomized experimental research in a subset of schools, and look at measures other than state test scores.
“The consistency of the effects across most of the 22 schools and the magnitude of the effects are pretty striking and impressive,” said Brian P. Gill, a senior social scientist for Mathematica and an author of the study. “We do a lot of education studies, and often the effects are nonexistent or quite small.”
At about half the KIPP schools, the study found that the gains in math for students after three years in the schools were equivalent of 1.2 years of extra instruction, and .9 years of additional instruction in reading, Mr. Gill said.
No Evidence of Skimming
Critics of charter schools often contend the schools “cream” higher-achieving students from regular public schools, but the study didn’t find any evidence that KIPP is systematically enrolling more high-performers from their school districts. On average, the report says, KIPP middle schools have students who are more likely to be living in poverty and are more likely to be black or Hispanic than are students from the schools around them. Back when they were in 4th grade, the study also found, a majority of the KIPP middle school students had lower test scores on average than did students in their local school districts.
The study also looked at student attrition rates, which at least one previous study has found to be relatively high in KIPP schools. The Mathematica researchers found, however, that while attrition rates vary widely among the schools, KIPP middle schools don’t have systematically higher or lower numbers of students leaving before completion than other schools within their districts.
“This report provides irrefutable evidence that KIPP is not getting [positive] results because of getting more able students in the door or losing students at higher rates than their counterparts,” said Steve Mancini, the director of public affairs for KIPP, which operates a total of 82 schools serving 21,000 students across the country.
But the study did show that KIPP middle schools are much less likely to enroll special education students and students with limited English skills than the school districts from which they draw.
Higher Retention
Students are more likely to repeat a grade in KIPP schools than in regular public schools in their districts, particularly in the 5th and 6th grades, the study also shows. The study authors write that the differences in retention rates “likely capture KIPP’s philosophy that students should be promoted to the next grade level only after they’ve demonstrated mastery of their current grade’s material.”
The study is the first report that applies a “rigorous (nonexperimental) methodological approach” across a national sample of KIPP schools, according to the report. It compares the educational progress of KIPP students on state reading and math exams to that of demographically similar students from the same districts.
Half of KIPP schools studied are producing achievement impacts estimated to be big enough to reduce the gap in black-white test scores in math in half within three years, the report says. Three-year effects for reading are also large, but not as large as the impact reported for math.
Bruce Fuller, a professor of education and public policy at the University of California, Berkeley, said the methodology of the study is sound and the math gains are “impressive.”
He said, however, that the reading gains are “modest and perhaps overstated.” The reading gains at KIPP schools may not be as large as indicated by the researchers, he said, because when they matched KIPP students with similar students in their local school districts, the researchers only looked at certain characteristics and not others that may lead to students’ success. For example, students were matched according to their starting achievement level and whether or not they were special education students.
“They didn’t have data on parenting practices or home attributes that probably increase the propensity for students to enter KIPP schools and lift test scores,” Mr. Fuller said. Such factors that weren’t considered include whether a parent sits with his or her child to do homework or takes the child to the public library, he explained.
Growing Network
But Mr. Gill said that he believes the matching methodology holds up because it includes the academic achievement in 3rd and 4th grades for students prior to enrolling in KIPP middle schools. “The only way that the results we’re seeing would be biased is if the parental effect were larger in grades 5 to 8 [the grades of students in the study] than in grades 3 or 4,” he said.
Robin J. Lake, the associate director for the Center on Reinventing Public Education, a nonpartisan research organization in Seattle, said one of the problems in education reform is scaling up models that work. The Mathematica study is strong methodologically and shows, she said, that KIPP has been able to scale up “on a reasonable level, and that’s remarkable.” She characterized the achievement effects for math and reading in the 22 middle schools as “impressive.”
Ms. Lake acknowledged that some people question the sustainability of KIPP because the schools have a longer school day than regular public schools and place a lot of demands on teachers. But she said KIPP is replicating schools faster than most charter-management organizations in the country, “so they’ve got to be doing something right in terms of organizational growth.”
The study doesn’t say, however, that KIPP pays more for the longer school day than other charter-management organizations, and relies heavily on foundation grants to do that, she said.
A Good Mix?
Ms. Lake said she also wondered if the 22 schools studied are representative of all of KIPP’s schools. She said that the researchers needed to pick schools that had been open long enough to show students’ test scores for a few years. “It makes me wonder whether some of the schools were older and more stable,” she said.
Mr. Gill said that the researchers attempted to get data for all 35 KIPP schools that were opened by 2005, but data from school districts and states were only available for 22 of those schools. The study found that only three KIPP schools did not show progress in any year for students in math and reading. The KIPP Foundation, which provides financial support to the schools, has withdrawn its affiliation from two of those schools.
In all, Mr. Mancini said, the KIPP Foundation has withdrawn its support from 9 of 91 schools in the history of the organization because they didn’t meet KIPP standards.
Gary Miron, an education professor at Western Michigan University in Kalamazoo, said that after reading the study, a question remains: “Could this be a viable model for traditional public schools that have to take all kids?”
He said he’s not surprised by the findings that students in KIPP schools have impressive gains. After all, he said, KIPP schools have a rigorous curriculum, longer school day and year, and don’t have as many special education students as regular public schools.
He said that while the report looks at attrition rates, it doesn’t look at whether KIPP schools take any students in middle schools to replace students who are leaving. By contrast, he said, regular public schools continue to have an inflow of students at all grade levels who may be from low-income families who move whenever their leases are up in their apartments. Not filling the spaces of KIPP students who leave would lead to more-selective enrollments, he said.
Mr. Mancini said KIPP schools do try to enroll new students to fill spaces that open in 5th and 6th grades, but typically not in 7th and 8th grades.
