Study of Harlem Children’s Zone Finds Gaps Closing

The Harlem Children’s Zone, a high-profile New York City initiative that combines charter schools with wraparound community services for minority students and their low-income families, is showing dramatic academic gains that effectively close the black-white achievement gap in most categories examined, a new study finds.

What’s less clear, the researchers say, is whether the improved student performance can be explained by the quality of the schools alone, or by the combination of the schooling with the web of community supports, such as early-childhood programs, parenting workshops, and asthma and anti-obesity initiatives.

That question is of particular consequence now, since the Obama administration has touted the Harlem Children’s Zone as a model anti-poverty strategy it hopes to help replicate in other urban areas. (“President Envisions Anti-Poverty Efforts Like Harlem’s ‘Zone’,” March 11, 2009.)

The report’s evidence suggests that “either the ... public charter schools are the main driver of the results or the interaction of the schools and the community investments is the impetus for such success,” write the co-authors, Roland G. Fryer Jr., an economics professor at Harvard University, and Will S. Dobbie, a graduate student at Harvard’s Kennedy School of Government. “Community investments alone cannot explain the results.”

The report was published this month as a “working paper” by the National Bureau of Economic Research. It is in process of being peer reviewed.

The findings come as U.S. Secretary of Education Arne Duncan this week talked up the Harlem Children’s Zone at a conference the nonprofit organization hosted in New York City.

“Children born today in Harlem have a continuum of support that middle-class children take for granted, but is shamefully rare for disadvantaged students,” he said. “President Obama was so impressed by [the effort] that he made it the template of his Promise Neighborhoods proposal during the campaign.”

Mr. Obama has requested $10 million in fiscal 2010 for that program.

Cautions Offered

Edmund W. Gordon, a professor emeritus of psychology and education at Teachers College, Columbia University, and at Yale University, said he believes the results are “cause for celebration,” but he expressed concern that the Obama administration may be moving “unselectively” to reproduce the model, when it may need to be adapted to local conditions.

He also cautioned that the study does not offer evidence of the effectiveness of the community supports as key to the success.

“What I’m hesitant to endorse is the idea that the whole package is ready for generalized replication,” said Mr. Gordon, who is the chairman of a research advisory committee for the Harlem Children’s Zone. “I think the idea of wraparound student services needs a lot more nuanced study.”

‘A Lot of Moving Parts’

The Harlem Children’s Zone runs a broad-based program to meet the educational, health, and social-service needs of residents in a 97-block area of New York City, including two charter schools that serve some 1,200 students in elementary, middle, and high school grades.

To gauge the academic results of the effort, the researchers used two statistical strategies. One was a randomized lottery method comparing a group of students who won the lottery and enrolled in the schools with a control group of students who applied, but did not win the lottery. The second strategy used the schools’ start date and institutional features of the Harlem Children’s Zone to construct a valid control group.

The study finds that a “treatment group” of students enrolled in the Harlem charter schools showed academic gains over three years in elementary mathematics and English language arts, and in middle school math, that were strong enough to close the achievement gap between those students and the average score for white students in New York City public schools, based on their performance on New York state tests. The black-white achievement gap for middle schoolers was reduced in English language arts by about half.

Put another way, the typical student before entering a Harlem Children’s Zone middle school was outscoring only about 20 percent of New York City’s white students in math, but after three years in the charter, was outscoring 45 percent of white students, the report finds. After adjusting for gender and free-lunch eligibility, the typical Harlem Children’s Zone student was outscoring 54 percent of white students.

Mr. Dobbie, the report’s co-author, said the gains were remarkably strong, far outpacing the benefits seen by many other interventions, such as reducing class sizes from 24 students to 16.

The report also examined the students’ performance on the Iowa Test of Basic Skills. Here, the results were “similar, if more modest,” but there was no control group scores available for comparison, making the findings “only suggestive,” the report says.

The researchers note that the charter schools themselves have a range of characteristics that may help students, from an extended school day and year to an emphasis on high-quality teachers, providing free medical and dental care, and offering students financial incentives for achievement.

“There’s obviously a lot of moving parts,” Mr. Dobbie said, suggesting that more study is needed to gauge the keys to success.

The report also examined two of the community initiatives, a parenting program and a preschool program, but found no evidence that they were positively associated with student achievement.

Richard Rothstein, a research associate at the Economic Policy Institute in Washington, said the effort shows promise. But he suggested the positive but less dramatic gains seen on the Iowa tests provide “a much better indicator of what’s going on,” because they are “not subject to the score inflation of high-stakes exams like the New York state tests.”

The report seems to underemphasize that the Harlem Children’s Zone schools themselves offer some non-academic supports that may help explain their success, he said, such as regular medical, dental, and mental-health services for students, as well as substantial funds for after-school programs.

The report says cost is “an important consideration” as the Obama administration and others look to replicate the approach. The Harlem Children’s Zone spends more than $19,000 per pupil annually for all the services involved. By comparison, it adds, the median school district in New York state spent $16,171 in 2006.