When Robert Pondiscio pages, I answer. Yesterday, the Boston Foundation released a study on the efficacy of charter and pilot schools, which had the advantage of including both observational estimates of these schools effectiveness (comparing the performance of students in these schools with those in traditional public schools, net of some control variables) as well as lottery-based estimates of their effectiveness (comparing those who won lotteries with those who did not).
Kudos to the research teams (Atila Abdulkadiroglu, Josh Angrist, Sarah Cohodes, Susan Dynarski, Jon Fullerton, Tom Kane, and Parag Pathak) - this is a well-done, careful study that provides us with a range of estimates of charter and pilot school performance. There is certainly enough positive evidence here to support the creation of more charter schools in Boston, but I want to offer two cautions.
1) Beware of extrapolating lottery-based estimates when only the most successful charter schools had oversubscribed lotteries to being with.The lottery-based estimates of the effects of charters on middle school math performance are simply huge - half of a standard deviation! But only a quarter of charter schools are part of the lottery sample. Zero of the 5 charter elementary schools, 4 of 13 charter middle schools, and 3 of 11 charter high schools are included in the lottery part of the study because other schools were not oversubscribed enough to require a lottery. To be sure, these lottery estimates are important to understanding what the most successful charter schools can do. But I think we will all be disappointed if we expect a charter school expansion to replicate the effects of these middle schools (Academy of the Pacific Rim, Boston Collegiate, Boston Prep, and Roxbury Prep), which are among the highest performing charter schools in the country.
2) Pay careful attention to what elements of these schools can and can’t scale. Harvard graduates don’t grow on trees. These schools have faculties drawn from the most selective colleges - faculties that are highly atypical of the public school teachers.