What’s the difference between research and rhetoric? In the public debate about parental choice in education, it’s getting harder and harder to tell.
In response to President Bush’s proposal for a federally funded voucher to give low-income children the opportunity to attend private school, Albert Shanker, the president of the American Federation of Teachers, has offered up some more of his “research’’ on public and private schools. For months Mr. Shanker has been arguing that private schools should be excluded from choice programs and that the choice concept itself is flawed. Why? Because private schools, which are subject to the competitive pressures that choice programs aim to stimulate, actually perform no better than public schools--a claim he argues is clearly supported by the latest math scores from the National Assessment of Educational Progress.
This is an awfully strong claim to base on one set of test scores. But now Mr. Shanker is claiming much, much more. Noting that the gap between average public and private NAEP scores is smaller in the 12th grade than in earlier grades, Mr. Shanker concludes that not only are private schools no better than public schools but also that “the longer students stay in private schools, the worse they do, and the longer students stay in public schools the better they do.’' This is preposterous. Unless Mr. Shanker is no better at math than the innumerate students he harshly criticizes, this is sheer demagoguery.
To begin with, the progress of individual students cannot be easily inferred from the test scores of different groups of students in different grades. The scores of today’s elementary-school students cannot tell us with any certainty the scores that today’s high-school students posted when they were in elementary school. To draw inferences about achievement growth with any confidence, one must have observations of the same students over time. Unfortunately, such data are difficult to collect and exceedingly rare: It’s tough to keep track of a large sample of students for a long period of time, and schools dislike repeated intrusions by researchers.
When the appropriate data have been collected, however, they show an advantage to private schools in promoting achievement growth. Some researchers dispute the conclusion, which we and others have drawn, that this private-school advantage is large and important. But no one has presented credible evidence or even argued, as Mr. Shanker now has, that the advantage goes in the other direction.
But what about the data that Mr. Shanker cites? Is it possible to draw any reliable conclusions about achievement growth from the latest NAEP scores? If one is willing to assume that today’s elementary-school students can provide valid measures of how today’s high-school students were doing eight years ago, yes. However, there is still a problem. Students drop out of school and switch between public and private schools, changing the composition of schools over time. To figure out what students who stay in either public or private schools learn, one must adjust high-school scores for students who have left school and have not been tested or who have switched schools and have received much of their education elsewhere.
Both of these adjustments will tend to increase the difference between public and private high-school averages. Public schools have much higher dropout rates than private schools do--twice as high, on average--and lose 20 percent to 25 percent of their elementary-school cohort by the senior year. The departure of typically low-scoring dropouts will inflate high-school averages in both sectors, but more so in the public sector with its higher dropout rate. In addition, public schools pick up a large group of private-school transfers--one-third of all private 8th graders switch to public high school--while private high schools take on a good number of public middle-school graduates. Because students who switch from private middle schools to public high schools tend to have higher test scores than the students already in public schools, these transfers raise the public-school average. Transfers in the other direction do not change test scores because the public middle schoolers who switch to private high schools tend to post test scores similar to other private-school students.
With aggregate data, it is difficult to say with confidence precisely how these population changes affect average high-school scores. But with a few reasonable assumptions, it is possible to make some plausible estimates. Assume, first, that public schools have a 20 percent dropout rate, private schools a 10 percent dropout rate, and that dropouts would score below the modal level for seniors (level 300) on the NAEP math tests if they remained in school and were tested. If dropouts are returned to their respective samples under these assumptions, the public sector places 36 percent of its students at the modal level--down from an initial estimate of 45 percent--and private schools place 48 percent at the modal level, down from an initial estimate of 53 percent.
Assume, next, that the students who switch from private middle school to public high school have test scores that are similar to private-school students who do not switch. With a little algebra it can be shown that the public-sector score would be 1 percentage point lower if 5 percent of all public high-school students weren’t from private 8th grades. Corrected for both dropouts and sector switchers, the public-school score for seniors becomes 35 percent at the modal level, the private-school score becomes 48 percent at the modal level, and the public-private attainment gap becomes 13 percent. This gap is statistically indistinguishable from the gaps at the modal achievement levels in the 4th and 8th grades, which are 15 percent and 16 percent respectively. Contrary to Mr. Shanker’s contention, the gap between public and private achievement does not narrow.
The gap may, in fact, go the other way. Consider what happens after students graduate. If Mr. Shanker is correct--if public- and private-school students are heading in opposite directions as they approach graduation, public students up and private students down--it is hard to understand why the October after graduation, half of all private-school graduates are enrolled in four-year colleges or universities but fewer than 30 percent of public-school graduates are so enrolled. It’s similarly hard to figure why six years later, 31 percent of all private-school graduates have earned bachelor’s degrees while only 13 percent of public-school graduates have.
Of course, private-school children come from families that on average are more affluent and have higher educational expectations than public-school families. But research has been unable to attribute more than half of the sector difference in college-completion rates to family influences. Consider, for example, these facts: White children are twice as likely as black and Hispanic children to earn a bachelor’s degree. Nevertheless, black and Hispanic graduates of private schools are more likely to finish college than white graduates of public schools--their six-year completion rates are 26 percent and 19 percent respectively--even though the families of white public-school graduates are more affluent and better educated than the families of black and Hispanic private-school graduates. Private-school students certainly do not fare worse than public-school students over time, and no responsible look at the data could suggest otherwise.
We fully recognize that reasonable people can argue, as researchers have argued for years, about the relative merits of public and private schools. We believe that private schools have valuable lessons to teach public schools--about the importance of high academic expectations, school autonomy, and parental choice. We believe that a careful and comprehensive look at the evidence on how schools are governed and organized, as well as how schools perform, supports this conclusion. We respectfully disagree with researchers who argue that the private-school advantage is too small to justify sweeping reforms based on choice. But we welcome the vigorous and thoughtful debate that is a healthy part of the research process. What we do not welcome is rhetoric disguised as research. That is what Mr. Shanker is now offering.
