Charters, Vouchers Earning Mixed Report Card

Second in a five-part series

Judged solely on its students’ test scores, SER-Ninos Charter School looks like a failed experiment.

The elementary school’s overall passing rate on Texas’ standardized exams last year was 27 points below the state average of 78 percent—29 points under the norm in reading, 27 in writing, and 26 in mathematics.

But those numbers mean little to parents like Adela Ibarra. Two years ago, the mother of two was unhappy with the all-Spanish instruction her daughter, Sara, was receiving at a nearby traditional public school, and she became convinced a teacher’s bullying was interfering with the timid kindergartner’s ability to succeed.

So Ms. Ibarra moved Sara to SER-Ninos, and put her son, Gerson, in prekindergarten classes at the school located here.

“For me, I’m very content to be going to SER-Ninos because my children are happier,” she said recently.

Word has apparently gotten around: SER-Ninos has a waiting list of 109 children, enough to fill more than a third of the school’s 270-student enrollment

Still, the question remains: Is this a good school?

In certain states, in certain grades, using certain criteria, charter schools like SER-Ninos seem to outperform their counterparts in the traditional public education system. Choose other data, and these publicly funded but largely independent schools might look like a waste of time and money.

“It’s difficult to make a blanket statement that these schools are doing better or worse,” said Eric Hirsch, a senior policy analyst for the National Conference of State Legislatures in Denver. “As with traditional public schools, we’re going to see wonderful charter schools and poor charter schools.”

It’s just as hard to grade the nation’s limited experiments with publicly financed voucher programs, which allow students to use taxpayer money to attend private and religious schools of their choice.

But the lack of conclusive evidence is only intensifying the debate over these new, more consumer-oriented approaches that could change the meaning of a “public” education.

If their proponents are correct—that freeing educators and parents from an entrenched traditional school system will result in greater student achievement—calls for widespread change will undoubtedly increase.

If, on the other hand, charter schools and voucher programs can’t show measurable academic improvements, critics say there is little reason to continue them.

One thing is certain: Until a consensus is reached one way or the other, the two camps will alternately embrace and deride each new piece of evidence in this high-stakes battle to determine the future of American education.

“This is fundamentally a political struggle,” said Diane Ravitch, a senior research scholar at New York University and a former assistant U.S. secretary of education under President Bush. “It will be resolved in the political arena. And the data, as it’s produced, will be used as ammunition.”

Just Getting Started

School choice advocates and some independent researchers argue it’s unfair and not particularly useful to draw hasty conclusions about charter schools and vouchers. They ask why, when most states have a hard time evaluating traditional schools that have been open for decades, anyone should expect definitive answers about school choice programs still in their infancy.

“Like most other education reforms, these programs will need time to develop and gain momentum,” said Richard C. Seder, the director of the Reason Public Policy Institute’s education policy program in Los Angeles. “Because more scrutiny has been placed on these arrangements, there is a magnifying glass over these schools and students. On the other hand, we only know generically about the quality of most traditional public schools.”

The nation’s first charter school opened seven years ago in St. Paul, Minn., but most of the nearly 1,700 such schools today are less than 3 years old.

As a consequence, there has been only one national analysis of student academic performance in charter schools so far, and it isn’t due to be released by the U.S. Department of Education until later this spring. Many of the 34 states with charter schools have either refrained from scrutinizing the achievement data, or are just starting to do so.

In Massachusetts, for example, a commission charged with evaluating the state’s charter school initiative avoided questions about student academic achievement in a report last December out of concern “that it would be too early to fairly judge.” The state’s first 15 charter schools opened in 1995.

Pennsylvania education officials, meanwhile, had no such qualms. In March, they released part of the first study of the state’s barely 3-year-old charter school program, proclaiming that “student achievement in charter schools is on the rise.”

“Charter schools are no longer an experiment,” Gov. Tom Ridge, a Republican, said at the time. “They’re proving themselves as innovative and effective educational opportunities for Pennsylvania students.”

It’s questionable whether enough proof is available to support that declaration, however. The initial report, prepared by Western Michigan University’s Evaluation Center, is based only on surveys of charter school students, parents, and teachers. Not until a second report is released in the fall will Pennsylvanians have a hard-numbers analysis of student testing data.

“There’s no hard evidence yet,” said Gary Miron, the university’s lead researcher on the project.

Evaluating publicly financed voucher programs is no less challenging.

Only three such programs are in place—in Milwaukee, Cleveland, and Florida—and the latter two are being challenged in the courts.

The Florida plan, the country’s only statewide voucher experiment, is barely a year old and so far affects only two Pensacola schools and 53 students. It is part of an accountability system that grades public schools on an A-to-F scale based on results from state tests. Students from schools that get failing grades two years in a row are eligible for vouchers that allow them to attend private or religious schools of their choice.

In March, a state judge ruled that the program violates the Florida Constitution. Though the decision is being appealed, it left many observers wondering if the program will end before it can offer any useful insights on student achievement.

But not everyone agrees the country should sit back and wait for the school choice movement to mature before testing its effectiveness.

“The newness of the movement is a legitimate argument to an extent, but we’re getting to the point where there are questions out there that need to be answered,” said Todd Ziebarth, a policy analyst with the Denver-based Education Commission of the States.

Mixed Bag

But even a choice program with a few years under its belt doesn’t necessarily provide clear answers about student achievement. Three studies of the Milwaukee voucher plan in the mid-1990s yielded three very different results.

A state-appointed evaluator found the achievement of voucher students was no different from that of public school students, while a Princeton University researcher concluded the voucher program, which began in 1990, had a positive effect on math achievement but didn’t improve students’ reading scores.

A team of researchers from Harvard University and the University of Houston, meanwhile, found that students who had used vouchers to attend private schools for three to four years scored higher in both math and reading than public school students.

Findings in studies of Cleveland’s 5- year-old voucher program have been similarly mixed.

Some observers suggest such contradictory results sometimes have as much to do with the biases of the researchers as with the particulars of the schools being studied.

“There are very few neutral, old-fashioned social scientists doing this kind of work,” said Jennifer L. Hochschild, a professor of public and international affairs at Princeton University who was not involved in any of the Milwaukee studies. “In my view, academics ought to be passionately committed to improving the idea of schooling, not passionately committed to a single method.”

Findings on achievement in charter schools are also inconclusive.

“I’ve been reviewing the independent research out there, and so far there’s no consistent or strong evidence that students at charter schools are doing better than traditional public school students,” said Amy Stuart Wells, an education professor at the University of California, Los Angeles, who led a 1998 study of charter schools in 10 California districts.

That has been the case even in states like Arizona, where charter schools are flourishing and have strong political backing.

A report conducted last year for the state education department by the Morrison Institute for Public Policy at Arizona State University offered a “preliminary” conclusion that charter school students on average were performing about as well as students in regular public schools, though they tended to lag behind their peers by middle school and high school.

And this year, a new scoring system the state uses to rank roughly 1,000 Arizona public schools showed that students in 45 charter schools ranged widely in academic performance over a two-year period.

State schools Superintendent Lisa Graham Keegan, a leading advocate of the choice movement, warns against using such data to draw broad conclusions about charter schools. Instead, she is proposing a plan for shutting down charter schools that aren’t making good on their “explicit call to improve student achievement.”

“Charter schools do not operate as a class, just as regular district schools do not operate as a class,” Ms. Keegan said. “Achievement data really only applies to individual schools, especially when we’re looking at gains.”

But observers like Ms. Wells say charter schools’ varying degrees of success should give state policymakers reason to pause before approving many more of them.

“I think that state legislators ought to be reassessing what it is they hope to accomplish via charter school reform, and shift the focus away from issues of sheer quantity to issues of quality,” Ms. Wells said. “If they can facilitate the creation of really good charter schools, however they define that, I think that such an agenda should take precedence over the creation of laws that simply foster large numbers of charter schools.”

Best Measure?

As with traditional public schools, many question whether standardized tests are even an appropriate measure of educational success for charter schools and voucher programs.

One-quarter of the nation’s charter schools opened specifically to serve students considered most at risk of failing. It’s no surprise then, charter proponents say, if their test scores are below average.

But if test scores must be used in evaluations, those supporters argue, more weight should be given to improvement from year to year than to raw scores.

SER-Ninos is located in the heart of a Houston barrio, and most of its students come from poor households where little or no English is spoken. Teachers here nevertheless believe that given time, their students will not only catch up with the statewide testing average—they’ll exceed it.

They may be right. In just one year, from 1998 to 1999, the school’s overall passing rate more than doubled, from 21 percent to 51 percent.

“I don’t think we’ve been around long enough for people to be bashing us on test scores,” teacher Chris Strane said of the school he has worked for since it opened four years ago in a row of old church classrooms. “Maybe 10 years out, if we haven’t improved . . . but our test scores have improved—every year.”

However, counting on test-score gains alone to show an accurate picture of student achievement can be risky. Because the average charter school is small, serving just 137 students compared with the typical public school enrollment of 475, test scores can fluctuate wildly from one year to the next.

Newland Academy, a charter school located in the rural Michigan town of Manistee, was lauded by the Michigan Association of Public School Academies when the percentage of students scoring at a satisfactory level on the state’s 4th grade math test jumped from 42 percent in 1996-97 to 80 percent the following school year.

But with just 12 students taking the exam the first year and 10 the next, Principal Mary Sue Wilkinson was pleased but wary. And with good reason. When 14 of her 4th graders were tested in math in 1998-99, the percentage of students scoring “satisfactory” dropped 31 points.

“In the years when the test scores go up, I want to stand on the roof and yell, ‘Hey everybody, look at this,’ ” Ms. Wilkinson said. “But I learned my lesson the hard way: These test scores in isolation don’t tell the whole story.”

Further complicating matters is the fact that many choice schools use their autonomy to escape the standard curriculum, not adhere to it.

West Michigan Academy for Arts and Academics, in Grand Haven, concentrates on instructing students in the fine arts. The Central Minnesota Deaf School in St. Cloud focuses on teaching deaf and hard-of-hearing students in a bilingual and bicultural environment. Roots Public Charter School in Washington is geared toward students interested in a “strong African heritage cultural learning environment.”

Whether state test scores can adequately measure the success of such schools is up for debate.

“People are trying to fit these charter schools into an old-school paradigm, and when they can’t do that, they say they don’t work,” said Jeanne Allen, the president of the Center for Education Reform, based in Washington, and a strong supporter of school choice. “I don’t think at the beginning of this movement anyone intended for there to be one measure for these schools.”

Parents’ Choice

Given the concerns about standardized tests, some analysts of educational choice point to parent satisfaction as the ultimate proof of a school’s success.

“Parents love these programs,” said Jay P. Greene, a senior fellow at the Manhattan Institute for Policy Research. “If this were almost any other policy area, that would be the answer. But people don’t believe parents—it’s not viewed as sufficient evidence.” In a study of the Cleveland voucher program after its first two years, Mr. Greene found that nearly half of participating parents were very satisfied with safety, discipline, moral-values teaching, and academic programs in the schools they chose for their children. That satisfaction rate was 30 percent for parents whose children attended regular public schools.

In studies of privately subsidized voucher programs in Dayton, Ohio; Washington; and New York City, a team of researchers led by Paul E. Peterson tracked individual students and compared the test scores of those who won private school vouchers in a lottery with those of students who didn’t get spots. The studies also included surveys of parents and students.

“That parents are satisfied with the schools they choose for their children is a consistent finding—no matter who does the research,” said Mr. Peterson, the director of the John F. Kennedy School of Government’s Education Policy Program at Harvard University.

Some skeptics, though, aren’t willing to make the leap that seemingly happy parents mean school choice is working.

“On what basis would we possibly expect parents to express dissatisfaction with the schools they choose for their children?” said David N. Plank, a professor of education at Michigan State University in East Lansing. “This is a very high-stakes choice, and it is hard to imagine parents confessing to strangers that they have made bad decisions for their kids and put them in bad schools.”

Here in Houston, SER-Ninos Director Charmaine Constantine says she gauges her program’s success largely by the feedback she gets from parents, because getting them involved and invested in their children’s educations is one of the main goals in her school’s charter.

“If the parents aren’t satisfied, or they don’t think we’re doing a good job, then maybe we aren’t, and maybe we should re-evaluate what we’re doing,” Ms. Constantine said. “If the parents are not seeing progress in their kids every day, then we probably aren’t doing our jobs.”

She also recognizes that demonstrating parent satisfaction isn’t enough.

A recent visit to the school found teachers and students gearing up for the spring round of the state’s standardized tests. That effort sometimes entailed straying from the school’s carefully crafted curriculum to drill the children on questions they’ll only encounter on the exams. But Ms. Constantine and her teachers acknowledge the importance of improving on last year’s scores.

“Being different is OK, but this is the real world,” Ms. Constantine said. “We have to be part of the real world, and we have to teach our kids that. I may not agree with [the state tests] as a true measure of what is going on here, but if the state says that’s the measure—that’s the measure.”

In the real world, SER-Ninos is also coming up on the five-year mark at which it will either have its charter revoked by the state or renewed. If there is any doubt about the future, it doesn’t show.

Students, teachers, and parents are busy planning a new campus that will sit on seven acres of newly purchased land nearby. The school will be big enough, they say, to give even more families a choice.