The earliest testing data from the first four schools run by the Edison Project show some positive results. But the results should be interpreted cautiously, experts stressed.
In two of the schools managed by the for-profit company during the 1995-96 school year--in Wichita, Kan., and Mount Clemens, Mich.--kindergarten students showed substantial gains in reading compared with control groups.
First graders at those schools also showed significant gains in reading; the results were statistically insignificant for 2nd graders.
The results, released in recent weeks, are based on tests given or overseen by the Educational Testing Service of Princeton, N.J., at the Edison Project’s request. They were interpreted by Robert J. Mislevy, a research scholar at the ETS who acted as a consultant for New York City-based Edison.
Reading results at Edison’s other two first-year schools were more difficult to interpret. The Boston Renaissance Charter School had no matching control school, and the elementary school in Sherman, Texas, had what turned out to be a poorly matched control group, Mr. Mislevy says in his report.
But at the Dodge-Edison School in Wichita, he writes, the results were somewhat promising. There were “large significant effects in favor” of the Edison reading program over the control group at the kindergarten level, “mixed significant and nonsignificant effects” in 1st grade, and no significant differences in 2nd grade, the report says.
In Mount Clemens, the results showed a similar pattern in the three grades, where students were tested for vocabulary, oral reading, reading comprehension, and other skills.
The testing was geared toward the Success for All reading program used by the Edison Project schools and by numerous other schools around the country.
“Success for All has been carrying out their own studies for five years now, so I suggested to Edison that a good way to carry out their own evaluation was to follow as closely as possible the evaluations of Success for All,” Mr. Mislevy said last week.
The reading tests included the Peabody Picture Vocabulary Test, the Durrell Oral Reading scale, and the Woodcock Reading Mastery Tests.
Instant Results
The issue of how educational experiments such as the Edison Project should be evaluated raises a host of important questions.
Because they have generated much controversy, the private efforts by Edison, Education Alternatives Inc., and other companies to manage public schools have been under a microscope. Often the very survival of a company’s contract with a school district depends on its ability to show improved student performance.
In Baltimore, EAI ran into trouble over the accuracy of test-score data from the 12 public schools it managed. Critics questioned whether the results showed the gains that the Minneapolis-based management company claimed.
The Baltimore school board terminated its contract with the company last year, citing financial issues.
Eric A. Hanushek, a professor of economics and public policy at the University of Rochester in New York, who has studied school reform, said that too often there is little thought given beforehand to how educational experiments will be evaluated.
“There is always this need to evaluate things immediately when in fact we believe many educational projects will only be effective over some length of time,” he said. “I think Edison has been better than most other private concerns and much better than the public schools, in that they have tried to set up an evaluation structure.”
Mr. Mislevy of the ETS sounded cautionary notes about the Edison results.
The positive results in Mount Clemens and Wichita “may have a lot less to do with the fact they are under a private management company than because they have a team that can get a successful reading program in,” he said. Also, he added, “you see more results fast when you focus your testing on things you are doing specifically in your program.”
Greater Exposure
Edison Project officials suggested that the main value of the results is that they will serve as a baseline for future years. Yet John Chubb, Edison’s curriculum director, was willing to draw some positive conclusions.
“The differences between Edison and control kids tended to favor Edison,” he said.
Mr. Chubb and Mr. Mislevy agreed that the greater improvement in kindergarten was expected, because kindergarten students have had the greatest exposure to Edison’s program--all of their formal education. A key question in future years will be whether those same students continue to show accelerated achievement, the experts said.
Among the study’s other conclusions are:
- The Boston Renaissance charter school had no control school, but its reading performance was comparable to that of the other Edison schools and of other schools that use Success for All.
- At the Washington-Edison school in Texas, the control group performed better than the Edison school. But researchers believe the control group was not properly matched.
The Edison school had many more students from groups “associated with lower reading performance,” including more black and Hispanic students, more poor students, and more students whose first language is not English, Mr. Mislevy said.
