Gates High Schools Get Mixed Review in Study

A new evaluation of the Bill & Melinda Gates Foundation’s campaign to improve U.S. high schools offers a decidedly mixed picture of the early returns the foundation is getting from the roughly $1 billion it has invested in the initiative so far.

The in-depth study, commissioned by the foundation and scheduled for release this week, identifies several shortcomings in the high schools that have been started or redesigned with the philanthropy’s support, especially in instruction and student performance in mathematics. It also cites signs of progress in reading and language arts, and concludes that many Gates-funded small schools, particularly those built from scratch, offer a positive learning climate for students.

Evaluation reports on the Bill & Melinda Gates Foundation‘s high school initiatives are posted by the organization.

“In summary, the first returns are promising for English/language arts but worrisome for mathematics,” the researchers say in the report.

The study’s authors caution, though, that the achievement data available to them were insufficient to draw definitive conclusions about student performance. They supplemented the data with analysis of teacher instruction and student schoolwork to provide a more meaningful portrait.

Among the most disheartening findings of that analysis—and one the researchers said also applied to comparison schools in their study that do not receive Gates support—was the lack of rigor in teacher assignments and student work, especially in math.

“[W]e concluded that the quality of student work in all of the schools we studied is alarmingly low,” the evaluation says. “This is not surprising, however, because students cannot demonstrate high-quality work if they have not been given assignments that require … deep understanding” and higher-order thinking skills.

The study, the third in a series financed by the foundation, was conducted by researchers at the Washington-based American Institutes for Research, or AIR, and the Menlo Park, Calif.-based SRI International. It comes as the foundation is awarding new grants this fall aimed at helping districts with high school improvements, including the school systems in Portland, Ore., Los Angeles, and Atlanta, with more expected in the coming weeks.

Tom Vander Ark, the executive director of education initiatives for the Gates Foundation, said the evaluation’s findings were consistent with information he’s gleaned over the past couple of years, and he said the Seattle-based philanthropy had taken steps to adjust accordingly.

He cited, for instance, a greater emphasis on districtwide measures intended to improve the quality of curriculum and instruction, as well as an emphasis on using proven school models.

“I didn’t find anything surprising in the report,” he said in an interview last week. “I think it’s fair to say we’ve learned a number of important lessons, and it’s quite clear that over the last six years I created a number of grant programs that were well-intentioned but had some weak assumptions.”

Math Found Lagging

The Gates Foundation has committed about $1 billion since 2000 to support the start-up of small high schools or the restructuring of large existing schools into smaller units.

Gates officials often emphasize that school size is not an end in itself, but a tool to build strong relationships between teachers, students, and families, and to deliver a rigorous curriculum that is relevant to students’ lives.

Foundation officials say that, in all, the philanthropy has supported more than 850 new schools and nearly 700 existing high schools. Funding has gone to schools in nearly 275 districts. (Education Week also receives funding from the foundation.)

In general, the nearly 300-page study found more positive outcomes, from achievement and student work to academic climate, in schools that were newly created than in those that had undergone redesign.

“Based on the data currently available, it appears that new schools have been successful with respect to attendance, test scores, and the quality of students’ work in English/language arts,” the researchers write.

Math appears to present the biggest problem, in student test scores as well as instruction and student schoolwork.

“In general, the math achievement level of students attending new schools is on par with or lagging behind other schools in the same district,” the report says.

The report provides standardized-test data from one or two school years from four unidentified urban school systems. The researchers also examined results from four other districts for purposes of analysis.

Attendance rates at new schools were generally higher than at other schools in the district. Average reading scores were lower than the district average in some systems, but that finding was reversed, the study found, after statistically controlling for the level of students’ prior achievement.

In two of the three districts where trends on state test data could be examined, the study found larger improvements in reading and English language arts achievement over time in foundation-backed schools than elsewhere in the district. The third district saw gains in that area “on a par with the rest of the district.” The reading gains were larger in the system with new schools than in the district with redesigned schools.

And the researchers stress that the Gates-funded schools tend to serve largely disadvantaged students who enter high school well behind grade level.

“When we look at 8th grade achievement levels, we find almost universally … students start substantially behind other students going elsewhere in the district,” said David A. Rhodes, an AIR researcher involved in the study.

‘A Huge Challenge’

For redesigned schools, the researchers were still more cautious about drawing conclusions on test-score data.

They found slightly lower-quality student work than in the Gates-funded start-up schools. But in examining the rigor of teacher assignments in math, the report says the results were “similarly poor” across the two school types.

“Half of the assignments at both types of [Gates-funded] schools exhibited little or no rigor,” the report says.

Mr. Vander Ark cited several factors that he believed contributed to the problems in math.

“Five years ago, the nation’s attention was … more on literacy than on mathematics,” he said. He also noted that in small schools, there may well be extra faculty support for addressing students’ language arts difficulties, since those areas overlap with several disciplines.

“The same is less true for mathematics,” he said. “In a small school, you may have two math teachers working on their own with students that are a number of years below grade level, and they face a huge challenge with limited support.”

As for student attendance, it was relatively strong at new schools but fell short at the redesigned campuses.

The researchers cautioned that the redesigned schools have a history of underperformance that must be changed.

“You really need to think of the redesign reform as on a longer time frame,” Mr. Rhodes of AIR said. “And that’s going to emerge out of an existing stream of data that, to be blunt, is going to be dismal to start with, because a school isn’t going to reinvent itself if it’s already doing a good job.”

Some of the most promising findings had to do with what the researchers termed “creating a culture for learning.”

Still, the redesigned schools were seeing what the researchers described as “slower progress as they work to change existing structures, cultures, and beliefs.” Those schools, too, showed signs of improvement over time, most notably in fostering the creation of a school culture in which students feel known by their teachers and supported by them both in academics and personally.

Strategic Shifts

Mr. Vander Ark of the Gates Foundation said the philanthropy has been shifting its grantmaking approach, particularly in the past past two years, to help address many of the problems the research spells out.

First, he said, “we try to approach school improvement in the context of a system, a school district. ... One way to think about it is, we took school-as-the-unit-of-change too far.”

Second, the foundation is focusing its school efforts on “well-specified school models that provide really strong support,” including a well-developed curriculum and instructional approach, Mr. Vander Ark said. Models he cited included the networks of schools affiliated with the Providence, R.I.-based Big Picture Company and the San Francisco-based Knowledge Is Power Program, or KIPP.

Mr. Vander Ark said the foundation has much more explicit expectations for school-level change than it initially did.

“Another thing I got wrong at the beginning was autonomy,” he said. “I visited 100 great schools and made the observation that they were all small, autonomous, and assumed that was a path to school improvement. It turns out giving a failing school autonomy is a bad idea.”

On the subject of change throughout whole districts, he cited the foundation’s $2.3 million grant last spring for a planning effort to improve Chicago’s districtwide approach to curriculum and instruction.

Another grant just issued, $1.4 million to the Atlanta public schools, will pay for district plans to redesign high schools’ curriculum and instruction, and will consider the role of new schools in that effort.

“With an investment of $2 million in a plan, you can help a city make better use of a billion-dollar budget,” he said. “Those plans typically incorporate small schools and small learning communities, but they pay more attention to curriculum and instruction.”

The AIR-SRI evaluation offers a series of recommendations, including a call for the foundation to support professional development, technical assistance, coaching around math content and instruction, and efforts to provide curricular materials.

It also focuses on the need for sustainability, noting that new and redesigned schools are “vulnerable organizations, with limited internal capacity and numerous external challenges.”

“The foundation and its grantees may want to focus more of their energy and resources on protecting the schools that have already been started,” the report says, “even if means starting fewer new schools.”