Few Studies Track Post-Katrina School Changes

For education researchers, the waterlogged rubble of New Orleans in 2005 was more than a perfect storm: It was called a perfect opportunity, a “blank slate” on which to test a wide range of education initiatives in a school district already scraping the bottom of the barrel and a community desperate for improvement.

“Two components that made it very compelling,” said Nadya Chinoy Dabby, the director of education programs for the Los Angeles-based Eli and Edythe Broad Foundation, which invested more than $8 million in the years following the storms. “There was the abject, dire need to do something in the wake of the storms … and there was really a chance to remake public education.”

Yet the blank slate has proved much more like an erased chalkboard. Missing data and the ghosts of overlapping improvement efforts make it difficult to get an accurate read on schools’ progress over the long recovery period. Five years later, researchers still struggle to translate the education changes in Crescent City schools into usable models for nationwide reform.

“Every school, the whole system has changed. We have new governance structures, more money than we had before, a new cohort of teachers; we’re paying teachers more, but we have no teachers’ union,” said Shannon Jones, the executive director of the Scott S. Cowen Institute for Public Education Initiatives. “When I think of all the things that have changed, it’s hard to drill down and say what kinds of things work, because there really hasn’t been a control group.”

‘Reinventing Wheels’

While research has become a buzzword in school circles, its use in crafting newer, more effective models of schooling remains spotty across New Orleans.

“Some of [the schools] are using research; some of them, not so much,” said Lauren J. Bierbaum, the research director for the Greater New Orleans Afterschool Partnership.

For example, nearly every school in New Orleans has extended the school day, week, or year in some form, but the practices have ranged wildly from school to school, and many are still “reinventing the wheel,” according to Ms. Bierbaum.

“We see a number of schools that are committed to longer days, but they haven’t done the due diligence to understand how you fill that time with something that improves student learning,” Ms. Bierbaum said.

And, while data are being collected on many different fronts, Ms.Bierbaum said, “What we don’t have is a lot of synthesis and people who can talk about it across silos.”

In the immediate aftermath of the hurricanes, philanthropies like the Broad Foundation, the Seattle-based Bill & Melinda Gates Foundation and the Bentonville, Ark.-based Walton Family Foundation funneled money to rebuild and test education reforms.

In 2006, Gloria Ladson-Billings, the then-president of the Washington-based American Education Research Association, said she argued in an impassioned keynote address that the recovery was “a chance to do more than write a check” and that education researchers had a “moral obligation” to study and support the schools’ recovery. Yet the U.S. Department of Education invested no research money into the Gulf, and the expected swarm of independent researchers never materialized.

“New Orleans doesn’t have a major education research school, so there wasn’t a centralized place to get the funding, skills, personnel to look at the unique changes that were taking place here,” said Brian R. Beabout, a University of New Orleans education researcher.

Information Lacking

Ms. Ladson-Billings, a professor of curriculum and instruction at the University of Wisconsin-Madison, has been “underwhelmed” by the amount of research in New Orleans so far, but also unsurprised. “People generally were in a research project, and they were reluctant to just drop what they were doing to run down there, and … there was so much disarray.”

Most schools lost student records during the flood or the various administrative changes, meaning there were no baseline data for researchers. Schools changed hands rapidly from year to year and administrators were more concerned with getting classrooms up and running than with answering researcher questions or filling data holes in academic files.

“It would be easy to blame it on Katrina, but I also think the chaos after Katrina opened up some windows for a lot of excuse-making in record-keeping or even not enrolling children in school,” said Kelly McClure, the academic director for the seven charter schools in New Orleans that are operated by the Knowledge Is Power Program, or KIPP.

Early on, Ms. McClure said the KIPP network had an “open-door policy” toward education researchers. “It was very sexy to be doing research on New Orleans, and it was good because it raised awareness that we desperately needed,” Ms. McClure recalled.

Yet Ms. McClure said school administrators grew wary of the agendas behind researchers’ requests and reluctant to put in the time required to handle them. Now, KIPP only opens its doors to research that school leaders think will “get the message out about the reality of the situation.”

The disaster also drew researchers from the outside with roots in the area. Adrienne Dixson, an associate professor of education at the Ohio State University in Columbus had taught 6th grade at what was then Charles E. Gayarre Elementary School, and she followed her former principal Roslyn Smith to the newly reopened McDonough 42 Elementary Charter School.

“In New Orleans, relationships with people and who you know really matter,” Ms. Dixson said. “You are welcomed when you come there, but people want to know where you’re from, who your mom is, you know? In most cities, you go to the central office and say, ‘I want to do research.’ … Here, you have to negotiate entry school by school.”

A complicated new governance structure, developed after the storms, played a role in that breakdown in communication for researchers. New Orleans includes five types of charter and regular schools, many serving as independent local education agencies, and accountable variously to the Orleans Parish School District, the Recovery School District, and the state board of education.

As district schools convert to charters, they become responsible for the data and reporting requirements normally assigned to districts: budgets and spending, teacher and student demographics, attendance counts, and behavior issues. Schools follow no uniform timeline or format for reporting the information, and some do not track data in the same way from year to year.

To gather reliable data for its annual “State of Public Education in New Orleans” reports, the Cowen Institute enlists staff members and graduate students to spend 20 hours a week calling the 88 schools and 50 operators to get information, Ms. Jones said.

The ease with which schools move from one governance structure to another also undercuts data quality. For example, Douglas High School has changed locations three times as part of the Recovery School District. After continuing to fail, the school closed last year, and this year, KIPP opened its first high school in New Orleans on Douglas’ last school site.

“If you’re a researcher out of, say, Madison, Wis., and you ask for data from the state department of ed, the state official may not know that there’s a new school [at] this site, or there are two schools at that site, or that the same school has changed locations,” Ms. Jones said.

Schools and nonprofit groups also have conducted basic research on their own. KIPP schools now require all new students to attend a three-week summer school where Ms. McClure and other educators conduct assessments to set baselines for the program’s data-heavy model and create or confirm Individualized Education Programs for special education students, where needed. The network is building an online student-data system that will allow educators or researchers to compare student reading and math achievement to attendance rates, IEP goals, and other indicators.

At the same time, Ms. Bierbaum said, foundations like Broad, Gates, and Walton also are transitioning from their initial recovery investment in the city, prompting community discussions on making support more sustainable. The foundations’ approach over the last five years has been largely aimed at managing crises, Ms. Bierbaum explained, so the question is: “How do we ask them to provide more sustainable support?”

Opportunities Ahead

The U.S. Education Department’s Investing in Innovation, or i3, grant program, also affords a new opportunity to study how the educational changes in New Orleans can be transferred to other urban districts. With one of the grants announced this month, the nonprofit group New Schools for New Orleans will partner with New Orleans’ Recovery School District and the Achievement School District in Nashville and Memphis, Tenn., to scale up a system of charter school networks for school turnarounds. The plan calls for identifying charter operators with proven success records, enlisting them to take over failing district and charter schools, and then tracking how those approaches affect their students’ academic progress.

According to Neerav Kingsland, the New Schools’ chief strategy officer, the grant provides “an opportunity to take us to a much more rigorous level of research here.”

The city also will hold its first formal conference on post-Katrina education research next month at the University of New Orleans.

“Originally, it was just to report on what is happening in New Orleans,” Mr. Beabout said, but that soon changed when U.S. Secretary of Education Arne Duncan came to the city earlier this year to talk about the rebuilding efforts. “It quickly became apparent that a lot of the logic that undergirded reforms in New Orleans had made it to the White House and was very much a part of the national discussion.”

Mr. Beabout expects the conference to get “a little heated,” as researchers, policymakers, and the community start to hash out what the next five years of reform and research will look like.

Yet, he added, “people are very hopeful about education. Under the old system, it was just scandal after scandal after low test scores after scandals, and people just couldn’t see a way out. Now, people do have hope that the schools can be better.”

At the same time, he said, “New Orleans is becoming more and more a model for things that might happen elsewhere.”

“I think it’s a bit early to do that.” Mr. Beabout added, “because we don’t have the evidence in yet.”