| The two major studies of the massive federal program designed to help poor children catch up in school have raised as many questions as they have answered. |
In 1984, after completing the first longitudinal study of Title I, the research project’s director released sobering news. The $40 billion in federal aid spent to help millions of poor children over two decades had, in the long run, done little to improve their achievement.
The low-achieving elementary school students who received extra help from Title I--the largest federal K-12 program--gained at a slightly faster rate than their peers who went without the services, according to a synopsis of the Sustaining Effects study. But those increases didn’t last.
“By the time students reached junior high school, there was no evidence of sustained or delayed effects of Title I,” Launor R. Carter, the retired vice president for the System Development Corp. and the director of the study, wrote that year in Educational Researcher.
Thirteen years later, the most recent longitudinal study of the program found that little had changed--even after the federal government spent another $78 billion from 1984 through 1997.
The two studies have concluded that well over $100 billion has been spent across three decades for a program that research says doesn’t reach its goal--of helping children overcome poverty’s negative impact on their school achievement. They do, however, note that many individual Title I programs succeed, while the poor performance of others drags down the effectiveness of the program overall.
A companion study to the most recent longitudinal analysis identified specific programs that are producing substantial, even dramatic, achievement gains, and other research has tracked areas where Title I produces results.
The program’s administrators are hoping that recent changes in the program will help correct the shortcomings the research has cited. And, given the political popularity and diverse numbers of schools participating in the program, the future of Title I seems secure.
The challenge, then, is for educators, policymakers, and researchers to focus on the question that has dogged the Great Society-era program throughout its history: How can it be structured so the lowest-achieving students will beat the long odds poverty lays against them?
The two major studies, as well as related analyses and other research completed and under way, may provide some answers for the program, which funds instruction, mostly in reading and math, intended to compensate for the learning problems children encounter because they live in or near poverty.
Continued Growth
“We cannot discern any ‘compensatory’ effect over time,” concludes the final volume of the four-year Prospects study, released last spring by Abt Associates, a research firm in Bethesda, Md.
Many researchers expected the four-year study to reach that conclusion. The first-year results, released in 1993, found essentially the same thing.
“There is a long series of studies that suggest ... we have never lived up to the expectations,” said Maris A. Vinovskis, a professor of history at the University of Michigan, who has tracked the research on Title I throughout its history. “Even if Title I is helping, it’s not helping enough to overcome children’s disadvantages.”
| Studies have concluded that well over $100 billion has been spent for a program that doesn’t reach its goals. |
|
Despite these negative assessments, the growth of the program has not stopped.
Congress, especially when it was under Democratic control, kept feeding Title I more money because it reaches almost every school district and thus provides jobs and services in every congressional district.
Members of Congress “see that people want the support, want the money, and want the teachers. That says something to them,” said John F. Jennings, who was an aide to House Democrats when Congress first enacted Title I and through the changes made in 1994. “I don’t think they hear the same criticisms locally as they do from these national evaluations.”
When Republicans took control of Congress in 1995, they tried to slash $1 billion from Title I’s annual appropriation, but were forced to retreat in the face of angry public reaction. Spending on the roughly $8 billion program is poised to jump between $100 million and $500 million for the 1998-99 school year, depending on the decision of a House-Senate conference committee now meeting.
“It’s not research driven,” said Denis P. Doyle, a Washington-based senior fellow for the conservative Hudson Institute, an Indianapolis think tank. “The issue raised is: Who gets the goodies? There’s no pressure to do program improvement.”
In 1999, policymakers will revisit the issue once again when Title I next comes up for reauthorization. As they debate the issue, lawmakers will be faced with data from Prospects and a smaller-scale study the U.S. Department of Education has commissioned of the revised program.
Early Analysis
In 1975, on the 10th anniversary of Title I’s passage, the U.S. Office of Education commissioned the System Development Corp. to conduct a longitudinal study of the program. Beginning with the 1976-77 school year, researchers from the Santa Monica, Calif.-based company tracked the test scores of 120,000 students for four years.
The early analysis was promising. According to a 1981 summary of the first two rounds of testing, Title I students at every grade level increased their test scores at a faster rate than others in a comparable group that did not receive services. Mathematics scores jumped faster than reading scores.
The growth even stayed with the Title I students over the summer, according to the analysis written by a researcher at the U.S. Department of Education, which replaced the Office of Education in 1979.
In 1982, the study’s directors released the findings in a series of draft reports for the department, which was by then under control of the Reagan administration. The new administration, which was seeking a dramatic reduction in the federal role in education, never published a final report. Mr. Carter, the study’s director, released summaries of the findings in congressional hearings and academic articles.
The final results were mixed and not as glowing as the early findings. Title I appeared to have a positive impact for some students over the short term, but it did not help many over the long haul.
That was the most troubling news from the study--that Title I had no “sustaining effects” on student achievement. When students reached junior high school, any positive effects had dissipated, the data showed.
Despite the bad news, the report offered hints about why Title I didn’t have the positive impact its advocates had hoped.
A big reason was that money went to whatever intervention local officials wanted, not necessarily to anything of proven merit, the study found. “There is no simple explanation or description of compensatory education; it is an amalgam of many different programs, practices, and services,” Mr. Carter wrote.
|
| “Even if Title I is helping, it’s not helping enough to overcome children’s disadvantages.” Maris A. Vinovskis, |
Because the instruction was often remedial, drill-and-practice instruction given in pullout programs, the average Title I student did not receive any extra attention from teachers. And often what special attention they did get was from less-experienced teachers than those in their regular classrooms, Mr. Carter concluded.
Positive Findings
While the Sustaining Effects study painted a bleak picture, the program’s supporters found other data to promote it.
For example, in a December 1983 article in the Phi Delta Kappan--the journal of Phi Delta Kappa, the education honor society--Benjamin D. Stickney and Virginia R.L. Plunkett cited scores from the National Assessment of Educational Progress and other standardized tests.
“The NAEP reported that students in schools eligible for Title I posted significantly greater gains between 1970 and 1980 than did students in non-Title I schools at all three age levels tested,” the article said.
And 1976 data from a comprehensive study on reading achievement found that the academic growth of students in Title I and other compensatory education “compared favorably” with that of students not in the programs, the authors reported.
These varied results played little role in the changes Congress made to Title I in the early 1980s. The impetus for change was the political debate between newly powerful conservatives, who wanted to scale back federal K-12 programs, and Democrats who wanted to preserve them, said Mr. Jennings, the former Democratic aide, who is now the director of the Washington-based Center on Education Policy.
|
| The idea behind whole-school funding is that disadvantaged students will benefit if the entire school improves. |
President Reagan’s education team used the Sustaining Effects data to support its plan to merge Title I into a block grant with other education programs and cut the total funding by 25 percent. The program’s supporters in Congress went on the defensive, fearing that if they endorsed modifications they would weaken their argument that it should continue.
“The defenders of the program became overly protective and didn’t want to make any changes,” Mr. Jennings said. “It caused a hardening of positions rather than reaching for solutions.”
40,000 Students Studied
In 1981, then, Congress reduced some of the program’s mandates--the biggest was the requirement that schools form Title I parent councils--and renamed the program Chapter 1, a name that lasted through 1994, when it switched back to Title I. Within a year, Congress passed so-called technical amendments to that bill that “put the program in effect where it was” before, Mr. Jennings said.
In 1993, nine years after the release of the Sustaining Effects data, Title I advocates received similar bad news in the early returns from the Prospects study.
Prospects, like its predecessor, followed a large sample of children over several years through elementary school. They both tried to determine whether the supplemental lessons from Title I helped low achievers improve their test scores to narrow the gap that existed with high achievers.
A “one-year snapshot” of student achievement in the 40,000-student Prospects study found that “there are essentially no differences in the relative annual gains made by students in low- and high-poverty schools, thus leaving the achievement gap between these students unchanged.”
The data did not deter Congress--then still under the control of the Democrats--and President Clinton’s education team from reauthorizing the program in 1994 and calling for increased spending.
But Congress decided to change the program’s long-term focus on remedial skills and required schools instead to use high standards to guide it. The 1994 overhaul also broadened eligibility for schoolwide projects, making it easy for educators to scrap pullout classes and replace them with programs for upgrading achievement in the whole school. The idea is that disadvantaged students will benefit if the entire school improves.
When the final Prospects data were released earlier this year, they painted the same picture as the preliminary results. Even after juggling the numbers every possible way, researchers could not claim victory.
“After controlling for student, family, and school differences between Chapter 1 participants and nonparticipants, we still find that participants score lower than nonparticipants and that this gap in achievement is not closed over time,” the authors wrote.
Like Sustaining Effects, Prospects found that the quality and depth of the programs varied dramatically from school to school. “Chapter 1 was not a uniform program, but a collection of services and practices provided in participating schools using federal resources,” the new report said.
In the final Prospects report, researchers said their data could not conclude definitively whether Chapter 1 had helped the students. They only showed that the program failed to decrease the achievement gap between those in the program and high-achieving students.
Conflicting Studies
As in 1984, other researchers found other numbers to declare Title I a success.
A 1993 RAND Corp. study concluded that Title I “achieves modest short-term benefits” and that many Title I programs produce “outstanding results.” Because the scores from stellar programs are averaged into laggard ones, the national picture often shows little impact, the Santa Monica, Calif.-based think tank concluded in its review of earlier studies.
In a comprehensive review of 1993 Prospects data, Sustaining Effects, and other significant studies, two University of Chicago graduate students concluded that the program did help the children it served.
“When you look at existing evidence ... compared to similarly disadvantaged kids, those kids who participated in the program tend to perform better than those who did not,” said Geoffrey D. Borman, the co-author of an article that presented those conclusions in the Winter 1996 issue of Educational Evaluation and Policy Analysis.
Title I is a “funding stream. And it’s not plausible to expect a funding stream ... to have an effect on teaching and learning.” David K. Cohen, |
|
Mr. Borman, who is now an associate research scientist at Johns Hopkins University in Baltimore, said he will reanalyze the final Prospects data to see whether they can show positive results in spite of the failure to close the achievement gap.
A Closer Look
While Mr. Borman and other researchers reconsider the Prospects data, others are focusing on its companion study.
The Special Strategies study, led by a team at Johns Hopkins, followed student learning in schools using specific programs over the same period as Prospects. It tried to document the impact of various reform models on student test scores. The study compared outside programs--such as the Success for All model and the Comer School Development Program--with locally developed strategies, such as extending the school year and pursuing schoolwide efforts to improve learning.
“What is new in Special Strategies is the strength of evidence that some programs, well implemented, appear to help students make dramatic academic progress,” the study concluded.
Among its findings were that “pursuing schoolwide change may well be worth the effort,” and that “intensive early intervention may yet be the best bet.”
But, the study warned, “after a third of a century of research on school change, we still have not provided adequate human and fiscal resources, appropriately targeted, to make large-scale program improvements a reliably consistent reality in schools serving students placed at risk.”
Successful Approaches
The Sustaining Effects, Prospects, and Special Strategies studies clearly showed that Title I is essentially a hodgepodge of whatever local school officials want to do. Some will use their money to run a recognized, national program; others keep running pullout programs with a remedial focus.
Special Strategies found the Success For All and Comer School Development programs to be especially successful. Success for All, developed at Johns Hopkins, puts an intense focus on fostering reading skills at an early age, while the approach created by noted Yale University psychiatrist James P. Comer builds support from the faculty and community for a curriculum that seeks to raise achievement.
One Success for All site, where the school followed the program closely, “produced very large academic gains,” and a “less-solid implementation also produced gains, although not as large,’' the Special Strategies study found. Likewise, “where the [Comer] program is well-implemented ... academic results can be impressive.” Other innovations, such as ones where a school operated a program of its own design, didn’t produce the same kinds of gains.
Such findings “challenge the view inside the Beltway that Title I is a program,” said David K. Cohen, a professor of education and public policy at the University of Michigan. “It’s a funding stream,” he added, and “it’s not plausible to expect a funding stream ... to have an effect on teaching and learning.”
There is a growing sense of optimism among those who work in the program that the changes Congress made in 1994 and is considering this year will make a difference.
“We finally know we’re going in the right direction,” said Mary Jean LeTendre, the director of the Title I office at the Education Department.
|
| Title I appeared to have a positive impact for some students over the short term, but it did not help many over the long haul. |
The 1994 changes, most say, take away the remedial, pullout focus and spur schools to do things that will create improvements in the whole school, with the disadvantaged students getting a better education because of them.
Further Changes
Congress isn’t done tinkering with the program, however. A House bill would set aside $150 million to pay for comprehensive reforms in Title I schools. The amendment specifically suggests the Success for All and Comer models as ones principals should consider. A House-Senate conference committee will decide whether the new money should stay in the final bill.
“Title I and Chapter 1 have never had anything that would give encouragement to schools to invest in proven programs,” said Robert E. Slavin, who started Success for All and is a co-director of the Center for Research on the Education of Students Placed at Risk, based at Johns Hopkins. “This could be the beginning of what Title I could be: fuel for reform rather than paying for pullout programs and aides.”
But some researchers and practitioners question the focus on changing the whole school--rather than improving performance of specific students.
“They keep trying to show that these whole-school change models work,” said Stanley Pogrow, an associate professor of educational administration at the University of Arizona.
Mr. Pogrow has created a pullout program--a design now out of favor in the program--that teaches high-level skills to students. By giving a specific population the help they need, he argued, schools will get the results they want.
Mr. Slavin and many others in the research and policy worlds disagree.
“Saying the schoolwide change doesn’t work is like saying medicine doesn’t work,” Mr. Slavin said. “Some medicines work. Some medicines don’t work. ... The details of [the schoolwide program] are all that matters.”
But skeptics have heard such optimism before.
“Parachuting in a reform is not going to work,” said Mr. Doyle, the Hudson Institute scholar. "[Mr. Slavin] may have a wonderful reform, but people should be free to adopt or reject it.”
