Bemoaning the quality of education research—and offering prescriptions for improving it—have become popular pastimes in recent years. Now, some scholars say, it is time to take a hard look at the consumers who use, underwrite, ignore, or misconstrue the knowledge born from studies in the field.
“The idea is really to think about the soft tissue that connects researchers to public officials, and to think about how do you create an environment where people are rewarded for doing good research and punished for not doing it, or not using it,” said Frederick M. Hess, the director of educational policy studies for the American Enterprise Institute, the Washington think tank that was slated to sponsor a daylong conference on the topic this week.
The May 21 conference, which was expected to draw upwards of 275 participants, comes at a time when the ecosystem that surrounds and breeds education research is in flux.
“It’s the best of times and the worst of times,” Jeffrey R. Henig, a professor of political science and education at Teachers College, Columbia University, writes in one of 10 papers scholars are scheduled to present and discuss at the gathering. “Paradoxically, education policy research is simultaneously charged with irrelevance and with being a powerful political weapon capable of swaying public opinion and legislative votes.”
Impact on NCLB
The paradox comes in part because the path from research to policy is more convoluted, and nuanced, than commonly perceived, the scholars say. A case in point, one of the papers argues, is the federal No Child Left Behind law, the centerpiece of President Bush’s education agenda.
On one hand, the law brought attention to the concept of “scientifically based research” by repeating the phrase more than 115 times, in a multitude of contexts.
The conference papers are posted by the American Enterprise Institute.
But in their paper, analysts Paul Manna and Michael J. Petrilli, the vice president for national programs and policy at the Washington-based Thomas B. Fordham Foundation, report that congressional aides have mixed views on how useful that repetition was.
“I would say researchers were at the table and political concerns played in, but they weren’t the whole story,” Mr. Manna, an assistant professor of government and public policy at the College of William and Mary in Williamsburg, Va., said in an interview. “But to say that writing this law was an exercise in looking at what the best research said, that’s not true.”
Less than a fifth of the 1,169 witnesses appearing in the 155 hearings leading up to the passage of the 5-year-old law were from the research profession, affiliated with think tanks, universities, or applied education programs, according to Mr. Manna and Mr. Petrilli, who was a politically appointed official for the U.S. Department of Education’s innovation and improvement office during President Bush’s first term.
Yet, among the most frequent witnesses with ties to the profession, few were affiliated with university-based education schools.
Inspired by educational research cited by three different advocacy groups, the authors note, members of Congress added language to the law requiring schools to ensure that teachers were “highly qualified.”
At the urging of teachers’ unions, though, lawmakers inserted what the analysts call a “loophole”—a provision permitting experienced teachers to demonstrate their knowledge through portfolios of classroom products.
An unnamed Republican staff member told Mr. Manna and Mr. Petrilli that, while repeating the phrase “scientifically based research” in the NCLB law so often was “kind of silly,” it was also important because the law was “aspirational.”
A Democratic staff member, also anonymous, worried that overuse of the phrase diluted its power.
A major force driving changes in the landscape surrounding research and policy, virtually all the scholars contend in their papers, has been the growth in recent decades of advocacy-driven research.
Groups doing such research include Fordham, which promotes educational choice and accountability, the Heritage Foundation, which tends to support market-oriented approaches to schooling, among other issue areas, and the Education Trust, which focuses on closing achievement gaps between students.
Web Drives Change
In one respect, Mr. Manna points out, such organizations provide a valuable pipeline for research by synthesizing findings and making them more accessible to policymakers. Yet they also tend to select and underwrite studies that support their perspectives.
Conference presenters also contend that the media and the growth of the Internet are also driving significant changes, both good and bad.
The Web has made it easier for researchers to quickly publicize their findings, for instance, rather than waiting for them to percolate through traditional academic channels.
Basic research made up just 3.5 percent of the U.S. Department of Education’s budget for research and development in fiscal 2005.
BASIC:
Study of fundamental aspects of phenomena without specific uses in mind
APPLIED:
Research focused on means for meeting specific, identified needs
FOUR D’s:
Development, dissemination, data, and direct services
SOURCES: National Science Foundation; Jeffrey R. Henig
Dan D. Goldhaber, another paper’s author, said that trend increases the likelihood that biased or flawed research will reach the public without much peer review.
Foundations, which often weigh the influence of the studies they pay for by the number of news stories they generate, also fan such trends, according to Mr. Goldhaber, the director of the Center on Reinventing Public Education at the University of Washington in Seattle.
The result, he said, is that public education debates become polarized.
“The more provocative or the more controversial that you can be,” he said in an interview, “the more likely you’re going to get your work picked up or to get more publicity,” while more measured findings tend to get lost in the mix.
Mr. Henig, for one, also raises a red flag about more-objective research organizations, such as the American Institutes for Research in Washington or the Rockville, Md.-based Westat, that do studies under contract to the Department of Education and other sponsors. For fear of offending potential clients, he said, such organizations can “fall into a kind of blandness in their reporting and accept restraints on what they can say in public from the folks paying for the research.”
Funding Seen as Meager
Another ingredient, most of the papers say, is a scarcity of dollars for education research. At the federal level, such research has historically been underfunded, according to Mr. Henig. “For every $100 spent on research, less than $2.25 goes to the social sciences and less than 41 cents goes to research within the Department of Education,” he writes.
Likewise, school districts rarely have the resources to sponsor studies and, when they do, they often end up doing small-scale replications of work done in other districts, according to Mr. Goldhaber. In academia, he added, neophyte scholars who are strapped for cash often build their reputations and gain tenure by doing low-cost studies, such as small, qualitative case studies or analyses of data collected elsewhere.
“The kind of work that you probably want most in education,” he said, “is work that is generalizable.”
Although the scholars have no plan as a group to offer recommendations for improving the “connective tissue” joining research and practice, some papers advance a few suggestions. Mr. Goldhaber called on districts, and possibly states, to collaborate in sponsoring large-scale studies tailored to their needs.
Role for Collaboration
James S. Kim, an assistant professor of education at Harvard University’s graduate school of education, suggests forging scientific collaborations between research and professional organizations, such as teachers or policymakers, to build receptivity for the findings.
Mr. Henig urged restraint. “I think we’d be better off socially if we set ourselves on a stage for pursuing research that will give us better information 10 years down the line than rushing into the field studies designed to give feedback 10 months down the line,” he said.
That point, however, drew disagreement from one of the researchers slated to discuss the papers at this week’s conference. “What is the purpose of the creation of knowledge if not to inform practices and processes and behaviors?” Gina Burkhardt, the chief executive officer of Learning Point Associates, a Naperville, Ill.-based research group, said in an interview. “The purpose ought to be for social good.”
