To revolutionize education, we first have to understand it.
Now that the No Child Left Behind Act has proclaimed that all kids can learn and it is the job of grown-ups to figure out how to teach them, enormous attention has been focused on exactly what grown-ups can do to help kids learn. But this has exposed the dirty little secret of education: For the most part, we have almost no idea what works.
Here’s an example: For children whose first language is not English, what is the best way to ensure that they learn not only English, but all the math, history, and science that they should learn? Is it best to teach math, history, and science in their home language while teaching English separately? Or is it to intensively teach English, leaving aside the other subjects until English is mastered? Does it make a difference if the child comes to this country when he is 2 years old or when he is age 12? Does it matter whether the child’s first language has a lot of overlap with English, like German or Spanish, or if it is completely unrelated, like Turkish or Chinese? Does it help or hurt to continue to use the home language outside of school?
For a nation of immigrants in the middle of a huge wave of immigration, it would seem important to know the answers to those questions. Unfortunately, we have no idea.
Individual educators may have implemented successful practices in their schools, but without linking those practices to research demonstrating that what they do could be successful with other kids, all we have are individual experiences, not standard practice. This leaves us with philosophies. We have a bilingual philosophy and an English- first philosophy, complete with testimonials about what worked for whose grandparents, but these really are political—bordering on the religious— arguments, rather than scientific ones.
G. Reid Lyon, the chief of the child- development and -behavior branch within the National Institute of Child Health and Human Development and the director of the institute’s reading-research program, has commissioned a major study on what methods of English-language instruction work best for which kids, but we won’t have the results for some time.
That multiyear study, however, is one small part of the massive revolution that the federal No Child Left Behind legislation is spurring in the field of education, where serious research is going on to determine, as one researcher put it, “what works for which children under what circumstances.”
The idea behind the What Works Clearinghouse is to set standards for judging the evidence about the effectiveness of educational approaches.
Mr. Lyon is fond of the following quotation: “The history of the profession has never been a particularly attractive subject in [professional] education, and one reason for this is that it is so unrelievedly deplorable a story. For century after century, all the way into the remote millennia of its origins, [the profession] got along by sheer guesswork and the crudest sort of empiricism. It is hard to conceive of a less scientific enterprise among human endeavors. Virtually anything that could be thought up for treatment was tried out at one time or another, and once tried, lasted decades or even centuries before being given up. It was, in retrospect, the most frivolous and irresponsible kind of human experimentation, based on nothing but trial and error, and usually resulting in precisely that sequence.”
The profession described in this quotation is not education—though Mr. Lyon says it could be— but medicine, and the person who said it was the eminent physician Dr. Lewis Thomas, the former president of the Memorial SloanKettering Cancer Center in New York City.
What changed medicine was that about a century ago the medical sciences adopted standards for medical research and requirements that every doctor master—and every medical school teach—a body of knowledge that included pathology, anatomy, organic chemistry, and so on. Later, the federal Food and Drug Administration set guidelines for evaluating drugs and other medical treatments, so that Dr. Whatsit’s Magic Cures could no longer call themselves medicines unless they met rather rigorous standards for safety and effectiveness, usually in controlled, randomized trials.
As an attempt to put education on the same kind of scientific footing as medicine, the National Academy of Science has issued some clear guidelines on how to make education a more research-based profession, akin to medicine or agriculture, both of which have established massive partnerships between practitioners (doctors and farmers) and researchers.
And, as something of a parallel to the FDA, the U.S. Department of Education has set up an entity called the “What Works Clearinghouse.”
The idea behind the clearinghouse is to set standards for judging the evidence about the effectiveness of educational approaches. Wouldn’t it be nice to know, for example, if “Hooked on Phonics,” really helps teach kids to read? Right now, all we have is the publishing company’s commercials to go on. Even if it is helpful, is it more or less helpful than Harcourt Brace’s reading program? Wouldn’t it be nice to know whether the math program your school system just adopted actually helps kids understand the major concepts and master the necessary skills of math? How about peer mediation, a program popular in many elementary and middle schools—does it really help reduce the number of personal disputes and prevent kids from developing such intense grudges that they bring AK-47s to school?
The No Child Left Behind legislation is scary because it requires a rate of improvement that would make NASDAQ during the dot.com frenzy look sluggish.
The What Works Clearinghouse is supposed to help clear away some of the underbrush and give educators reliable information about those and other topics, but it has a long way to go. After all, we really don’t know much about what helps kids learn. We have lots of ideas, but what we know through reliable, replicable research is rather limited. We know a little about early- childhood experiences; we know a little about reading instruction; we know a little about classroom organization; we know a little about what makes a good teacher. After that, it’s a stretch to say we know much about anything.
“And if you turn to math, we are really pathetic.” That’s how Phoebe Cottingham puts it. Ms. Cottingham is one of three commissioners on the Department of Education’s Institute of Education Sciences, and oversees the What Works Clearinghouse. “Education is one of the most important sectors of society, and we have largely ignored the fact that we have no basis for making decisions about our programs, our policies, even what teachers do or what kinds of materials they use,” she says. “We have many hypotheses, many hunches, and that’s all well and good. But we haven’t field-tested them.”
She is careful to say that the clearinghouse will be very different from, say, the Food and Drug Administration, because it will not have approval power. It will act more like Consumer Reports, subjecting educational programs to a number of criteria and then reporting the results. But if it continues as planned, it could get important information about effectiveness to teachers, principals, superintendents, school boards, and parents, who in the past have had to rely on what Dr. Thomas called, in a different context, “the crudest sort of empiricism.”
In December, the Institute of Education Sciences issued its criteria for what standards should be used to judge education research (“Identifying and Implementing Educational Practices Supported by Rigorous Evidence: A User Friendly Guide”). (“Ed. Dept. Issues Practical Guide to Research-Based Practice,” Jan. 7, 2004.) The standards are bound to be controversial, in part because they relegate most of the education research that has been conducted thus far to second-rate status. Most education research, after all, does not rely on what the institute calls the “gold standard” of research: “well-designed and implemented randomized controlled trials,” such as are conducted in medicine and psychology.
For example, the guide says, if school officials want to test whether a new math curriculum is more effective in teaching math to 3rd graders than the current math curriculum, the best way to study that would be to randomly assign a large number of 3rd grade students to either the old curriculum or the new curriculum and keep all other conditions for the children the same. By keeping track of how students in each group perform, it should be possible to draw conclusions about the effectiveness of the new curriculum.
This is certainly not what most schools or school systems have done in the past. If they have studied the effect of a math curriculum at all, it has been to introduce a new math curriculum and then see where test scores were in comparison to the past. The Institute of Education Sciences calls that “pre-post study design,” and says it often produces erroneous results because it cannot answer “whether the participants’ improvement or decline would have occurred anyway, even without the intervention.”
The What Works Clearinghouse will be using these new criteria as it evaluates whether publishing companies and curriculum designers have tested their products in ways that give substantive information about what works for which kids under what circumstances.
We really don't know much about what helps kids learn. We have lots of ideas, but what we know through reliable, replicable research is rather limited.
Educators will then be able, if all works as planned, to see what kind of research supports—or doesn’t support— the use of, say, a newly published basal reading series. A superintendent who has serious discipline problems that interfere with instruction should, in theory, be able to see what programs have been proven in other places to reduce fighting and disruptive behavior and produce increases in academic achievement, and under what circumstances.
This really represents a turning point in education, because until now, educators have had to rely on the claims of those who produced the programs themselves or on research that may or may not have followed rigorous design standards. For the first time, they should be able to get the kind of information they need in choosing what to do to make improvements in their schools and classrooms.
There is no question but that our knowledge base about what helps kids learn is still weak. But with clear, rigorous standards by which we can judge education research, the education world should be able to begin to build a foundation of knowledge comparable to that established in other fields.
The No Child Left Behind legislation is scary because it requires a rate of improvement that would make NASDAQ during the dot.com frenzy look sluggish. Few organizations have ever improved at the rate required by this federal law. And it is doubly scary because of this weakness of our knowledge base about what helps kids learn.
But the law’s implacable demands are helping spur on an enormous intellectual revolution in education, a revolution comparable to what happened in medicine a century ago, and should produce the kinds of improvements in learning that medicine has provided in health.
Karin Chenoweth is a columnist for The Washington Post. Her Homeroom column appears in the newspaper’s special sections for Montgomery County and Prince George’s County, MD.
