Federal

‘Scientific’ Label in Law Stirs Debate

By Debra Viadero — October 16, 2007 5 min read
  • Save to favorites
  • Print

While other ideas for revamping the No Child Left Behind Act are taking center stage, a quiet debate is unfolding over proposals to tinker with the law’s definition of what constitutes “scientifically based research” in education.

The phrase is one of the most oft-repeated in the lengthy text of the nearly 6-year-old law. Sprinkled through the federal education statute more than 100 times, the references to “scientifically based research” require educators to rely on such studies in choosing everything from approaches to reading instruction to anti-drug programs for students. And that’s not to mention the law’s use of such related terms as “evidence-based” research.

But the legislative definition of “scientifically based research,” which favors randomized or experimental studies over other kinds of research in determining what works in schools, has also been criticized for promoting a narrow view of educational scholarship.

Leaders of the House Education and Labor Committee, in a draft proposal for reauthorizing the NCLB law circulating since late summer, would tone down that emphasis on scientific experiments by stipulating that studies aimed at determining whether an educational program or practice works may include—but are not limited to—random-assignment experiments.

The proposed change has raised concerns among some federal education research officials and research groups.

“I’m not thrilled with that provision,” said Grover J. “Russ” Whitehurst, the director of the Institute of Education Sciences, the research arm of the U.S. Department of Education. He has been spearheading the department’s efforts to promote rigorous education research.

“I don’t know of any credible expert who doesn’t think a randomized research design doesn’t give you the best possible answer to causal questions,” he added.

The Gold Standard

The Knowledge Alliance, a Washington group representing a mix of research and research-and-development centers, also voiced opposition to the draft bill’s proposed change to members of Congress.

But other Washington-based education groups, such as the American Educational Research Association and the Software & Information Industry Association, say the new language would be an improvement over current law.

BRIC ARCHIVE

“For a bill like NCLB, we think it reflects a much more appropriate understanding of research methodology,” Gerald E. Sroufe, the director of government relations of the 25,000-member AERA, said of the proposed change. “My opinion about RCTs [randomized-control trials] is that they are underutilized in educational research and overemphasized in political discussions.”

Known as the “gold standard” for answering cause-and-effect questions, randomized field trials are those in which participants are randomly assigned to either experimental or control groups. Such studies are rare in education, partly because educators balk at giving an intervention to one group of students but not another. While such experiments can determine what works, scholars say, they can’t explain why.

The House draft bill’s proposed definition of “scientifically valid” research is modeled on one used in the Education Sciences Reform Act of 2002 , which created the IES. But the draft legislation would omit the preference for randomized studies that is now expressed in both the NCLB and the ESRA laws.

Scientifically valid research, the House committee’s proposal states, would include studies that are “rigorous, systematic, and objective” and that present findings and make claims that are “appropriate” to the methods used. The discussion draft also gives a nod to examining the weight of evidence—in other words, the consistency of findings across multiple studies or sites—and presenting clear, detailed data so that findings can be replicated.

“I think there’s a general movement away from the more rigorous definition of the so-called ‘gold bar,’” said Charles Blaschke, the president of Education TURNKEY Systems Inc. of Falls Church, Va., which tracks federal education legislation for publishers.

BRIC ARCHIVE

James W. Kohlmoos, the president of the Knowledge Alliance, said the apparent backlash against the current law’s references to “scientifically based research” on Capitol Hill may stem from bad publicity over the Education Department’s management of the Reading First program. The program requires states to use practices grounded in “scientifically based research.”

In a series of reports issued since fall 2006, the department’s inspector general said program administrators may have steered the grant-application process for the $1 billion-a-year initiative to ensure that particular reading programs and instructional approaches were widely used by participating schools, and that others were left out—including some that had solid research track records.

“Reading First didn’t help promote the legitimate implementation of ‘scientifically based research’ and caused a lot of confusion about the concept,” Mr. Kohlmoos said.

‘Rigorous’ and ‘Appropriate’

Meanwhile, a pair of bills introduced in the Senate last month offer yet another view of what “scientifically based research” should look like. Sponsored by Sen. Richard G. Lugar, R-Ind., and Sen. Jeff Bingaman, D-N.M., the proposed Proven Programs for the Future of Education Act would grant schools that use “research-proven” programs in Reading First a competitive preference of 10 points. A companion bill, the Education Research and Development to Improve Achievement Act, would authorize up to $100 million for the development and evaluation of such programs.

Both bills say “research-proven” programs are those evaluated by at least two studies that use academically equivalent control and treatment groups, include sample sizes of five or more classes or 125-plus students, and use standardized assessments rather than developer-created measures, among other criteria.

“The hope is that some of what’s in these bills could get folded into NCLB,” said Robert E. Slavin, a co-founder of the Success For All Foundation, whose allegations that the Reading First program was being mismanaged launched the inspector general’s inquiries. Mr. Slavin helped write the proposed definition in the Senate bills.

In the end, though, said Mr. Kohlmoos, lawmakers need to be consistent within NCLB and across other education laws.

“You should have clarity and consistency or the effort to promote rigor goes nowhere,” he said.

A version of this article appeared in the October 17, 2007 edition of Education Week as ‘Scientific’ Label in Law Stirs Debate

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Professional Development Webinar
Inside PLCs: Proven Strategies from K-12 Leaders
Join an expert panel to explore strategies for building collaborative PLCs, overcoming common challenges, and using data effectively.
Content provided by Otus
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Science Webinar
Making Science Stick: The Engaging Power of Hands-On Learning
How can you make science class the highlight of your students’ day while
achieving learning outcomes? Find out in this session.
Content provided by LEGO Education
Teaching Profession Key Insights to Elevate and Inspire Today’s Teachers
Join this free half day virtual event to energize your teaching and cultivate a positive learning experience for students.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Federal Trump Shakeup Stops Most Work at Education Department's Civil Rights Office
President Donald Trump is downsizing a federal office that he's also using to carry out his policy agenda for schools.
9 min read
President Donald Trump listens as Elon Musk speaks in the Oval Office at the White House, Tuesday, Feb. 11, 2025, in Washington.
President Donald Trump listens as Elon Musk speaks in the Oval Office at the White House, Tuesday, Feb. 11, 2025, in Washington. The department's office for civil rights, which enforces federal civil rights laws in schools, has been hamstrung by the Trump administration's goal of shrinking the agency.
Alex Brandon/AP
Federal Trump’s 4th Week: Musk’s Team Pushes Ed. Dept. Cuts as McMahon Faces Senators
Linda McMahon appeared before U.S. senators, answering for an already turbulent time at the Education Department before she's taken charge.
6 min read
A shouting protester is removed from the hearing room as Linda McMahon, President Donald Trump’s nominee to be Secretary of Education, testifies during her Senate Health, Education, Labor, and Pensions Committee confirmation hearing, at the U.S. Capitol, in Washington, on Feb. 13, 2025. A shouting protester is removed from the hearing room as Linda McMahon, President Donald Trump’s nominee to be Secretary of Education, testifies during her Senate Health, Education, Labor, and Pensions Committee confirmation hearing, at the U.S. Capitol, in Washington, D.C., on Thursday, February 13, 2025. (Graeme Sloan for Education Week)
A shouting protester is removed from the hearing room as Linda McMahon, President Donald Trump’s nominee to be Secretary of Education, testifies during her confirmation hearing before the Senate Health, Education, Labor, and Pensions Committee at the U.S. Capitol in Washington on Feb. 13, 2025.
Graeme Sloan for Education Week
Federal Trump Picks Long-Serving State Chief With Bipartisan Fans for Top Ed. Dept. Role
Trump nominated North Dakota State Superintendent Kirsten Baesler to a key post overseeing K-12 policy at the U.S. Department of Education.
5 min read
North Dakota Superintendent of Public Instruction Kirsten Baesler announces the gathering of a task force to look into future options the state has for the assessment of students during a press conference May 8, 2015, at the state Capitol in Bismarck, N.D.
North Dakota Superintendent of Public Instruction Kirsten Baesler announces the gathering of a task force to look into future options the state has for the assessment of students during a press conference May 8, 2015, at the state Capitol in Bismarck, N.D. President Donald Trump has tapped Baesler to serve as assistant secretary of elementary and secondary education.
Mike McCleary/The Bismarck Tribune via AP
Federal Opinion Education Research Is in the Trump Administration's Cross Hairs
DOGE took a "chainsaw" to the Ed. Dept's grant programs. Morgan Polikoff has four suggestions for his research colleagues.
5 min read
Conceptual illustration of classroom conversations and fragmented education elements coming together to form a cohesive picture of a book of classroom knowledge.
Sonia Pulido for Education Week