Federal

‘Value Added’ Pioneer Says Stinging Critique of Method Is Off-Base

By Debra Viadero — May 05, 2008 3 min read
  • Save to favorites
  • Print

Includes updates and/or revisions.

The statistician who pioneered the use of “value added” research techniques for documenting students’ year-to-year academic progress is disputing a critique of his approach that was published recently in a prominent academic journal.

In the article, which appeared in the March issue of Educational Researcher, Audrey Amrein-Beardsley, an assistant professor at Arizona State University in Phoenix, takes aim at the Education Value-Added Assessment System, or EVAAS. The EVAAS is an offshoot of the Tennessee Value-Added Assessment System developed by William L. Sanders, which is widely credited with helping to launch nationwide interest in value-added approaches for measuring students’ academic progress.

Ms. Amrein-Beardsley contends that the EVAAS is opaque, is possibly flawed, and has not been subjected to enough external review to merit its widespread use in school systems.

“No one is looking inside the model to see if the assumptions being made within it are true,” Ms. Amrein-Beardsley said in an interview. “The bottom line is to open this up for debate and not have these things be adopted blindly.”

‘Factual Errors’ Alleged

See Also

A recently posted response to Audrey Amrein-Beardsley’s critique is now available online. The response is written by William L. Sanders and S. Paul Wright.

But Mr. Sanders, for his part, contends that Ms. Amrein-Beardsley’s report is “rife with factual errors,” and he points to three studies in which researchers have duplicated and tested his model. Among the most recent was a 2007 study, published in the peer-reviewed Electronic Journal of Statistics, in which a team of researchers from the Santa Monica, Calif.-based RAND Corp. compared the Sanders method with other value-added techniques.

Now working in private industry as a senior manager for value-added assessment and research at the Cary, N.C.-based SAS Institute, Mr. Sanders has never shared the computational algorithms for the EVAAS. But he and a colleague described the model in a 1997 paper, which the RAND researchers and others have used to create their own software mimicking Mr. Sanders’ model.

Ms. Amrein-Beardsley also raises technical concerns with the EVAAS, some of which echo cautions that other experts are voicing about value-added modeling in general. (“Scrutiny Heightens for ‘Value Added’ Research Methods,” May 7, 2008.)

She notes, for instance, that the EVAAS does not take into account demographic and socioeconomic differences between students.

“Family income, ethnicity, ability, and other background variables unquestionably affect levels of student achievement and the progress that students make from year to year,” she writes.

Mr. Sanders, though, has long maintained that such statistical adjustments are not needed because the EVAAS compares students’ year-to-year academic growth with their own progress from previous years. “A kid is coming from the same home this year as he was the year before,” he said.

To illustrate her case, Ms. Amrein-Beardsley reanalyzed data from the appendix of a controversial 2005 study of North Carolina teachers that Mr. Sanders was recruited to do for the National Board for Professional Teaching Standards, an Arlington, Va.-based group that awards national credentials to teachers who meet the requirements of a process based on documentation of their performance.

While Mr. Sanders concluded that board-certified teachers did no better on average than those who had failed to earn certification and those who had never tried, Ms. Amrein-Beardsley said the data showed a positive student-achievement effect. She said that, on average, students of board-certified teachers outperformed those of teachers with only regular certification by three-quarters of a month’s worth of growth in mathematics and one week in reading.

Mr. Sanders said his conclusions stemmed in part from another finding in his study: The variation in student-achievement levels for teachers within the NBPTS-certified group was greater than it was for teachers across different categories.

Joseph A. Aguerrebere Jr., the board’s chief executive officer, said most studies of board-certified teachers, such as the one by Ms. Amrein-Beardsley, also find that students of those teachers outperform their peers.

Coverage of education research is sponsored in part by a grant from the Spencer Foundation.

Events

School & District Management Webinar Crafting Outcomes-Based Contracts That Work for Everyone
Discover the power of outcomes-based contracts and how they can drive student achievement.
School & District Management Webinar EdMarketer Quick Hit: What’s Trending among K-12 Leaders?
What issues are keeping K-12 leaders up at night? Join us for EdMarketer Quick Hit: What’s Trending among K-12 Leaders?
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Artificial Intelligence Webinar
Teaching Students to Use Artificial Intelligence Ethically
Ready to embrace AI in your classroom? Join our master class to learn how to use AI as a tool for learning, not a replacement.
Content provided by Solution Tree

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Federal A Bill to Kill the Education Department Is Already Filed. Here's What It Says
The bill represents another attempt at a long-term Republican goal.
6 min read
People walk outside the U.S Capitol building in Washington, June 9, 2022.
People walk outside the U.S Capitol building in Washington, June 9, 2022. Legislation has been introduced in the Senate to abolish the Department of Education.
Patrick Semansky/AP
Federal The K-12 World Reacts to Linda McMahon, Trump's Choice for Education Secretary
Some question her lack of experience in education, while supporters say her business background is a major asset.
7 min read
Linda McMahon, former Administrator of Small Business Administration, speaks during the Republican National Convention on July 18, 2024, in Milwaukee.
Linda McMahon speaks during the Republican National Convention on July 18, 2024, in Milwaukee. McMahon has been selected by President-elect Trump to serve as as the next secretary of education.
J. Scott Applewhite/AP
Federal What a National School Choice Program Under President Trump Might Look Like
School choice advocates—and detractors—see a second Trump term as the biggest opportunity in decades for choice at the federal level.
8 min read
President Donald Trump listens during a "National Dialogue on Safely Reopening America's Schools," event in the East Room of the White House, on July 7, 2020, in Washington.
President Donald Trump listens during a "National Dialogue on Safely Reopening America's Schools," event in the East Room of the White House on July 7, 2020, in Washington. He returns to power with more momentum than ever behind policies that allow public dollars to pay for private school education.
Alex Brandon/AP
Federal 5 Things to Know About Linda McMahon, Trump's Pick for Education Secretary
President-elect Donald Trump’s selection, the former CEO of World Wrestling Entertainment has long spoken favorably about school choice.
7 min read
Small Business Administrator Linda McMahon speaks during a briefing at the White House in Washington on Oct. 3, 2018.
Linda McMahon speaks during a briefing at the White House in Washington on Oct. 3, 2018, when she was serving as head of the Small Business Administration during President Trump's first administration. McMahon is now President-elect Trump's choice for U.S. secretary of education.
Susan Walsh/AP