For an updated version of this story, see “Review of Math Programs Comes Under Fire.”
The first round of a Consumer Reports-style review for instructional materials paints a dismal picture of the textbook-publishing industry’s response to new standards: Seventeen of 20 math series reviewed were judged as failing to live up to claims that they are aligned to the common core.
The reviews, released online Wednesday by the nonprofit EdReports.org, were conducted by small groups of teachers and instructional leaders from across the country. They looked at digital and print K-8 mathematics materials from widely used publishers—including Pearson, McGraw-Hill, and Houghton Mifflin Harcourt—as well as from some lesser-known providers whose texts passed state review processes. The results echo previous alignment studies conducted by university-based researchers.
“In general, the results are pretty bad for all the publishers,” said Morgan Polikoff, an assistant professor of education at the University of Southern California, in Los Angeles, who studies common-standards alignment but was not involved in the EdReports.org project. “I think people really will pay attention to this, and I think it will affect [curriculum] adoption processes going forward.”
In all, just one curriculum series stood out from the pack. Eureka Math, published by Great Minds, a small Washington-based nonprofit organization, was found to be aligned to the Common Core State Standards at all grade levels reviewed.
A McGraw-Hill series, My Math, was deemed aligned to the standards in two of the five grade levels evaluated. All other series reviewed were found to only “partially” or not at all meet the criteria for common-core alignment.
EdReports.org was spearheaded by Maria M. Klawe, the president of Harvey Mudd College, in Claremont, Calif., and was launched in August. Its nine-member board includes representatives from the National Board for Professional Teaching Standards, the National Council of La Raza, and the Education Trust.
The project is funded primarily by $3 million in grants from the Seattle-based Bill & Melinda Gates Foundation—which also was a major financial backer of the development of the common core—the William and Flora Hewlett Foundation of Menlo Park, Calif., and the Leona M. and Harry B. Helmsley Charitable Trust in New York City. (The Gates and Hewlett foundations also support, respectively, news coverage of college- and career-ready standards and deeper learning in Education Week.)
EdReports.org plans to eventually move on to secondary math and K-12 English/language arts curricula.
Collecting Evidence
The 46 reviewers, half of whom are practicing teachers, worked in teams of four over several months to review the K-5 or 6-8 instructional series. Team members combed through texts independently and then met in a weekly videoconference to discuss their findings.
“There were weeks we could not actually agree, and we would table that discussion until we could find more evidence,” said Kimberly Osbourne, an assistant principal in Murfreesboro, Tenn., who was on a review team. “We worked hard to come to an agreement on the evidence we all collected.”
The curricula were first evaluated on whether they meet the common core’s expectations for focus and coherence—that is, whether they stick to grade-level content and follow a logical sequence for math learning. If a text passed that first threshold, or “gateway”—and a majority did not—the reviewers then moved along to gateway two, which looked at whether the curriculum meets the expectations for rigor. The third and final gateway measured usability.
Jay A. Diskey, the executive director of the Association of American Publishers’ P-12 learning group, based in Washington, said the use of gateways was a major concern. “The review process was halted if there were minor instances of perceived lack of alignment, resulting in a very shallow, incomplete review,” he wrote in an email.
The EdReports.org website uses a three-tiered rating system—meets criteria, partially meets criteria, or does not meet criteria—for each gateway. The site also has more-detailed reports for each textbook, which include documentation on how the reviewers reached each score. Users can search by series title, publisher, or grade level, and can compare curricula with one another.
Eureka Math far surpassed all other curricula evaluated. It was found to be aligned for all grades, K-8, and passed through all three gateways. On usability, Eureka’s offerings for grades K-5 meet the criteria, while its materials for grades 6-8 only partially meet the criteria.
Eureka Math differs from other texts in that, rather than being an update to existing material, it was designed specifically for the common core.
“None of our work in any subject predates the Common Core State Standards,” said Lynne Munson, the president and executive director of Great Minds.
Among the highlights of the reviews:
• My Math, a K-5 instructional series by McGraw-Hill, was deemed fully aligned for grades 4 and 5. The kindergarten text in that series was found not aligned, and the 1st and 2nd grade texts partially meet criteria for alignment.
• Seven Houghton Mifflin Harcourt instructional series were reviewed. Of those, four partially meet the alignment criteria for at least one grade level.
• One of the four Pearson texts reviewed were deemed partially aligned for at least one grade level.
• All texts by Agile Mind, Big Ideas Learning, Edgenuity, Kendall Hunt, and TPS Publishing Inc., were deemed not aligned to the common standards.
More than 40 states have adopted the common core, which covers English/language arts and math. The standards, released in 2010, were the result of an initiative led by the Council of Chief State School Officers and the National Governors Association.
Early Reactions
Publishers were given two weeks to respond to their reviews before the EdReports.org website went public. Responses of up to 1,500 words are published along with the reviews.
In its response to the review of the K-5 series Investigations in Number, Data, & Space, Pearson wrote that the reviewers “applied a very narrow standard for measuring focus that we believe compromises the intent of the [Common Core State Standards for math] with regard to instructional materials.”
Houghton Mifflin Harcourt wrote in its response for Math Expressions, which was found to partially meet criteria for K-2 and not meet criteria for grades 3-5, that the program “offers many resources that were not included in the review process, including digital and assessment materials that reflect student conceptual understanding, fluency checks, and performance tasks.”
In a webinar for reporters, Eric Hirsch, the executive director of EdReports.org, said “the goal was to review student and teacher editions—not to review an entire instructional program.”
Four of the nine publishers reviewed responded to Education Week’s requests for comments by deadline.
Ms. Munson of Great Minds said she was “thrilled” with the review by EdReports.org. “We consider all our materials to be living documents that are under continual improvement. ... We’re extremely keen to have their advice and input and will certainly be acting on it.”
Edgenuity pointed to concerns with the EdReports.org methodology. “The fundamental flaw in the EdReports process is their gateway approach, which unfortunately will mislead readers,” John Hartz, a spokesman for the Scottsdale, Ariz.-based Edgenuity, wrote in an email. “The limitations of the EdReports approach stand in contrast with many state and district adoption processes that have allowed reviewers to analyze the full scope of Edgenuity’s content—including in states such as California, Georgia, West Virginia, and Utah where Edgenuity’s courses were approved.”
TPS Publishing Inc. and Houghton Mifflin Harcourt also said the results are in opposition to state and district adoptions.
Regarding the conflicting reviews, Mr. Diskey of AAP said, “it has to be asked whether some of the differences here have to do with the type of rubric and alignment tool that was used” by EdReports.org.
EdReports.org is not the only group looking at common-core alignment. Learning List, a for-profit company based in Austin, Texas, analyzes digital and print educational resources for alignment with the standards. In a press release issued in advance of the EdReports.org release, Learning List clarified what it sees as key distinctions between the groups, including that Learning List is a paid subscription service, that it reviews for alignment to a variety of standards rather than just common core, and that subscribers and publishers can request certain materials be reviewed for Learning List.
University-based researchers William Schmidt, the co-director of the education policy center at Michigan State University, in East Lansing, and Mr. Polikoff of USC have conducted alignment studies as well. Both have come to the same conclusion as EdReports.org: Claims of common-core alignment are generally unfounded.
After his own analysis was released, “I can’t tell you how many emails I got from folks in school districts all over the country who are just desperate for that kind of info,” said Mr. Polikoff. “I do want a vibrant, competitive marketplace for materials to drive that competition and innovation in a way we really couldn’t before.”
