Educators Evaluate Array of Formative Testing Products

When he was teaching kindergarten, Shawn C. Rubin used a clipboard and green, yellow, and red crayons to track students’ progress on paper. Adjusting his teaching for each student’s ability level had the greatest possibility for advancing learning, he said, but it was also the most difficult assessment approach to do.

Now working to advance blended learning—a mix of face-to-face and online education—in all Rhode Island school districts and charter schools, he sees the magnitude of the formative-assessment challenge multiplied exponentially. Elementary teachers need to evaluate “30 students a day in six subject areas with two competencies in each subject, and high school teachers might be tracking 200 students in one subject,” said Mr. Rubin, the director of blended learning for the nonprofit Highlander Institute, a Providence, R.I.-based community of educators and professionals, and the CEO and co-founder of Metryx, a Providence-based company that has designed an ed-tech tool that allows teachers to track and analyze student mastery without crayons.

Metryx is just one of dozens of ed-tech products on the market today competing to help teachers and students with some aspect of the formative-assessment process, in which educators gauge learners’ understanding to pinpoint precisely what each has mastered, and adjusts teaching accordingly. The competition is heating up, in large part because formative assessments’ on-the-spot feedback about student mastery lies at the heart of scaling personalized learning.

But truly effective formative assessments need to go beyond simply evaluating students’ knowledge against standards and creating bar charts and graphs to show degrees of mastery, say some observers.

The most important question is: “How does this support the learning process?” said Margaret Heritage, the assistant director for professional development for the National Center for Research on Evaluation, Standards, & Student Testing, or CRESST, at the University of California, Los Angeles. She describes formative assessment as an embedded process in which teachers are collecting evidence while students are learning.

In evaluating the growing collection of formative-assessment products on the market, Ms. Heritage said educators should be asking whether those products give them the items or tools that are linked to their immediate lesson goals and that provide actionable information. “Most things I see out there don’t do that,” she said.

Market Demand Rising

Analyzing growth trends in the testing- and assessment-market segments, “we found a lot of excitement—and therefore, demand—around formative, adaptive assessments that support personalized learning,” said John Richards, the president of Consulting Services for Education and the author of “Behind the Data: PreK-12 Testing and Assessment Market,” a report that the Washington-based Software & Information Industry Association was finalizing at press time. Formative-assessment products are now capturing some of the most sophisticated and nuanced learning measurements, including assessing skills such as collaboration, problem-solving, and conceptual understanding, according to Mr. Richards.

Educational technology products that help conduct formative assessments are being packaged in three distinct ways, said Julia F. Freeland, a research fellow for the Clayton Christensen Institute, a San Mateo, Calif.-based think tank that studies trends in personalized learning. It is embedded in online curricula, such as ST Math, Lexia Learning, and DreamBox Learning; it is offered as a stand-alone assessment tool like MasteryConnect and its recent acquisition Socrative; and it is provided as part of adaptive-learning platforms such as Knewton, that aggregate data from numerous assessments and use those results to generate recommendations for individualized learning pathways.

For products that embed questions in online curricula, Ms. Freeland observes varying levels of trust among educators about the quality of those assessments.

In response to that concern, some ed-tech providers—such as MasteryConnect and Renaissance Learning with its STAR Custom—rely in whole or in part on teacher-submitted assessments that can give educators the more-customized feedback they want.

Increasingly, ed-tech companies are matching their content and questions to specific standards, including the Common Core State Standards, which can be problematic. “A single question could be tagged against one or multiple standards,” she said. If a student answers a multiple-standard question incorrectly, it can cause confusion for teachers. “How are they expected to unbundle that,” she asked.

Common Core Fuels Interest

The expected rollout this school year of the more complex common-core tests from the Partnership for Assessment of Readiness for College and Careers, or PARCC, and the Smarter Balanced Assessment Consortium brings with it an increasing interest in formative testing that goes beyond multiple choice and true/false responses. Some questions will require test takers to demonstrate the transfer of learning by asking them to show their work, or write a passage, for instance. To measure true transfer of learning, tasks must be “designed to provide students with opportunities to adapt or apply their knowledge in new or unique ways,” said Eric M. Carbaugh, an associate professor in the department of middle, secondary, and math education at James Madison University in Harrisonburg, Va. “You can’t assess true transfer at all with multiple-choice questions,” he said.

Mr. Carbaugh, who consults with schools and districts under the auspices of Alexandria, Va.-based ASCD Professional Learning Services, said many experts recommend a simple “3-2-1" exit-card approach to helping teachers use formative assessments to gauge student understanding: “Give me three big ideas you took away, two questions you have, and one way you think you can use it,” he said he tells teachers. That information can go on a blog, in a journal, or on an index card. “It’s a little messier, from a teacher’s standpoint, but we need to measure more than whether kids can get a right or wrong answer,” he said.

Smarter Balanced recently released its digital library, which features mathematics and English/language arts literacy resources to help teachers with the formative-assessment process that can take the form of assignments, classroom discussions, and other approaches. The consortium contracted with Amplify, based in New York City, to build the interactive online modules, which are “meant to demonstrate how the formative-assessment process is used to implement the intent of the Common Core State Standards,” said Chrys Mursky, Smarter Balanced’s director of professional learning.

Integrating Response Data

The Northwest Evaluation Association, which is based in Portland, Ore. and produces the Measures of Academic Progress interim assessments that are adapted to students’ instructional levels, also owns the Formative Assessment Item Bank it acquired from the Princeton, N.J.-based Educational Testing Service. The 81,000-item bank, which is operated independently of the evaluation association, is usually licensed to platforms that can store and build tests and scan and report on data. It is also licensed directly to some districts.

Some companies are also creating products and services to help educators use polling software and “clicker” technology to have students register responses to formative-assessment questions. Turning Technologies, a Youngstown, Ohio-based company, is one provider that integrates the data from its response systems with interactive whiteboards and on mobile devices. Poll Everywhere is an app that does the same.

Acuity, the McGraw-Hill Education CTB Web-based assessment platform, offers 10- to 20-minute tutorials for students with embedded assessments to check students’ understanding of the material they are trying to learn, and it incorporates an online community where teachers can share resources and tests they have created.

Other formative-testing developments are also worth noting.

For instance, CRESST’s Ms. Heritage said a new K-3 formative-assessment platform in North Carolina that is under development with a state university, using Race to the Top and state funding, looks very promising. At the click of a button, teachers can view a progression and learn about strategies for collecting evidence. Plus, teachers can capture audio and video evidence, and add notes, to document student learning, then review information that would help teachers interpret the evidence, she said.

For the Christensen Institute’s Ms. Freeland, the fast growth of blended learning and an emerging competency-based education system—similar to what is in place in New Hampshire—are harbingers of future formative-assessment opportunities for schools, and companies, if it is done well.

“As content becomes more widespread and commoditized,” she said, “it’s going to be all about online assessments to verify what students know.”