States Rush to Join Testing Consortia

Spurred by the promise of $350 million in Race to the Top money for improved tests—as well as an opportunity to strengthen bids for part of the federal fund’s larger $4 billion pot—states are scrambling to join consortia to develop common assessments.

Six state consortia are now engaged in discussions about common tests, and the multiple partnerships may put to rest for now speculation that federal support inevitably will lead to a single national set of exams.

Since it isn’t yet clear what the U.S. Department of Education expects to see from the consortia, most states have hedged their bets by signing up for more than one partnership. For the most part, their signatures on the “memorandums of understanding” listing the principles of each consortium are nonbinding.

The situation remains fluid, with the number of states involved in each consortium changing almost daily.

At this point, signing a memo “is not a meaningful gesture. ... Many states did it to get extra points in their Race to the Top applications,” said Stanley N. Rabinowitz, a senior program director of assessment and accountability services at WestEd, a San Francisco-based technical-assistance provider.

Banding Together—Again and Again

Nearly all states have joined multiple consortia that are crafting common assessments to qualify for Race to the Top money.

Source: Education Week; Center for Assessment

“That shouldn’t be interpreted to mean that states aren’t serious,” he said. “It’s a courting process, a development process, and states are feeling each other out.”

In fact, there are signs that several of the consortia might try to merge even before March, when the Education Department is expected to release its application notice for the $350 million in assessment funding, which is being provided under the American Recovery and Reinvestment Act.

To that end, the Maine commissioner of education, Susan Gendron, who is also the board president of the Council of Chief State School Officers, last week sent a letter to state chiefs to arrange a meeting of all the state consortia. The nation’s governors also were scheduled to discuss common assessments last week, she said in an interview.

Still, observers warn that even if more clarity on the format of such consortia is reached, many obstacles stand in the way of their making headway. They include potentially conflicting goals, delicate political questions about curricula, and widely divergent procurement laws.

There’s even a more basic calculus at work: whether the largest consortia will be able to work effectively, or prove unwieldy and inefficient.

Connecting the Dots

Four states have yet to join any of the consortia. Two of them, Alaska and Texas, are also the lone holdouts from a state-led project to devise common English/language arts and mathematics standards. A third state, Virginia, “remains committed” to its own Standards of Learning testing program, according to Charles Pyle, a spokesman for the Virginia education department. Nevada also has not joined a consortium.

Observers agreed that, with few details available yet on the format of the competition, it’s reasonable for states to try to shape matters by joining several collaboratives, although most of the consortia are being led by states that already have some experience with the principles laid out in their respective memorandums of understanding.

One consortium, called SMARTER, will focus on adaptive-testing technology. Computer-adaptive tests adjust their difficulty depending on whether a student has correctly answered previous questions. Proponents say that such tests home in more precisely on a student’s mastery of standards, though the technology has its detractors, too.

The consortium is being led by three states, including Oregon, the only one using an adaptive-test model to meet the annual assessment requirements of the federal No Child Left Behind Act.

Another consortium, nicknamed MOSAIC, will focus on professional development for classroom-based formative-assessment techniques, which help teachers adjust instruction in real time, said Pat Roschewski, the director of statewide assessment for Nebraska.

The state, which is leading the endeavor, has trained teachers on such assessments for a decade, though for NCLB accountability purposes it is phasing out the local tests in favor of statewide ones.

The Balanced Assessment Consortium, which Ms. Gendron leads with Steven L. Paine, the West Virginia superintendent of education, seeks a system that would include curriculum-embedded, performance-based tasks scored by teachers throughout the year. Such tasks would help guide instruction and would be coupled with year-end exams.

“One of the things that’s critically important is how we change practice in the classroom that enhances teacher effectiveness,” Ms. Gendron said. “We know from experience that assessments by themselves do not change practice.”

A Florida-led consortium has as one of its goals common year-end assessments with a shared proficiency definition and grade-by-grade benchmarks toward college- and career readiness.

The National Center on Education and the Economy, a nonprofit that works to improve linkages between education and the workforce, will focus on high school exams. The idea, said the group’s president, Marc S. Tucker, is to align curriculum, instruction, and testing in the early high school grades with comprehensive, syllabus-based “board examinations,” such as those used in Britain.

Rounding out the consortia is one organized by Achieve, one of the groups leading the Common Core State Standards Initiative. Its organizers said it will work to keep the other consortia focused on crafting exams that produce comparable results across states.

Procurement Headaches

In the Assessment Game

Experts and policymakers express their views on how the consortia will shake out over the coming months.

LARRY BERGER
Chief Executive Officer, Wireless Generation

“The fact that [the federal assessment competition] is going to be a secret, with only tea leaves for us to read until it hits the street, is creating confusion, but it is productive confusion. … States are joining multiple partnerships in part because it’s a good principle when we don’t know what we’re buying.”

SUSAN GENDRON
Board President, Council of Chief State School Officers

“We absolutely think that [merging the consortia] is the right thing to do. I’ll be honest, there might be one or two that say, ‘We still want to go alone.’ And then the states will have to decide how they want to proceed.”

STANLEY N. RABINOWITZ
Senior Program Director, WestEd

“All of [the consortia] are still nowhere near to being a final product or a final understanding. Over the next month or so, some will survive and some won’t, based on how well they evolve to meet the states’ broad interests.”

ERIC J. SMITH
Education Comissioner, Florida

“I think you’re going to see in coming weeks and months a lot of modifications. ... My sense is there’s probably more commonality between the consortia than differences.”

Eric J.Smith photo by Ray Stanyard

Ms. Gendron hopes that convening state officials will provide an avenue for some of the consortia to merge or strike a governance arrangement to share their work.

For the most part, leaders of the consortia say they have found much commonality among the six.

“Our greatest need now is continued conversation around the various options on the table,” said Eric J. Smith, Florida’s education commissioner. “It’s healthy and smart. We’re going to find that most states have a common interest in how this gets fleshed out.”

Even so, many tensions are still lurking below the surface.

One is the fundamental difference between traditional year-end accountability tests, which tend to be curriculum-neutral, and those that would depend on projects aligned closely with curricular units.

To draw comparisons across states using the latter type, discrete curricular units would probably need to be taught across multiple districts or states, said Larry Berger, the chief executive officer of Wireless Generation, a New York City educational technology company.

“No one is advocating a national curriculum or even one across a state, but ... that’s still politically complicated territory,” he said.

Experts also have questions about the procurement processes for developing and executing testing specifications. Generally, procurement laws prevent a technical expert advising the design process from bidding on actual test development.

Laws and legal precedents differ in their stringency, but a consortium might be bound by the most restrictive state’s policy unless some other model was reached, warned Mr. Berger, who testified on the issue at one of the Education Department’s public hearings on the competition.

Mr. Berger, a board member for Editorial Projects in Education, the publisher of Education Week, believes that the consortia should begin bidding competitively on technical partners now, or risk preclusion from using such experts later.

Too Many Cooks?

Other state experts say they are aware of such challenges and are considering new avenues for dealing with procurement, but the details, for now, are sketchy at best.

And in what comes as a caution against merging the consortia, some observers suggest that smaller partnerships would be able to operate more nimbly and effectively than those with dozens of members.

The New England Common Assessment Program consortium, which crafted common NCLB tests, worked, observers say, because the states involved—Maine, New Hampshire, Rhode Island, and Vermont—had similar testing issues, and each had an equal say in decisionmaking.

“That was four states. With 36 states, it’s a different order of magnitude keeping everyone on the same page,” said Lauress Wise, a scientist at the Human Resources Research Organization, an Alexandria, Va.-based analysis group.

The World-Class Instructional Design and Assessment consortium, which created a common English-language-proficiency test to meet NCLB requirements, worked despite its many players, because the participating states sought a fixed product, not a flexible model, added WestEd’s Mr. Rabinowitz. But given that federal officials have indicated that they want both innovation and commonality, consensus on a common assessment system could prove more challenging to reach, he said.

He hopes the federal competition will reserve money for smaller consortia—or subsets of the larger ones—to work on “niche” issues.

On at least one area, participants generally agreed: If the government chooses to finance more than one consortium, it should allow for movement between them.

“My personal hope would be that they fund a couple of different approaches and then do some formal evaluation of the effectiveness of these approaches for meeting the objectives, and let states add on to the consortia that prove to be effective,” said Mr. Wise.

“I think the [competition notice] will give more specifics on the broad questions,” added Mr. Rabinowitz, “but I’m not sure it’s going to solve fully the real implementation issues, which will drive a lot of the consortia discussions and the ability to be successful.”

Assistant Editor Catherine Gewertz contributed to this report.