States and districts that want to “test-drive” the next generation of assessments will have the chance, thanks to one of the most buzzed-about pieces of the Every Student Succeeds Act.

But, as the state officials who have crammed conference rooms to hear about ESSA’s “Innovative Assessment” pilot are learning, trying out a new testing system in a handful of districts isn’t for the faint of heart.

The law allows a small handful of states to use local tests in lieu of the state exam, as long as these districts are trying out a system that will eventually go statewide. And ESSA places a number of stringent conditions on the pilot, what the law’s architects like to call “guardrails.” These “guardrails” are aimed at making sure that the new kinds of tests states develop are of high quality, and that all kinds of students—including English-language learners and students in special education—have access to them.

Those conditions can challenge everything from a state’s technical know-how to its approach to professional development—all without additional federal funding, at least for now.

Still, experts say, the pilot has the potential of helping to point the way to brand-new methods of measuring student learning, both for the states and districts that decide to give it a shot, and those that don’t.

“We see this as a bridge,” said Lillian Pace, the senior director of national policy at KnowledgeWorks, a non-profit that works to better “personalize” learning for students. Pace has studied the pilot and its implications closely.

When lawmakers sat down to write ESSA, the field wasn’t ready, she said, to “paint the picture of the next generation of assessments"—the kinds of tests that can give teachers a fuller, real-time picture of what their students know while still helping states and district improve schools.

“While states may not be successful on the first try, we expect this to be a learning process for all,” Pace said. “The goal is for participating states to build and refine a system over time that really transforms student learning.”

What’s more, the pilot also presents challenges for the U.S. Department of Education, which will have to strike a balance between ensuring that the necessary quality and equity measures are in place, while giving states and districts room to experiment.

Where did this idea come from? It’s included in ESSA. Check out pages 84 through 92 of the law if you want to read about it first-hand.

But the pilot was inspired in large part by New Hampshire, which got permission under the No Child Left Behind Act to begin experimenting with locally-developed performance assessments, in a handful of districts, with the idea of eventually taking the system statewide. (Andrew actually visited the Granite State last year and saw the pilot up close and personal, read about it here.)

What kinds of tests are we talking about? There’s a wide range. States and districts can try out performance tasks (like New Hampshire is doing), competency-based tests, and more.

How many states can participate in the pilot? At first, at least, the pilot participation is limited to seven states, either working alone, or as a consortium. (In this case, a consortium can only have up to four members)

You said “at first.” Does that mean more states can jump on later? Maybe. The first three years of the program are supposed to represent a demonstration period, allowing the federal government, states, and districts to work through the challenges of setting up and monitoring these new testing systems. After that, the Institute for Education Sciences, the research arm of the Education Department, will write a report taking stock of how everything went.

Once that report is out, the education secretary—whoever it is by that point—can allow more states, even all states, to apply join the pilot. But importantly, that’s not a must. The question of whether to expand, or not expand, the pilot, will be up to whoever is in control of the department at the time.

How long do states have to get these new systems up and running? States that are selected to participate in the initial pilot get five years to develop their systems, show that the tests are comparable to the state assessment, and bring the whole system statewide.

Some experts consider that timeline very ambitious, particularly for a state that’s just getting started when it comes to a complex area like competency-based education. So states can ask for a two-year extension.

What about those guardrails you mentioned? States that want to participate in the pilot have to make sure that:

• These tests are valid and reliable, for all students and for particular subgroups of students, including English-language learners and students in special education, two populations that aren’t always easy to test.
• The results of these tests are “comparable” across districts, so that a particular score or outcome means the same thing from one district to the next. New Hampshire does this by having students take the state assessment in certain grades.
• By the end of the demonstration period, the districts include a representative sample of students from around the state. For instance, a state with a large ELL population (like, say, California) would need to ensure that there were plenty of ELLs taking the tests in the trial districts.
• These systems have to be able to be scaled statewide, by the end of the pilot period.

That last requirement might be the toughest part, said Scott Marion, the executive director at the Center for Assessment, an non-profit organization, which is working with New Hampshire on its performance tasks.

“The thing that scares ... everybody is scaling statewide,” he said. “They want you to have these guardrails, so it’s not like any district could just waltz in” to the pilot. “You have to earn your way in.”

But Donna Harris-Aikens, the ESSA implementation lead at the National Education Association, thinks those challenges are surmountable.

“I think it will be challenging to scale up statewide,” she said. “But I don’t think it will be impossible. There are states that are looking for a different way to demonstrate [student] progress. We are hoping and encouraging people to spend some time trying to figure out if they can do this.”

Is there additional federal money for this? Probably not. In its fiscal year 2017 budget request, which will fund the first year of ESSA implementation, the Education Department asked for an extra $25 million in new competitive funding to help states improve their testing systems. That money could, theoretically, help the pilot states and districts. But Congress is unlikely to pony up the increase.

States looking for funding to try out these systems might be better off working with the foundation community, which has shown interest in the work in New Hampshire and elsewhere.

How many states are interested? Good question! Even experts who work with states aren’t sure at this point. New Hampshire has gotten permission to keep going on its work, since it is in sort of a no-man’s land between its waiver from provisions of the NCLB law and the pilot. And Colorado has actually passed legislation saying that it must apply—but the state chief, Richard Crandall, recently stepped down.

A big group of states, including Delaware, Kentucky, Maryland, New Hampshire, Ohio, Tennessee, and Vermont, along with the District of Columbia and more, participated in a webinar that the Council of Chief State School Officers held last month to talk about the pilot’s potential. But that doesn’t mean all those states will apply—folks may just have wanted to learn more about this program.

Do states have to participate in the pilot in order to get flexibility on testing? Not necessarily. The CCSSO is advising states to take a look at the testing flexibility already in place in ESSA, to see if they can accomplish what they want on assessments without having to join the pilot. (CCSSO even made a “decision tree” to help states sort through their options, which you can check out here.)

Under the new law, states can try out computer-adaptive assessments or use a series of short-term or interim assessments instead of one big summative test—all without having to participate in the pilot.

What’s the department’s timeline for regulations? They’re working on it. The department is expected to release draft regulations governing the pilot sometime this summer. Those could touch on tricky issues, like how exactly states and districts show the tests are “comparable” statewide and how the timeline might work, including potentially giving states time to plan their systems before the pilot’s clock officially starts.

What’s the timeline for actually applying? It’s unclear for now just when states will be able to apply for the pilot, and, importantly, whether it will be the Obama administration or a president-and-secretary-to-be-named-later who will decide which states can participate.

What else has the department said about this so far? Just that officials are excited to get rolling on this. In fact, Education Secretary John B. King Jr., told the Education Commission of the States they are coming soon. “We are looking forward to releasing draft regulations related to assessments under the Every Student Succeeds Act soon,” said Takirra Winfield, a spokeswoman for the department in an email. “These regulations will help us the seize the new law’s opportunity to help states and districts restore the balance on testing, develop innovative assessment strategies, and reclaim the promise of a well-rounded education for every child while maintaining critical information for parents and educators.”

And in speaking at Education Week‘s Leaders to Learn From event earlier this year, King expressed excitement about New Hampshire’s approach, while making it clear he doesn’t think that what works for one state will work for everyone.