What Happens When States Un-Standardize Tests?

Few educators are fans of fill-in-the-bubble standardized tests that don’t yield results until after students leave the classroom.

But when states had a chance to try out richer forms of assessment under a new pilot program established by the Every Student Succeeds Act, all but two demurred, in part because the pilot comes with tough technical requirements and no extra federal funding.

That doesn’t mean, though, that states are planning to stick with the Scantron sheets over the long haul.

In fact, several are looking to experiment with new performance assessments that ask students to complete some sort of hands-on task to show what they know or are able to do. Others are moving to use shorter tests throughout the year, instead of one big exam at the end, or redesigning tests to better reflect material students see in the classroom.

“We’re seeing a lot of momentum in states across the country around this idea of creating assessments that address student learning in more meaningful ways,” said Lillian Pace, the senior director of national policy at KnowledgeWorks, which supports state and federal policymakers interested in personalized-learning systems. “Nobody right now has the answer, but there is a lot of energy to try to figure out what we could do better.”

Equity Implications

Some states, like Georgia, are planning to apply to the Innovative Assessment pilot to support this work, now that the U.S. Department of Education has opened a new window for applications. Three other states—Hawaii, Kansas, and South Carolina—have also formally signaled interest in the pilot.

But others—including Kentucky—are keeping their plans separate from the federal flexibility, at least for now.

One big knock on new forms of assessments is that they aren’t helpful for equity. It can be hard to compare the results from one district to the next or measure the performance of particular populations of students, like English-language learners and students in special education.

Natasha Ushomirsky, the director of P-12 policy development at the Education Trust, a research and advocacy organization in Washington, is uneasy about the potential downside of the changes for vulnerable children. These students are often given lower-level classroom work than their more-advantaged peers.

“Statewide annual assessments are a check on those differences in expectation,” Ushomirsky said. “They give parents an objective picture of where their students are in relation to grade-level standards, with the knowledge that that bar is the same across all students, across all schools within the state.”

States that want to shift away from that model need to think carefully about how they will continue that comparability and objectivity, she said.

But fans of the new approaches—including some top state officials—say that over the long haul they might actually end up providing a better gauge of whether vulnerable students are learning, and where their trouble spots are.

“The same kids who have been behind for the last 30 years are still behind,” said Scott Marion, the executive director of the Center for Assessment, a nonprofit that works with states on testing. “The big advantage of using these richer types of assessments is that it will hopefully reveal to people what do these kids need to do next.”

Lindsay Jones, a vice president and the chief policy and advocacy officer at the National Center for Learning Disabilities, is proceeding carefully.

The current system has helped shine a light on the performance of students in special education, she said.

“We are not willing to reverse the progress that has been made. So we’re cautious about any new approach,” she said. At the same, “I don’t think the assessments, the current assessments, do a good job of really showing what students with disabilities, or frankly any other student, really know.”

Classroom Exposure

Some states that are moving forward on new testing regimes say they place equity concerns at the center of their approach.

Louisiana is just one of two states currently part of the federal pilot. (New Hampshire, which is working on a system of performance assessments, is the other).

The Pelican State is planning to combine literacy and social studies tests, using passages from books that students have already been exposed to in class, as opposed to something that’s brand new and just for the test. Students will take the tests a few times a year, instead of as a traditional end-of-the-year exam.

The new regime could end up being a better gauge of what vulnerable children know, said John White, the state commissioner of education.

Right now, standardized tests such as the ACT college assessment strive to be accessible to students from all backgrounds because they typically measure skills like reading comprehension using passages from books no student has seen before, he said.

But those tests may be more biased than educators think, White added. For instance, if a supposedly “content-agnostic” test includes a passage on France, a student who has been to that country will likely do better than ones who have no prior knowledge, even if their reading skills are at the same level, White said.

“It may be that actually pretending that you can have a knowledge-agnostic assessment is the less-equitable thing to do, and instead what is more equitable is actually to be transparent as to what content will be on the assessment and to give everybody an equal opportunity to access that knowledge, prior to the assessment,” he said.

Measuring ‘Soft’ Skills

Can tests measure tough-to-quantify skills like collaboration alongside math and science? Kentucky officials are hoping so.

The state is in the early stages of getting started on what David Cook, the state’s director of innovation, hopes will eventually be a widely used, competency-based learning system, complete with assessments that measure so-called “softer” skills—like communication—alongside the traditional reading, math, and social studies.

The state is starting small. Two districts have agreed to try out to find ways to gauge a student’s ability to be a critical thinker, empowered learner, collaborator, engaged citizen, and communicator. Eventually, academic subjects will be incorporated into the mix, and more districts will be part of the project.

The districts want to move to “something other than just a paper and standardized bubble-in test, they want something more than that,” Cook said.

For now, at least, those districts will have to test students using both the performance tasks and the traditional state test, because of federal and state accountability rules.

But they are OK with that, Cook said.

“They’ve committed to say we’re willing to serve two masters for as long as it takes,” Cook said.

The parallel testing systems might not go on forever. Kentucky may decide to apply for the federal pilot if the state believes that every district would eventually be ready to embrace competency-based learning and performance assessment, Cook said.

Cook thinks that, eventually, the performance-based assessments will capture what vulnerable groups of students know and are able to do better than traditional tests, which expect every student to be in the same place, on the same day.

A competency-based system would allow students to master skills on their own timetable, within reason, he said.

“I get that there’s a lot of concern about performance tasks not being equitable, but if you think about them right they can be more equitable. Way too many kids who don’t do well in the current environment,” Cook said.

Formative Assessments

Georgia is going a different route to make over its testing system: formative assessments.

The Peach State ran its own miniature innovative assessment competition, and ended up giving three consortia of districts the green light to work on assessments that help teachers gauge where students are and adjust instruction. The formative assessments would be rolled into a “summative” score for the year.

The overall goal is to give teachers a better picture of how their students are doing in real time. The state is hoping these locally created assessments will be a better fit for districts than the state test, called Georgia Milestones.

Eventually, state officials are hoping that all Georgia districts will have a choice of any of the three formative assessment systems.

Matt Jones, the chief of staff for the Georgia Department of Education, said he thinks the more frequent tests will help better capture what vulnerable students know.

“The more instances you have to measure a learner, the bigger picture you’re going to have of learning. The formative piece is going to paint a fuller picture,” he said.

Meanwhile, Michigan is just beginning to contemplate new kinds of tests that would be tied into the “competency-based” instruction that seven of its districts have been experimenting with for the past couple of years, said Andrew J. Middlestead, the state’s director of the office of educational assessment and accountability.

The state may apply for the innovative assessment pilot down the road—but not in time for the December 2018 deadline set by the federal Education Department for the next round.

If it applies for the pilot, the state will need to carefully think through how its plans for a new kind of test will impact vulnerable groups of students, Middlestead said. The state wants to make sure that it can create “something that is approachable for all kids,” before moving forward, he explained.

Districts in Virginia are working on “student led” assessments, which ask children to demonstrate their learning and knowledge in a meaningful way and to reflect on their own performance.

And some districts in Colorado are developing portfolios, capstone presentations, and other outside-the-box demonstrations to show students are ready for the workforce or postsecondary education—a graduation requirement for the class of 2021 and beyond. Districts can also use standardized tests, like Advanced Placement or the SAT, or industry certification to demonstrate that their students are prepared for what comes after high school.

Elliott Asp, a senior partner at the Colorado Education Initiative, and his colleagues are working with more than a dozen districts and schools that want to go the performance task or presentation route. In his mind, they can be just as rigorous as the nationally recognized tests.

“It’s not an easy way out for kids,” Asp said. “We’re trying to raise the bar for every kid through this process.”