Standardized testing has been a staple of American education for more than a half century, and has become particularly prominent in public schools—some might say invasive—since the passage of the No Child Left Behind Act of 2001. But while it’s not likely that standardized tests in schools will be going away anytime soon, there are definite signs that the assessment field is in a period of transition—brought about by, among other factors, widespread dissatisfaction with conventional testing formats, advancements in technology, and the implementation of the Common Core State Standards.
To get some insight on what the near future of assessment might look like for teachers, we talked to James W. Pellegrino, a widely recognized expert on cognitive science, psychometrics, and educational technology. Pellegrino is co-director of the Learning Sciences Research Institute and a distinguished professor of education at the University of Illinois at Chicago. He was also co-chair of the Gordon Commission on the Future of Assessment in Education, which produced a high-profile report last year arguing that testing needs to be radically re-imagined and more closely integrated with classroom-based teaching and learning.
Pellegrino is on the technical advisory committees of Smarter Balanced and the Partnership for Assessment of Readiness for College and Career, the consortia charged with developing assessments aligned to the Common Core State Standards.
Teachers frequently complain that current standardized assessments are not adequate—or even very good—measures of their students’ learning or their own performance. Are they wrong about this?
I wouldn’t say they’re wrong. I think they have a legitimate concern. And their questioning of standardized tests really gets at some of the important aspects of what it is we want students to learn and be able to do and whether our assessments are doing a good job of measuring and fostering those things. So teachers are right to worry about this. Right now, we tend to rely on highly restrictive kinds of assessment formats. And we tend to focus on particular kinds of knowledge, like being able to retrieve facts or execute procedures, and not really on getting at deep understanding of important ideas and principles.
Are there reasons for teachers to be more positive about assessment in the near future?
Yes, I think that if you look at the efforts of the two consortia to develop the assessments that are aligned with the common core, what you see are attempts to represent more complex kinds of thinking, and reasoning, and performances. The simple fact that both Smarter Balanced and PARCC have performance tasks as a major part of their assessments is one indication that they’re trying to get at things that are represented in the new standards that aren’t easily measured in typical selected-response type questions.
What is your impression of how those assessments are playing out and how successful they’ll be?
There’s been a lot of good work on item development, but there’s still a lot of field testing that needs to be done of the items and the tasks. In all honesty, I have to say that what the consortia were asked to do in four years’ time was a pretty herculean task—to try and develop assessments in grades 3-8 and high school in two subjects aligned to new standards and do it in a way that adequately got at the core concepts of what’s supposed to be mastered in the standards. So I think we’ll find that assessments are less than what some people might have imagined was going to happen after four years of work. But those were unrealistic expectations. On the other hand, I think the new assessments will teach us a lot about building better assessments. They’re not going to be perfect, but hopefully they will move us forward in a way that enables us to more adequately measure student achievement and also teaches what we still need to think about.
What are the key issues in assessment that you think need to be addressed over the next several years?
I think we need to give a whole lot of thought and emphasis to the development of quality tools and materials that can support teachers in enacting assessment as part of a learning environment in the classroom. In other words, we need to spend less time being so preoccupied with the large-scale standardized accountability tests. We still need to have assessments that do that kind of monitoring, but I and a lot of other people in the assessment field think that we need to reverse the emphasis. We need to focus on assessment for learning at the classroom level. We are so hung up on assessment of learning for purposes of accountability or monitoring that we’ve lost track of the investment we need in quality resources and materials to help teachers and help teachers help students.
The report by the Gordon Commission on the Future of Assessment in Education in which you were involved makes a similar point, specifically recommending “new assessment resources and tools that better integrate assessment with classroom teaching and learning and better represent current thinking on how students learn and on changes in the world at large.” Any sense of what that would look like for teachers in the classroom?
Well, in some ways, really good assessment of the type we’re thinking about doesn’t look all that different from really good instruction—or the kinds of tasks we would want students to engage in the classroom as part of their learning. So there tends to be a misconception that we’re just talking about building a bunch of new tests. What we’re really talking about is building tasks and situations in which students engage in the kinds of thinking and reasoning we want them to do with texts or mathematical solutions or scientific concepts—and engage in them in ways where they have to grapple with the key ideas. Some of this is about building better sets of tasks or scenarios that teachers can use as part of their instructional practice to help guide student learning. The tasks should allow students to show what they understand and do not understand—and therefore help the teachers enact formative assessment practices. It’s complicated because it means that we in the assessment community have got to get much closer to the world of curriculum and instruction.
What’s the role of technology in this?
I for one believe that technology has a very big role to play, because I don’t think you can enact some of these more powerful uses of assessment at the instructional level without technology to help manage the process, to help manage the information, so that you don’t put the entire burden on teachers. If the teacher has to deliver the tasks, has to score the tasks, has to make sense of the evidence, and then figure out what to do about it, it’s not going to happen. The problem we’ve always had in terms of teachers implementing formative-assessment practices is that it’s hard to do because there’s so much information to manage. Unless we can come up with technological tools and supports to help them manage that information—and we do have systems like that now—then I don’t think teachers have much of a chance to do the kinds of things we’d like them to do.
What are the specific systems available now that you would point to as models?
One good example is a program called “ASSISTments,” which comes out of the Worcester Polytechnic Institute. This is a web-based platform where students can work on various kinds of items, and problems, and get feedback and help. The system keeps track of how students are doing and it can give information back to teachers so that teachers can know how their class is doing and take action accordingly. To me that’s a great example of how we reduce the burden on teachers and use technology in smart ways that don’t take the important decisions out of the hands of teachers, because the teachers still need to be able to make judgments as to how to pace and refine instruction.
Should teachers be worried about losing autonomy or responsibilities in the instructional scenario you’re envisioning, since a lot of the learning activities would be facilitated online?
Not at all. In fact, they have a great deal of control over how systems like this function. They select the types of tasks that students are asked to work on and then use the information from the system to decide where to spend their time in class. A lot of the learning that is facilitated in the online environment is focused on very specific topics or learning issues that teachers might be challenged to cover effectively for all students in the regular classroom.
Would you have any advice for teachers on adapting to or preparing for this kind of environment?
Well, I think my advice to teachers would be to implement these kinds of tools and technologies in a sort of incremental way. Take it a bit at a time. Don’t try to do everything at once. It takes professional development to use these kinds of tools and to change one’s teaching practice to incorporate them. School leaders and policymakers need to be respectful of teachers given the fact that they’re going to need time and support to do this. So I guess one piece of advice I’d give to teachers is, don’t try to go it alone. If you want to incorporate assessment tools like we’re talking about, do it as part of a community of learners so that you have a support structure.
Do you have any concerns about overtesting?
Oh, yeah! I think that in the United States, particularly when it comes to large-scale testing, especially since No Child Left Behind, we are testing too much. We test every child every year in math and English/language arts, and we’re doing that because we like to get scores on every kid so we can use those scores for all sorts of purposes, including teacher accountability. We need to break out of that mindset—we need to say, “Look, we do need to monitor how well we’re doing in educating our children and how well they’re learning but it doesn’t necessarily mean that every kid, every year has to be assessed using large-scale, high-stakes tests.”
As I’ve been suggesting, a lot of what we want to know about student learning could be gleaned from information that could come out of the classroom. And if we want to monitor the system as a whole we could use more effective strategies like student sampling and matrix sampling of tasks. There are ways to do it that are more efficient to answer the kinds of questions that we want to have answered as a society, without requiring every kid, every year to take exactly the same test just so we ensure that Johnny’s score can be compared to Jimmy’s score.
What do you think testing will look like 20 years from now?
I think we will still have some legacy of what I call large-scale standardized testing in some form—I think it’s going to take a long time to shed that out of our educational DNA. You have to remember that the U.S. is the place that invented large-scale standardized testing, so it’s not something we can easily let go of. So I think we will still have some of that, but hopefully it will be used in more judicious ways, for monitoring purposes. Again, my sense—my hope—is that we will learn to use technology in more effective ways to gain the information we need about student learning out of classrooms and other contexts where students are doing their work, because much of the work will be mediated through technology. In this way, we’ll hopefully be able to get better, more useful, and more timely information. So it won’t be that we’re constantly testing kids, but we’ll be getting information from assessments of students’ performance that are more, you might say, ubiquitous as part of the work they’re already doing.
Here’s the metaphor I often use: It’s not perfect but I think it helps. Remember how, in the retail industry and major department stores, they used to have to close down stores once or twice a year to do inventory? That’s how they figured out what was selling and what they needed to restock. But then bar codes and scanners were invented. And now what happens is that every time you check out and buy something, inventory control is taken spontaneously. Stores are constantly accumulating information as to what’s being sold and what’s on the shelves, and they can do much better inventory control—and they don’t have to shut down twice a year to have someone go in and count everything.
I think the analogy applies to the way we do assessment. I’m not advocating bar-coding kids, but in a world where much of the work that we do is mediated by technology and is done in technology-rich environments and learning systems, we could glean a lot of information spontaneously that would help us to track important things. And we could embed into those environments periodic assessments that students take to demonstrate their competence, instead of having to shut down the school once a year for multiple weeks while kids take large-scale tests.
That raises another question, especially in light of recent news: Do you have privacy concerns about that kind of framework?
Yes, absolutely. I think that when making that type of analogy, we need to be very careful about the issues of privacy and tact. One obvious concern is the potential for the sharing of information well beyond the bounds of those who need direct access to it. So checks and balances would need to be put in place to secure and monitor access to the data that’s collected.
But there’s also the danger of going overboard with respect to the collection of data. We certainly don’t want kids or teachers to feel like their work is constantly being used as evidence. This is not a Big Brother scenario we’re trying to create. We want kids to be comfortable doing school work online without being judged—that’s part of the learning process. But then there are times when we want and need to monitor progress and collect evidence of learning to see if students have mastered important knowledge and skills. These wouldn’t be “stealth” assessments but rather ways for students to show they have mastered important standards in the context of their ongoing work. But we do need to be very clear about when and why we are collecting certain types of information.
