When it came time for Rachel Patterson to show what she’d learned during her eight-month senior project on sign language, the Barrington High School student didn’t just turn in a research paper.
Before a panel of five judges on a recent afternoon, the poised 17-year-old delivered a 10-minute presentation on the use of sign language to communicate with babies and people with autism. And in an approach particularly suited to her subject matter, Ms. Patterson expressed her thoughts by speaking and signing at the same time.
“It’s a really cool concept to show that you can make your thoughts and feelings visual,” she said.
Having students show their skills in three dimensions, known as performance-based assessment, dates back at least to Socrates. And individual schools such as Barrington High—located just outside of Providence—have been requiring students to actively demonstrate their knowledge for years.
But this spring, Ms. Patterson and the rest of Rhode Island’s high school graduating class became the first in the nation to face performance-based assessments as a state-mandated requirement for earning a diploma.
To be sure, no one is saying that Rhode Island’s trailblazing move means it’s time to start writing the obituary for machine-scored standardized exams. After all, even Rhode Island still uses them, and most experts agree that multiple choice is here to stay.
But as was underscored at the inaugural New England Symposium on Performance Assessment, held here May 28, more state education officials are starting to at least consider a type of testing that goes far beyond filling in the proverbial bubble, and that may help keep students better focused on their studies, both in high school and beyond.
“It really ramps up the meaning of senior year,” said Kevin Blanchard, an English teacher at Barrington High who helped pilot the school’s performance-based assessment model. What’s more, he added, requiring students to actively engage in a topic tends to better prepare them for college-level academics, as well as the work world, where on-the-job performance is generally the only gauge of competence.
An Unusual Mix
The requirement taking effect in Rhode Island this year stems from a 2003 policy change by the state board of regents and the guidance of Peter J. McWalters, the state’s outgoing commissioner of elementary and secondary education.
In addition to students’ class grades and scores on the New England Common Assessment Program, or NECAP—the standardized test Rhode Island shares with Vermont and New Hampshire—graduating seniors in the Ocean State must choose and pass two of three possible performance-based assessments: a portfolio of work selected from their four years of high school, a senior project, and a comprehensive course assessment.
For more stories on this topic see Testing & Accountability and Assessment.
Students who choose to assemble a portfolio must defend their body of work, including a research project that spans all four years, in front of a panel of judges.
Senior projects have included designing and implementing a poetry-writing course for adults, and building a snow machine and then using it to open a backyard sledding hill.
At least half of each end-of-course comprehensive course assessment must incorporate applied-learning and performance elements, such as presenting the results of original research. Students often work with mentors or work as interns in disciplines or on issues that interest them, such as architecture or the problem of child soldiers, and must build Web pages, PowerPoint slides, or other technology to present their findings.
The state’s unique mix of assessments was the main topic of conversation at the recent symposium on performance assessment hosted by the Rhode Island Department of Education. The event was co-sponsored by the Coalition of Essential Schools, a nonprofit group based in Oakland, Calif., that works to create more personalized and intellectually challenging schools, and the Portland, Maine-based Great Schools Partnership, an initiative of the Senator George J. Mitchell Scholarship Research Institute that works to redesign and strengthen education.
“Rhode Island is showing us the way, and so we’re here to learn from Rhode Island,” said Louis Cohen, the coalition’s executive director. He praised state education officials’ boldness and innovation, but noted, “The question they raise is, can they do this at scale?”
Officials acknowledge that the system remains a work in progress.
“There are some districts that haven’t done this to scale, even though it’s required,” said Sharon K. Lee, a middle and high school redesign specialist in the state’s K-12 education department. She added, though, that only five of the state’s 55 high schools were out of compliance.
Performance assessments “take a lot of time for us teachers to develop, test them out. And they take a lot of time in the classroom, because a lot of kids have no experience with this,” Theodore R. Sizer, a prominent education author and a former dean of Harvard University’s graduate school of education, said at the symposium.
Mr. Sizer, who founded the Coalition of Essential Schools, said a teacher trying to squeeze a history curriculum into class time, while juggling preparation for both standardized and newly implemented performance-based assessments, would likely “barely get to World War I by May.”
A Window Opening?
Large-scale performance-based assessment has traveled a rocky road over the past two decades. Even before the federal No Child Left Behind Act sent states scrambling en masse to roll out more standardized tests, the inherent subjectivity of grading student portfolios and dissertation-defense-style presentations had sunk some previous attempts to implement an alternative to machine-scored testing.
In 1990, Vermont began piloting what would become the first statewide assessment program to measure student achievement in part on the basis of portfolios. Officials there started backing away from the exclusive use of assessed portfolios, though, after a 1992 report by the Santa Monica, Calif.-based RAND Corp. found significant flaws in the way they were graded.
A decade later, New York state Commissioner of Education Richard P. Mills batted down a proposal by a network of nontraditional schools that wanted to substitute individually tailored projects for the standardized Regents exams in English that the state had began to require for graduation.
The network of 28 schools—the New York City-based New York Performance Standards Consortium—has since received a waiver that keeps its students from having to take Regents exams. But the exemption doesn’t cover the English exam, and the waiver runs out with the class of 2013.
But amid the disenchantment with the federal NCLB legislation that has spread since it became law in 2002, educators and alternative-assessment supporters in Rhode Island and elsewhere see a chance to show not only that performance-based assessments are superior, but also that they can be implemented on a statewide scale.
Eight other states have expressed interest in Rhode Island’s unique system, according to education officials here.
“There is a window opening for performance assessment now for the second time,” said Raymond L. Pecheone, who designed the nation’s first performance-based teacher-licensure system when he was in Connecticut, and is now the co-director of the School Redesign Network at Stanford University. But, he said, “This time we have to be smarter.”
“We are in a moment where there is a possibility for great big transformative change” around performance-based assessment, agreed Paul K. Leather, who’s in charge of high school redesign in New Hampshire, “but it won’t happen if our policies aren’t coherent.”
Coherency—specifically, its lack—helped doom Vermont’s experiment in the 1990s. The RAND report’s author, Daniel M. Koretz, now a professor of education at Harvard, found that Vermont’s “rater reliability’’—the extent to which portfolio graders agreed about the quality of individual students’ work—was very low.
The state eventually made portfolios optional and reintroduced standardized tests, which some experts say is only practical.
“Some things need to be demonstrated by performance, and that’s a fact,” said Chester E. Finn Jr., the president of the Washington-based Thomas B. Fordham Foundation, citing driving tests and music auditions. “But making those things count on a large scale and in a high-stakes environment is fraught with difficulties.”
The relatively recent addition of an essay portion to the SAT college-entrance exam, Mr. Finn added, shows that performance-based assessments still have their place. Still, he contended, “Vermont and Koretz pretty much killed off” the idea of statewide performance-based tests.
Roy M. Seitsinger Jr., Rhode Island’s director of middle and high school reform, begs to differ. “Been there, done that? No way,” he said. “This is just beginning of getting the fruit of decades of work.”
‘Refer Back to the Rubric’
Back at Barrington High School, before the afternoon’s set of performance assessments, work was the operative word for the 225 educators and community members gathered for a 90-minute tutorial on judging students’ presentations and portfolios.
Stoked with school-provided coffee, they lobbed questions from their seats in the school auditorium to Judianne P. Point and Stephen A. Lenz, Barrington’s senior-project coordinators:
“What if I know a student?”
“What if a topic doesn’t seem as hard as another student’s?”
“What happens if they go over 10 minutes?”
“In all cases, refer back to the rubric,” Mr. Lenz told them, standing on the auditorium’s stage and indicating the double-sided sheet of paper each judge would use for the students, grading them on 24 criteria—everything from the effectiveness of audiovisual features and the amount of eye contact to whether their presentations’ content adequately supported the main ideas.
Roughly half of each panel of judges is made up of regular citizens, but the other half comes from the statewide teaching pool. Ms. Lee and other state education officials say the experience of reading the portfolios of their colleagues’ students has increased teacher collaboration, and helped improve the consistency of scoring across schools.
Regardless of the students’ topic, said Ms. Lee, “We want them to have their own individuality. But we want the outcomes in terms of accountability to be the same.”
Therein lies the rub, some experts say.
“I don’t see how it’s possible to do the ‘fair and consistent’ aspect of what is always a subjective judgment,” said Mr. Finn. “If I sing a song and you cook an omelette, we’re each engaging in a performance, but it’s impossible to imagine the judges being equally well suited to evaluate both.”
“We have long encouraged performance assessment,” said Robert Schaeffer, a spokesman for the Cambridge, Mass.-based National Center for Fair & Open Testing, or FairTest, a watchdog group. But, he added, “Methodologically, there are some issues about doing high-stakes assessments with regard to performance.”
Rhode Island officials, however, say they are constantly tweaking the process, and are confident in their system.
In addition to school visits and spot checks by state officials and a cadre of teacher leaders and retired administrators, Mr. McWalters said, the state’s “collegial system” ensures that should the RAND Corp. want to again test the consistency of performance-assessment grading, Rhode Island’s system would pass muster.
Still, it is unclear whether Rhode Island’s performance-assessment system will be seen as a model for other states, especially in light of the budget constraints afflicting education departments nationwide.
Kenneth R. DiPietro, the superintendent of Rhode Island’s 5,500-student Coventry school district, estimated the cost of implementing the performance-based-assessment system at about $2 million just for the district’s one high school. Such costs add up to real money in a state facing a projected $430 million gap in its $7 billion fiscal 2009 budget.
Contracts and Caution
What’s more, schools’ ability to adequately implement the system depends on the goodwill and flexibility of teachers’ unions, because it requires extra staff planning time not covered in pre-existing teacher contracts.
“We have to give a little credit to our union,” said Raymond E. Spear, the chairman of the Coventry school board. “We made that a major issue in the last contract, to build in the flexibilities we felt we need to move forward with our reform and reorganization of our high school.”
That’s one union. But Rhode Island has 36 districts, and not all union locals may be receptive. “We have 11 districts that have contracts coming up,” said the state education department’s Ms. Lee, “so we’re getting a little nervous.”
It’s perhaps little wonder that other states are taking a wait-and-see attitude.
“I would just want to approach anything like [Rhode Island’s system] cautiously,” said Gail Taylor, Vermont’s director of standards and assessment. Voicing concerns over “the complexity and what you gain for that, given issues of capacity,” she added that “we have no plans at this point to mandate ... specific portfolio requirements again as we did in the past.”
New Hampshire is taking a similar approach, encouraging performance assessments in districts, but not mandating them.
Mr. Sizer called Rhode Island “a vital test ground” for statewide performance assessments, but acknowledged the hurdles of running such a program in a large state. “It’ll be different in different places,” he said. “I don’t know what you’d do in Texas.”
Rhode Island officials, however, are staying focused on the reason they adopted the performance system.
Driving back to Providence from Barrington High School in Mr. Seitsinger’s car, Ms. Lee described how well Rachel Patterson had done on “Unheard Words,” her presentation on using sign language.
Under the scoring procedure, the judges’ highest and lowest grades are dropped, and when the other three were averaged, she ended up with a score of 34.6 out of a possible 40—among the top scores of her group, and well above the passing threshold of 27.
In the defense phase of her presentation, Ms. Lee said, Ms. Patterson told judges that learning to sign had opened up a whole new realm of communication, especially because she has dyslexia and thus trouble reading and spelling written words.
Mr. Seitsinger beamed, but he also shook his head at Ms. Patterson’s high score. Referring to the paper-and-pencil-only standardized test Rhode Island students must also take, he said: “That’s the kind of young lady who would blow up on NECAP.”
