A privately funded effort to develop a national examination system took a major step forward last month when some 450 educators and policymakers met here to lay the foundation for such a program.
During a weeklong workshop, teachers and curriculum specialists from 15 states and six school districts put together assessment tasks in mathematics, reading, and writing that could form the building blocks for a national exam.
At the same time, a group of local, state, and national officials agreed on a rough outline of the examination system, and began discussing steps needed to put it in place.
Lauren B. Resnick, director of the Learning Research and Development Center at the University of Pittsburgh and co-director of the effort, known as the New Standards Project, called the meeting a resounding success. As a result, she predicted, a pilot examination in one or two subject areas in one grade level could be in place by next spring.
“We’d like to get on the road,” Ms. Resnick said. “The only way to learn how to do it is by doing it. By trying it out, you get farther than by sitting in an ivory tower trying to see how to do it perfectly.”
Meeting participants said that, after some struggle, they were able to develop a consensus on a set of challenging tasks in math and literacy. But they warned that reaching an agreement in other subjects might be more difficult.
Their caution was underscored by a bitter impromptu debate over multicultural education between James A. Banks, professor of education at the University of Washington, and Albert Shanker, president of the American Federation of Teachers.
The educators also pointed out that even if the project can produce an examination system, its larger goal of transforming the education system to ensure that all students are capable of doing well on the exams is likely to be much more difficult to achieve.
Many teachers and parents remain unconvinced of the need to make fundamental changes, observed Leslie Paolitti, a math teacher at Choate-Resemary Hall School in Wallingford, Conn. To win over “teachers, parents, and the general public, a huge education job has to be done,” she said.
But while Marc S. Tucker, president of the National Center on Education and the Economy and co-director of the New Standards Project, acknowledged the magnitude of the task ahead, he expressed optimism that the changes would occur.
“People view [the difficulty of transforming the education system] the same way they viewed the impossibility of a national examination system,” Mr. Tucker said. “It ain’t necessarily so.”
Filling a Vacuum
Funded by grants from the John D. and Catherine T. MacArthur Foundation and the Pew Charitable Trusts, the New Standards Project is one of several initiatives currently under way or under discussion to create a national assessment system. (See Education Week, July 31, 1991 .)
President Bush also has endorsed the concept and made it a centerpiece of his education strategy.
Unlike several other plans, however, the New Standards Project calls for an overall system of assessment, rather than a single national test. Under such a system, districts and states could develop their own tests, which would be compared with national standards.
The project also emphasizes the use of alternative forms of assessment-including performance tasks, portfolios, and projects--instead of traditional multiple-choice tests.
In addition, the New Standards Project differs from other plans in its emphasis on creating a new education system in which all students attain “world class” standards.
“If all we have are beautiful assessments, it’s not worth it,” said Ms. Resnick. “Without an assessment, it’s not going to happen.”
Gov. Roy Romer of Colorado, co-chairman of the National Council on Education Standards and Testing, a Congressionally authorized panel that is examining the feasibility of national assessments, applauded those involved in the New Standards Project for getting out in front and working to put their plans into action.
But he warned conference participants not to deliberate too long. Otherwise, he said, policymakers may opt for a quick solution--such as expanding the National Assessment of Educational Progress to test all students, not just a sample--that may not meet the project’s goals.
“The White House is interested in [national testing]; there is interest in the Department of Education,” Governor Romer said. “But there is a concern that Tucker-Resnick takes too long.”
If the project delays putting something in place, “they’re going to reach for something that’s on a shorter timeline,” he continued. “Somebody will fill that vacuum.”
Mr. Tucker responded that he was aware of the political imperative.
The project is aiming to develop a pilot examination, he explained, that could then be improved over time.
“You don’t need to have everything perfect at the outset,” he said.
‘Radical’ New Assessments
As part of its effort to create an exam system, the project had already held a meeting, in July in Sagamore, N.Y., to try to determine if assessments from different states could be judged according to similar standards. The purpose of the exercise, according to Ms. Resnick, was to demonstrate whether it is possible to calibrate separate tests to a national standard.
Using writing tests--the most commonly used type of performance assessments--teachers and assessment experts from 10 states evaluated student responses from other states and compared scores.
The results indicated a “stunning” degree of agreement among state raters, Ms. Resnick said.
“Without preparation, and without any effort to set a shared standard in advance,” she said, “we are, for those 10 states, and I suspect the rest, close to a shared standard for the dimensions of good writing.”
Ms. Resnick added, however, that the states “do not yet agree on what is good enough.”
To begin that task, the project invited teachers and curriculum specialists to Snowmass to develop timed performance tasks, portfolio tasks, and long-term projects that could be used in an examination system.
The process they used is the reverse of traditional test development, Ms. Resnick explained. Usually, standards are set, and items are created from the standards. In this case, she said, the standards will be drawn by “an induction process” from the tasks.
“These are new psychometrics,” Ms. Resnick said. “We’re doing a quite radical new type of assessment.”
Agreement Still Elusive
Although teachers involved in the process said they found it difficult, they emerged with a set of tasks officials called promising. (See samples, this page..)
Over the next year, the teachers are expected to try out the tasks in their classrooms, while they and project officials hone them into a prototype national examination.
Victoria Young, a project director in the division of student assessment in the Texas Education Agency, said those developing reading and writing tasks were hampered by a lack of an agreement on how the subjects should be taught. By contrast, she said, math educators, led by the National Council of Teachers of Mathematics, have neared a consensus on what should be taught in that subject.
“We were concerned that we don’t have what the math people have,” Ms. Young said. “We were working in a vacuum.”
But math teachers--particularly those from states and districts that have just begun to implement alternative forms of assessment--said they were having their own share of trouble in coming up with assessment tasks that would measure a high level of knowledge and skills.
“It was difficult,” said Ray Fisher, a math teacher at Mount Ararat High School in Topsham, Me. “There are not a lot of models for what we are trying to do.”
Participants also suggested that the fiery debate over multicultural education between Mr. Banks of the University of Washington and Mr. Shanker of the A.F.T. was a sign that agreement on standards in other subjects, particularly the social studies, might prove elusive as well.
In remarks to the conference, Mr. Banks emphasized the need to expand the curriculum to include “multiple voices and perspectives.”
“Multicultural education is education for freedom,” he said. “It’s as much for whites as it is for students of co lot.”
In an impassioned response, however, Mr. Shanker argued that the type of education advocated by Mr. Banks risks divisiveness and a loss of public support. While expressing agreement with “95 percent” of what Mr. Banks said, the union president attacked the remainder as a “political agenda” not shared by most Americans.
“If there isn’t a set of common understandings kids should know and be able to do,” Mr. Shanker said, “we have no business being in business.” Mr. Tucker agreed that the debate pointed up the difficulty of achieving a consensus in the social studies. But, he argued, a problem in one field need not impede the entire project.
“It may mean we’ll end up with a national examination system in reading, writing, math, and science,” and not have one in social studies, Mr. Tucker said. “We’d [still] be better off than we are now.”
‘Menu’ of Tasks
In addition to developing tasks, participants at the workshop here began to grapple with issues surrounding the structure and implementation of the exam system.
Working with some leading psychometricians, the project’s policy advisory group came up with a complex solution to the problem of how to allow local flexibility while maintaining a national standard, according to Ms. Resnick.
“The heart of the answer,” she said, “is that the [national exam] isn’t a test all packaged with a ribbon around it, but a set of performances, taken together, that describe what the standard is, and allow inferences about what students can do.”
“If California wants to be in the ... system,” she added, “it will build an exam, in part, from this menu.”
Ms. Resnick stressed that the “menu” would include each of the “three P’s"--performance tasks, portfolios, and projects--since they measure different aspects of student abilities.
Eva L. Baker, co-director of the federal center for research on evaluation, standards, and student testing, said having a menu of tasks from which states and districts could choose, rather than a single core national exam, would eliminate the possibility that a national exam would narrow the curriculum.
“The argument for a menu of tasks was to avoid the specter of having an anchor test,” she said. “People worry about teaching to that.”
‘Social Contract’
In addition to deciding the structure of the exam system, the policy group agreed that states and districts participating in the project should agree to a “social contract” to ensure that students are able to attain the high standards the exam measures.
“We’re moving from sorting to making opportunities available by saying we have an obligation to help you meet [the standards],” said Sharon Robinson, director of the national center on innovation at the National Education Association.
David W. Hornbeck, former state superintendent of education in Maryland, suggested that such a commitment could open up a new category of school-finance-equity cases. Rather than file suit against states for failing to ensure adequate resources, districts could sue for failing to provide the support needed for adequate performance on the exams, said Mr. Hornbeck, who as a consultant played a central role in the design of Kentucky’s court-ordered school-reform law.
But Lorrie A. Shepard, professor of education at the University of Colorado at Boulder, said that the commitments made by states and districts must be explicit, or else students could be sorted into high- and low-quality programs, as they are now.
“I’m terrified of the rhetoric of legal implications,” she said. “I need to have independent evidence teachers are making changes in instruction, and that kids are not being sorted to improve [a school’s] performance on the assessment.”
Teacher Education Stressed
Mr. Shanker argued that, to be effective, the exam system must include consequences for students and school professionals who meet--or fail to meet--the standards.
Using an analogy from a different policy realm, Mr. Shanker maintained that efforts to conserve energy have failed because there have been no incentives to do so. “Without having stakes in there, people are not going to listen,” he said.
But Edmund W. Gordon, professor emeritus of psychology at Yale University, said that attaching consequences to exam results might prove harmful to most students, since few can currently meet the standards the project envisions.
Rather than impose the consequences immediately, suggested Ms. Resnick, states and districts could phase them in as they upgrade their instructional programs.
Others at the meeting here warned that it might prove difficult to implement the changes in curricula and instruction the project’s sponsors consider necessary.
Several states and districts have already revamped their curricula and assessment programs, and may be unable to change to reflect the project’s aims, noted Cheryl Tibbals, division director of performance testing for the Kentucky education department.
“I’ve got to see how this dovetails with what we are doing,” she said. “Our time line is so much faster.”
In other states, few teachers are prepared to teach the way the examinations would demand, Ms. Young of Texas pointed out. Enabling them to bring students up to high standards will require massive funding for teacher education, she said.
“The truth is, a lot of teachers are not very skilled,” Ms. Young said. “We’ve assessed writing for 10 years, and teachers are still asking basic questions, like ‘How many words do [students] need to pass?’”
“We’re talking about re-educating the nation’s teachers,” she said.
Kathy Murray, a math teacher at A.C. Moore Elementary School in the Richland School District in South Carolina, said she agreed that the project would require a great deal of teacher education and that many teachers might resist it.
Even so, she said, “A drastic vision needs to be imagined so changes can
“I don’t know how it’s going to work,” Ms. Murray said, ‘but this week was a great start.”
