Plans for a radically more ambitious and expensive version of the nation’s “report card’’ on student achievement were unveiled last week by a blue-ribbon panel convened by Secretary of Education William J. Bennett.
In its report, the 22-member study group advocates redesigning the National Assessment of Educational Progress to provide state-by-state data, measure learning in more core subjects, include out-of-school 17-year-olds, and provide a larger sampling of private-school students.
The panel also recommended a new governance structure for NAEP that would place its future direction in the hands of an independent council “buffered from manipulation by any individual, level of government, or special-interest group in the field of education.’'
Such changes would “vastly improve our ability to keep track of what our children know and can do,’' the study group argues in “The Nation’s Report Card: Improving the Assessment of Student Achievement.’'
Current assessments, it states, “are not producing answers to the questions most often asked at this moment in history by parents, by concerned citizens, and by educators.’'
At a press conference held here to release the report, Mr. Bennett warmly endorsed the panel’s work. Few reform proposals of the 1980’s, he said, would have a “more significant long-term impact’’ on education in the United States.
“While I still want to study the details,’' the Secretary said, “I certainly intend to move forward with legislation and to seek authorization to put an improved report card into the nation’s hands.’'
Mr. Bennett said he needed more time to consider the panel’s proposal that annual federal funding for NAEP be increased five-fold, from about $5 million to $26 million.
But he added: “We want to go ahead with this. I will be looking for the money. ... I am pledging today a serious and earnest effort on our part to find whatever monies we can.’'
According to the panel, the budget it has proposed is a “minuscule sum within an elementary and secondary education system that is spending approximately $170 billion this year.’'
It urged quick adoption of its recommendations, noting that it would be a number of years between the redesign of the assessment and the availability of test results based on those changes.
Officials from the Education Department predicted that even if the Congress enacted the proposals immediately, most of the changes would not be reflected in NAEP tests until 1992.
The Reagan Administration’s budget request for the coming fiscal year would increase funding for NAEP from approximately $5 million to $7.9 million--$1 million of which would be used to start planning for potential changes in the assessment.
Additional changes in NAEP’s budget to reflect the study group’s recommendations would not appear until fiscal year 1989, Mr. Bennett said.
A review of the report by a committee of the National Academy of Education--a select group of scholars and practitioners in the field--described the lack of support for NAEP in recent years as “fiscal starvation.’' That review was published along with the report.
18th Year
Now in its 18th year, NAEP is a program authorized by the Congress to test a sample of students in reading, writing, mathematics, and other subjects on a regular basis.
Mr. Bennett formed the study group last May to find ways to make the assessment more informative, useful, and efficient.
Lamar Alexander, former Governor of Tennessee, chaired the panel of educators, testing experts, state officials, and private citizens.
With funds from the Education Department and a number of private foundations--including Exxon, Ford, Hewlett, MacArthur, and Matsushita--the panel commissioned 46 background papers and convened nine committees to examine specific questions about NAEP.
State Data
The most important and expensive of its recommendations is that the number of students tested by NAEP be increased, to allow individual states to gauge how their students are performing over time and in comparison with those from other states.
The federal government would fund the entire cost of this expansion, estimated at $13.5 million per year.
At present, though states can pay to obtain some state-level figures, NAEP only supplies data at the national level, and for four broad regions of the country.
The expanded sample would provide each state with data on student performance in the core subject areas of reading, writing, and literacy; mathematics, science, and technology; and history, geography, and civics. NAEP would also continue to provide national data in these and other subject area.
States and localities could also pay to obtain additional information in these other subject areas; or about particular subgroups of students, such as minorities; or at the district and school levels.
Although states would not be required to participate in the national assessment, Mr. Bennett predicted that they would.
“There is very great public interest in this question of ‘How are we doing? What are our children learning?’'' he said.
New Attitude
The panel is only one of many groups now proposing that states compare student performance. The Council of Chief State School Officers, for example, has been working on a plan for collecting such data, and has suggested that NAEP might provide the most effective way to do so.
Ramsay Selden, director of the C.C.S.S.O.'s state education-assessment center, said that members of the group still have to review the panel’s recommendations. But he predicted that “most of our members will find this an attractive way to proceed, as an alternative to trying to do something parallel ourselves.’'
The change in attitudes toward state-by-state comparisons since NAEP was first created in the 1960’s is striking, according to many observers.
“What is being discussed today wouldn’t even have been considered 20 years ago,’' said Michael W. Kirst, a member of Secretary Bennett’s study group and professor of education at Stanford University. “Anyone who had proposed it would have been laughed out of the room.’'
Interest in comparing student performance among states has been fueled in recent years by the billions of dollars going into state-level education reforms, and by the annual “wall chart’’ on educational progress produced by the Education Department.
The wall chart, which uses scores on the two national college-admissions tests to compare students’ performance across states, has been roundly criticized by educators as inadequate and misleading.
At the press conference last week, Mr. Bennett admitted that the wall chart was flawed, but said the department would continue to produce it because “it’s the best we have.’'
Other Recommendations
The study group stressed that whatever changes in NAEP ultimately are adopted, the redesigned assessment must maintain continuity with its existing data base.
“The information that NAEP has generated since 1969 is the best ‘baseline’ data available concerning what children know and can do,’' according to the report.
Some of the key recommendations of the study group include:
- Regular assessments in the core subjects of reading, writing, and mathematics should be expanded to include reading, writing, and literacy; math, science, and technology; and history, geography, and civics.
- The tests should focus more on “higher order’’ thinking skills, such as problem solving, and should explore the use of new measurement techniques that go beyond standard multiple-choice formats.
- NAEP should continue to gather data on children ages 9, 13, and 17. But grade-level samples should be changed from the present grades 3, 7, and 11 to the more important “transition” grades of 4, 8, and 12.
In addition, NAEP should regularly test out-of-school 17-year-olds, and should include even older age groups--perhaps 21- and 25-year-olds--in its studies of literacy.
- The sample of private-school students should be enlarged to “draw valid conclusions about student achievement in individual grades and major subgroups of private schools.’'
Just as a state could pay for “add ons’’ to get more detailed information under the envisioned restructuring, so, too, could organizations of private schools. In addition, the panel has proposed creating an independent agency, the “Education Assessment Council,’' to set policies and future directions for NAEP.
Governance
The program would be run by a separate testing contractor, under contract with the federal government.
Although NAEP now has a committee of outside experts, its members are appointed by the NAEP contractor, currently the Educational Testing Service. Each time the contractor is changed, the committee is reconstituted.
The new council’s members would be appointed by the Secretary of Education--for overlapping five-year terms--from among candidates recommended to him by a permanent, statutory nominating committee. Council members would represent a broad array of testing experts and national, state, and local representatives.
The federal government would award a grant to an independent organization, chartered by law, to house the council.
It would receive up to $2.5 million a year, and employ perhaps 10 professional staff members.
The panel recommended that the new council be in place and functioning in time for the 1990 assessment.
‘Compliments and Caveats’
In its critique of the report, the National Academy of Education stressed that the new council must be independent, and that its policies and specifications must be followed in designing the contract for NAEP.
It criticized the relationship between the council and the Secretary of Education, as described in the report, as “somewhat ambiguous.’'
Robert Glaser, chairman of the academy committee that reviewed the report and director of the learning, research, and development center at the University of Pittsburgh, characterized the academy’s comments as a mixture of “compliments and caveats.’'
“As the mission of NAEP expands,’' the committee noted, “the possibility of problems arising from misinterpretation of data and overzealous use of test results may grow.’'
“We are not predicting that any of these problems will necessarily emerge from a new NAEP,’' it noted, “but other testing experience shows that such events are possible.’'
“It would not be prudent to assume that the new NAEP will be free of unintended consequences.’'
Some of the academy’s “caveats’’ include the following:
- NAEP should be cautious about assuming responsibility for a broad range of related research activities.
“The new NAEP, with its very full agenda, is in danger of spreading its efforts too widely and thereby sacrificing quality in its principal activities,’' the committee cautioned.
That view was endorsed by Linda Darling-Hammond, a member of the study group and director of the RAND Corporation’s education and human-resources program, in a statement included with the report.
- A group should be created--outside NAEP itself--to monitor the test’s effect on schools.
“Although school practice has been largely impervious to NAEP activity in the past,’' the committee noted, “this may change dramatically’’ once the assessment is changed, and “not all these changes may be for the best.’'
- NAEP should avoid developing hierarchies of skills and subskills based on assumptions about how students learn.
More research in this area is needed, the committee noted, and “it is far from clear that a single pathway to learning can ever be appropriate for all youngsters.’' Such hierarchies, it cautioned, could form the blueprint for a curriculum.
- The committee similarly advised against making broad generalizations about the factors that lead to success or failure in school based on NAEP data.
"[F]ew such questions are well suited for examination within the current NAEP design,’' it stated.
- Continued caution should be used in making state-by-state comparisons based on average test scores.
“Many factors influence the relative ranking of states, districts, and schools,’' the committee stated. “Simple comparisons are ripe for abuse and are unlikely to inform meaningful school-improvement efforts.’'
Important Goal
Despite such cautions, however, H. Thomas James, vice chairman and study director for the Secretary’s panel and president emeritus of the Spencer Foundation, said he did not expect much opposition to the panel’s recommendations from the education community.
“The opposition of the teachers, of the school administrators, began to evaporate very early in the history of NAEP and has virtually disappeared,’' he said.
Albert Shanker, president of the American Federation of Teachers and a member of the academy’s review committee, said that revamping NAEP had been “third or fourth or fifth on the agenda of most education groups.’'
But he added that there was a “realization on the part of all of these groups that even though it’s not a constituency issue,’' strengthening NAEP is “one of the most important things we can accomplish in this round’’ of education reform.
Copies of the report are available for $9 each, prepaid, from the National Academy of Education, 108 Longfellow Hall, Cambridge, Mass. 02138.
