OBJECTIVE: All states have high content standards in English, math, science, and history for all children and assessments that measure whether students meet the standards.
Last year, at the National Education Summit, governors and business leaders concluded that “efforts to set clear, common ... academic standards for students in a given school district or state are necessary to improve student performance.”
Without standards to define what children should learn and how well they should learn it, we are traveling down a road with no destination.
Without assessments that reflect those standards, students, teachers, and the public have no way of knowing how far they must go to reach their goals.
The drumbeat to raise academic standards began in the early 1980s, when A Nation at Risk and other reports highlighted the mediocre performance of American schools. But “standards” in those days referred primarily to the number of courses that students took in the core academic subjects. And “achievement” was measured primarily by score on standardized, multiple-choice tests. These tests compared students against each other and not against some set level of performance. And they typically measured low-level skills and the simple memorization off acts.
Since the late 1980s, our definition of “standards” has changed considerably. Both educators and policymaker now talk about standards for the content of courses, what students should know and be able to do at various points in their schooling, and how well they should be able to demonstrate their knowledge and skills.
“Content standards” clearly define what students should know and be able to do in specific academic area by certain points in their schooling.
“Performance standards” define what students must demonstrate, and how well, to be considered proficient or accomplished in each subject.
The vast majority of states now either have or are developing content standards in the core subjects. But we know very little about the rigor of those standards.
No group evaluates standards across the 50 states to see whether they are rigorous or of high quality. And we did not attempt to measure that rigor here--although we would like to do so in future reports.
Beginning in 1995, the American Federation of Teachers took an important first step. It reviewed whether a state’s standard in the core subjects are “clear and specific enough” to be of use to classroom teachers or to build a curriculum. But while the AFT study is the most timely and comprehensive available, its “clear and specific” criterion does not really address the rigor of the standards.
We know even less about performance standards because the term means different things in different states.
Some people suggest that in a mobile society, what we really need are national standards. They argue that math should not mean something different in Arkansas than in Alaska. To others, even voluntary national standards raise the unthinkable specter of a national curriculum. We are not likely to settle this debate soon. The result is likely to be a patchwork of standards that vary from state to state-and even district to district.
Developing content standards is only the first step. Unless those standards are aligned with a state’s testing system, they will have little impact. So we also looked at whether states have developed an assessment program that could tell them whether their students have met their standards in the core academic subjects.
Most states have invested in an assessment system that measures student performance. The majority say they either have or are developing assessments that are “aligned” with their standards. In other words, the tests measure the content and skills that the state has said student should master.
Many states also are beginning to construct “performance” assessments that require students to demonstrate what they know by writing an essay, for example, or conducting a scientific experiment. A few states, such as Vermont and Kentucky, are collecting actual sample of student work over time through the use of portfolios.
But six states still rely on norm-referenced tests that compare how well their students perform relative to other students and not against some set standard of performance. These off-the-shelf tests rarely coincide with a state’s own content standards. And many of them stress low-level skills rather than the more ambitious goal that states are now setting for students. They also conflict with the premise that all children can achieve at high levels. Such tests distribute students along a bell curve, based on their performance relative to each other. Under that scenario, 50 percent of children can never meet the standard.
Three states--Iowa, Nebraska, and Wyoming--have no state testing system and no plans to develop one because they are strong believer in local control.
Finally, we tried to look at whether states use the results from their testing programs to hold schools, school districts, or students accountable for performance. One assumption behind the standards movement is that states will begin to monitor results--which is what they really care about-rather than regulating the minutiae of classroom life.
Today most Americans agree with the need for high academic standards, at least in theory. According to a national telephone survey of 1,200 American by the research group Public Agenda in 1995, 71 % of those polled said higher standards would lead youngsters to “pay more attention in their school work and study harder.” And 72% said young people would “actually learn more.”
In our own survey of 1,200 educators, 96% of superintendents and principals and 93% of teacher agree that “an effective public education system must be built around rigorous content standards that describe what students should learn in language arts, mathematics, science, and history.” And 89% of those surveyed agree that “states that set standards should also set performance levels that measure whether students have attained advanced, proficient, or inadequate mastery of the standards.”
However, as states and districts have quickly learned when they try to implement standards, the public often likes the idea better in theory than in practice. Debates are now raging across the country about whether state standards focus enough on academics, whether they ignore the basics, or whether they are too rigorous.
The furor surrounding the release of some of the voluntary national standards--particularly those in U.S. history--suggests just how controversial standards-setting can be. National standards have been published in most subjects, and states can choose whether to adopt them or develop their own.
Studies suggest that rigorous standards aligned with meaningful assessments can raise the quality of the education system. Cornell University Professor John H. Bishop looked at states, nations, and provinces that required students to pass exams tied to their curriculum at the end of high school. He found that such systems had higher standards for beginning teachers, paid higher teacher salaries, targeted more resources on core instructional functions, had students who scored higher in mathematics and geography, and employed more teacher with a major in the subject that they teach.
At the same time, we recognize that standards and assessments alone will not produce higher achievement. “Standards and assessments are the slices of bread holding the sandwich of educational reform together,” argues Colorado Gov. Roy Romer. “But the meat of the sandwich is the delivery system--the quality of teaching, the access to technology and laboratories, the depth and challenge of the curriculum.”
In choosing indicators for this section, we have relied heavily on the 1996 AFT report “Making Standard Matter” and used some of the information from that report. We did not use all of the data, however, because some of the criteria the AFT used for its approval of standards involve definitions over which reasonable people can disagree. Some states that have a major commitment to setting standards and developing assessments (like Kentucky) did not do well in the AFT evaluation.
A letter to the union from Kentucky officials illustrates some important differences. The letter notes that Kentucky’s “fails to meet” rating results from fundamental differences of opinion about the most appropriate way for the state to proceed with its standards-setting. The AFT criterion includes consequences for students; Kentucky holds schools, not students, accountable for student mastery of the standards. The AFT criterion holds that a state-designated core curriculum should account for as much as 80% of the curriculum; Kentucky allow more local choice. And the state disagreed with the union’s criterion for specificity, choosing instead to be “parsimonious rather than exhaustive in the specification of facts and other specific content, such as dates, places, and time periods for history.”
Our indicators, therefore, differ from the AFT’s. And, because we agree that “you get what you test,” we focused more on assessment.
We did not give a grade to California, Colorado, Massachusetts, or Minnesota because their assessment systems are currently being developed. They were marked “incomplete.”
THE INDICATORS
Standards. 1. Has the state adopted content standards in the core subjects? We have based one-fourth of a state’s final grade on whether it has adopted content standards in the core subjects. This is a surprisingly difficult piece of information to get. The answer varies depending on whom you ask and when you ask it. One problem is that states define standards differently. One state’s standards may be another state’s curriculum frameworks. To find out whether state have standards that they are using, we examined material from a variety of sources, including: data gathered by the AFT, the Council of Chief State School Officers, and the North Central Regional Educational Laboratory; copies of state documents; and interviews with state officials. States that have adopted standards in the core subjects score an A, or 100%; those that have adopted standards in only two or three subjects score a B, or 85%; those that are developing content standards score a C, or 75%; states that have taken no action score an F, or 50%.
A few states, such as Colorado, Michigan, Montana, and Wyoming, are developing “model” standards at the state level and then encouraging districts to adopt them or to develop their own. We have only given credit to those states in which the model standards are backed up with aligned assessments or the standards that districts develop must be at least as rigorous as the models provided by the state.
2. Do the standards set high expectations for student learning? We have no answers to this crucial question. The AFT grades standards on their clarity and specificity, which is useful. But it is possible to have clear and specific standards that are watered down. Until we know whether a state’s standards are rigorous, or comparable to those in other state and nations, we do not know very much.
Assessments. 3. How does the state measure student performance? We have based one-half of a state’s final grade on whether it has made the commitment to develop an assessment system aligned with its standards and the extent to which it looks at actual student performance. We have weighted this more heavily than whether states have developed content standards because: (a) it represents a more sizable commitment on the part of state, (b) it is the best indicator of whether state are using standards to drive reform, and (c) without aligned as assessments states cannot tell if they have reached their goals.
However, this indicator does not measure the quality of a state’s assessments. Instead, it reflect whether the assessment method a state has chosen are aligned with the state’s content standards. We also give more credit to states that have developed writing assessments and other performance-based exams that could yield richer information on student learning.
As with standards themselves, states can define what they mean by “alignment” differently. One state may develop test items based directly on its content standards. Another state may compare its test items with its content standards only after the item, are developed. We used our best judgment in determining whether a state’s assessments are, in fact, aligned with its standards based on information provided by the states and gathered by national organizations.
In scoring this column, we gave an A, or 100%, to states that developed their own criterion-referenced assessments aligned with their standards, as well as performance assessments that present a more multifaceted picture of students’ work. We gave a B+, or 88%, to states that developed their own criterion-referenced tests and use a writing assessment.
We gave a B, or 85%, to states that have developed a criterion-referenced test only. We gave a C+, or 78%, to state that use a writing assessment and a commercially produced criterion-referenced test that may be only partially aligned with their standards. We gave a C, or 75%, to states that only u e an off-the-shelf criterion-referenced test. We gave a D+, or 68%, to states that use a norm-referenced, standardized test and a writing assessment. We gave a D, or 65%, to states that use only a norm-referenced test. We gave an F, or 50%, to three states that have no state assessment system--Iowa, Nebraska, and Wyoming.
States that are developing assessment systems but have not yet put them into place received a grade of “incomplete.”
A few states--Connecticut, Illinois, Indiana, Louisiana, and Maine--currently have assessments that are aligned with their existing state goals, standards, or frameworks. However, these states are updating their standards or developing new ones, and we will track whether their assessments are revised over time.
4. Are all four core subjects assessed? This indicator is worth 5% of the state’, final grade. States whose assessments measure the “core plus"--namely, math, English, science, social studies, and at least one other subjects receive an A+, or 105%. Those that measure all four core subjects--math, English, science, and social studies--receive an A, or 100%. States that measure only three of those subjects receive a B, or 85%. States that measure only two subjects, typically reading and mathematics, receive a C, or 75%. States that do not have an assessment system score an F, or 50%. So do states whose assessment system is “incomplete” with no pieces yet in place.
Additional indicators. We have based the final 20% of a state’s grade on the following three indicators, which are given equal weight.
Accountability. 5. Will students have to master the standards to graduate or be promoted? This information comes from the AFT survey. According to Public Agenda’s survey, almost half of Americans believe it is possible to get a high school diploma without possessing even the most minimal skills--the ability to read, write, and do basic arithmetic. But public-opinion polling indicates that most Americans think students should be passed only when they have mastered the required material. In our own survey, 81% of superintendents, 85% of principals, and 84% of teachers agree that students should be required to demonstrate mastery of the content standards in order to be promoted or to graduate from high school.
States that have or are planning to link their standards to student graduation or promotion get an A, or 100%. Those that do not get a C, or 75%. We did not fail any state because some argue that there should be no high stakes for students until the standards and assessments have been in place for a reasonable period of time and all students are guaranteed access to the resources and teachers needed to achieve the standards. This is a reasonable argument.
6. How does the state hold schools and/or districts accountable for student performance? This indicator measures whether states have an accountability mechanism in place. States that provide incentives or penalties for schools or school districts based on their performance get an A, or 100%. Those that regularly report student test scores to the public get a C, or 75%. Those that do not even report their test scores in a readily accessible form get an F, or 50%.
7. Did the state participate in the most recent National Assessment of Educational Progress? NAEP currently provides the best national information we have on whether students are making progress toward high level of achievement. It has monitored trends in student performance in the core curriculum areas since 1969. It can provide states with some indication of whether their own standards are rigorous enough. Studies have shown that many states set their own criteria for what constitutes “proficient” performance much lower than those set by NAEP. States that participated in the 1996 NAEP reading assessment get an A, or 100%. Those that participated at one grade level only get a C, or 75%. Those that did not participate get an F, or 50%.
