In the paneled Cabinet room of the modern state Capitol here, a group of educators and public officials recently laid out the future of Florida education.

From Risk to Renewal:
About This Series

Part 1

February 10, 1993

• • Charting a Course to Reform: The Next 10 Years
• • Mission Impossible? Educators Confront Tough Task of Rethinking Goals
• • Obstacle Course: Barriers to Change Thwart Reformers at Every Twist and Turn
• • The Future of School

Part 2

February 24, 1993

• • Intro: The Balance of Power
• • Time and Space
• • The Coherent Curriculum
• • States Move From ‘Inputs’ to ‘Outcomes’ in Effort to Regulate Schools
• • Basic Training
• • Dollars and Sense
• • Signing Up the Public

Part 3

April 21, 1993

• • Charting a Course: The Next 10 Years
• • Kentucky Moves To Enact Reform Plan
• • Policymakers, Educators Seek Strategy

Known as the Florida Commission on Education Reform and Accountability, the panel approved a set of measures to assess the progress of schools toward seven performance goals set in a 1991 state law. Schools, which under the law are required to develop their own plans for meeting the goals, must show “adequate progress’’ on such standards as insuring that all students can communicate and can solve mathematical problems. Schools that fail risk intervention by the state.

The new system holds schools responsible for results, but frees educators from mandates about how to achieve them. And it represents a 180-degree shift in the way that Florida governs its schools.

Florida’s bold action places it at the leading edge of a national movement on school accountability that is essentially shifting the emphasis in schools from “inputs’’ to “outcomes.’'

To accomplish their educational goals, states have traditionally dictated in considerable detail how time and resources should be used in schools--from the number of teachers and books in the school library to the academic schedule and the content of every course. As a way of holding schools accountable for meeting the goals, students were required before graduation to take a basic-skills test to demonstrate their levels of achievement.

This approach resulted in more students mastering the basic skills and passing minimum-competency tests, but it didn’t do much to foster higher-order thinking skills. And it severely limited innovation and creative teaching.

Now, states are developing a new compact--a new arrangement in which broad standards are set for what students should know and be able to do and schools are free to determine how to get their students there. In return, though, schools will be held accountable for results.

Charles W. Fowler, the superintendent of the Sarasota County public schools and a member of the Florida accountability commission, says the new approach is much more likely than the previous effort to lead to high levels of student performance.

“It’s a welcome opportunity for professional educators, who for years functioned under the handicap of having other people tell them how to do their job,’' Mr. Fowler says. “Now we’re saying, ‘We will not tell you what to do, but there is going to be accountability for what students ought to learn.’ ''

Not surprisingly, Florida and other states are finding out that putting these new accountability systems into place is a lot more difficult than designing them on paper. Among the most contentious issues are:

• Defining performance standards. Deciding what outcomes will be expected from students has sparked heated debates in a number of states, but particularly in Pennsylvania. There, the House last month voted to kill its entire outcome-based program, under fierce pressure from critics who contended that a set of outcomes adopted by the state board of education fostered the teaching of “values,’' rather than academic skills and knowledge.

• Eliminating existing regulations. Getting rid of the “process’’ regulations that until now have largely defined educational systems has proved equally controversial. In New Hampshire, education groups rebelled when the state board proposed to remove rules that specify how services are delivered in schools. They argued that, in the absence of such rules, districts would fail to provide equitable services for all students.

• Developing assessments. Performance-based systems demand that students exhibit abilities seldom measured by traditional tests and that they demonstrate they can actually use knowledge, not simply fill in pre-selected answers. In places where new outcome standards have been defined, the rich assessments needed to measure progress toward them don’t yet exist. And, in the interim, states are using existing tests that are not aligned with the curricula and pedagogies being used in the schools.

• Creating incentive systems. To motivate schools to produce the desired results, states are developing systems of rewards and penalties. In the process, policymakers are struggling with questions of efficacy and fairness. Doubts are often raised about the power of such arrangements to motivate individuals at the school level, and whether the school is the proper unit of accountability.

• Agreeing on delivery standards. Even those who support outcome-based systems argue that they could exaggerate the inequities that exist among schools unless all students have access to a high-quality education. In a report issued last fall, Linda Darling-Hammond, the co-director of the National Center on Restructuring Education, Schools, and Teaching at Teachers College, Columbia University, argues that, for accountability to succeed, states must also insure that all schools have the capacity to provide the type of education that will enable all students to meet the standards.

“Outcome standards alone cannot guarantee accountable schools,’' Ms. Darling-Hammond writes.

‘Outcomes’ Come to Mean Many Things to Many People

Like many movements in education, the idea of “outcome-based education’’ has swept through the nation rapidly. In addition to Florida, 26 states claim to have developed such systems, and another nine are moving toward them, according to the Education Commission of the States.

But the term has also taken on the characteristics of a buzzword, meaning different things to different people. Several states are using student outcomes as part of their systems for accrediting schools; others say they are outcome based if they have implemented performance-based assessments.

“Anything with a test on the end is ‘outcome based,’ '' says Lauren B. Resnick, the director of the Learning Research and Development Center at the University of Pittsburgh.

In general, educators use the term “outcome-based education’’ to refer to a system that specifies the performance--such as an understanding of math concepts or an appreciation of different types of literature--that students must demonstrate. Such systems generally include new forms of assessment that enable students to demonstrate such performances.

By contrast, most education systems specify “inputs’'--such as requirements that students study four years of English and three years of math--and assume that requiring the inputs would lead to desired outcomes. Tests generally measure a survey of knowledge in such coursework.

As a document from Florida explains the shift, the new system asks not “How many students are enrolled in a class called Algebra I?’' but “How many students can successfully apply algebra in solving a problem?’'

Florida’s Long History

Few places have adopted the outcome-based approach as comprehensively as Florida. And few states have taken such a long road to get there.

In an 11-page memorandum to the state’s accountability commission, Sen. Bob Johnson, a former member of the panel, showed that Florida has had a 30-year history of attempting to hold schools accountable for student performance.

As early as 1971, the memo shows, the legislature passed a comprehensive “educational accountability act,’' which, among other provisions, required the state commissioner of education to establish educational objectives for each subject area and grade level. The law also directed the commissioner to develop a statewide assessment system to determine “the degree to which established educational objectives had been achieved.’'

In carrying out the law, the state department of education created a basic-skills test and required students to pass it in order to graduate from high school. In a landmark ruling that helped spur the spread of minimum-competency testing throughout the country, a state judge ruled in 1983 that the state could legally withhold diplomas from students who failed to pass the test.

Michael W. Kirst, a professor of education at Stanford University, says the accountability system of the early 1970’s served its purpose at the time by insuring that all graduates attained a minimum level of knowledge and skills. That approach “may have been the right thing for that period of American history,’' he says. The newer program, however, in keeping with today’s reform ideas, is aimed at raising student performance to a higher level.

Use of ‘Report Cards’

Florida was not alone in attempting to hold schools accountable for student performance.

In a wave that began in the 1970’s and intensified after the 1983 release of A Nation at Risk, most states adopted some type of statewide testing program. Today, 47 states have such a program.

States have also used the test results in a variety of ways. Thirty-six states now use “report cards’’ that show schools and the public at large how schools are performing. The report cards also include data on such other indicators of school quality as attendance and graduation rates.

The goal of such programs is to focus the attention of educators and the public on the results so they will strive to improve them. As a recent report by the Southern Regional Education Board states: “What gets measured gets taught. What gets reported gets taught twice as well.’'

And that is especially true when “high stakes’’ are attached to the tests. As a number of researchers have found, teachers have tended to focus almost exclusively on the narrow skills and knowledge the tests measure. And, as a recent study by researchers at the State University of New York confirms, schools have also attempted to boost their standing on the tests by placing large numbers of students in special-education programs or by retaining them in grade so as not to have to test those students.

Moreover, the published report cards have failed to stir the public to agitate for improved performance as much as their advocates had hoped. Nor have they prompted schools to change their programs, according to Mr. Kirst.

“There is not any evidence that districts really paid attention to these,’' he says.

But Lynn M. Cornett, the SREB’s vice president for state services, says more recent versions of report cards, which include information on each school in a state, are more likely to generate pressure for improvements.

“People think, where change has to occur, they need information,’' she says. “They need it at the school level.’'

Rewards and Penalties

In addition to issuing report cards, a handful of states have also attempted to hold schools accountable for student performance by rewarding schools that perform well and penalizing schools with high levels of failures.

According to a 1991 SREB survey, seven states--including Florida--had school-incentive programs, and, as part of its landmark reform package, Kentucky was scheduled to begin one in 1994.

But the survey found that the results have been mixed. A program in Pennsylvania, for example--which provided funds for schools where reading and math scores increased, dropout rates decreased, or the number of students taking the Scholastic Aptitude Test increased--had little effect on most schools. Teachers and principals in schools that did not receive awards were not familiar with the program.

In addition, in Pennsylvania and other states, budget problems forced states to cut funding for the incentive programs, which decreased their effect.

In Florida, for example, 585 schools shared $10 million in 1989, compared with 343 schools that shared $19.5 million in 1986. The program was scrapped in 1990.

Similarly, programs to provide penalties for failing schools have had, at best, a mixed record.

A study by the Consortium for Policy Research in Education, which examined the effects of efforts in Kentucky and New Jersey to take over school districts that are failing academically, concludes that “takeover programs can have little to do with long-term educational improvement.’'

The report notes that, in Kentucky, which attempted to take over the Floyd and Whitley county districts, the state intervention was focused on complying with state rules, rather than raising student achievement. And in New Jersey, which took over the Jersey City schools, the state emphasized the need to alter the governance and management of the district.

“In our opinion,’' the CPRE report states, “if takeover fails to improve teaching and learning, it is not worth the stigma suffered by school personnel and communities.’'

Others question whether schools should bear the primary brunt of accountability measures, or whether districts and states should be held equally responsible for their roles in providing all students with access to a high-quality education.

A Growing Sense That Top-Down Measures Failed

In Florida, at least, educators contend that the accountability measures failed to do much to improve schools because the state imposed too many mandates to allow schools to educate students effectively.

“The previous accountability systems didn’t work because they were largely top-down,’' says Mr. Fowler, the Sarasota superintendent. “When you’re trying to do quality control on a system of two million children and 110,000 teachers--even if they are only in 67 school districts--you just can’t do that from Tallahassee.’'

Mr. Fowler says the heavy degree of state control was not limited to education, and reflects the rapid growth in the state.

“As the population grew, and communities grew, the legislature stepped into the breach,’' he says. “If anybody had a problem, anywhere in Florida, it ended up on a legislator’s desk.’'

“Education was not excepted from that,’' he continues. “If somebody tells a legislator kids don’t write well, the next thing, there was a bill to improve writing.’'

But the effect of the mandates was to hamstring teachers and administrators, says Ruth Holmes, a Chapter 1 resource teacher from Pensacola and a member of the accountability commission.

“A lot of times, money was for a certain program, and you could only use it for that certain program,’' she says. “If you saw children needing services, you couldn’t use the resources, because they were from a mandate from the state legislature.’'

‘It Hasn’t Worked’

By 1991, the message that Florida’s accountability system had failed had reached the legislature.

“The legislature has mandated and mandated, ordered and ordered school boards, and it hasn’t worked,’' says Rep. Douglas L. (Tim) Jamerson, the chairman of the House education committee.

In response, lawmakers agreed to a completely new system, known as “Blueprint 2000.’'

The legislation establishes seven education goals, modeled after the national education goals. The goals address children’s readiness to start school, the graduation rate, student performance, the learning environment, school safety, teachers and staff, and adult literacy.

The statute also directs each school to conduct a “needs assessment’’ and to create a school-improvement plan to meet the goals. Schools that fail to demonstrate adequate progress after three years of assistance and intervention will be subject to penalties.

To provide greater flexibility to schools, the measure also places more than four dozen statutes in abeyance and waives another 15. Certain laws, however, including those dealing with civil rights, student safety, and exceptional children, are not waived.

To carry out the plan, the legislature also created the commission on accountability and charged it with developing performance standards for each goal area and methods of assessing progress. Last year, the panel came up with 22 such standards, including one that states: “All Florida students graduate from secondary schools with a certificate showing mastery of the student performance standards and outcomes.’'

The 10 student-performance standards range from “Florida students communicate in English and other languages using concepts, prose, symbols, reports, audio and video recordings, speeches, graphic displays, and computer-based programs’’ and “Florida students use numeric operations and concepts to describe, analyze, disaggregate, communicate, and synthesize numeric data, and to identify and solve problems,’' to “Florida students display responsibility, self-esteem, sociability, self-management, integrity, and honesty’’ and “Florida students work cooperatively to successfully complete a project or activity.’'

Philip D. Lewis, a former state Senate president and a member of the accountability commission, says the legislation was born out of “total frustration’’ with the quality of education in Florida.

“The legislature came to the conclusion that something dramatic had to be done,’' he says. “This is dramatic. I don’t even know if they know how dramatic this is.’'

‘Radical Change’

But in Pennsylvania, critics of that state’s outcome-based education program are well aware of how radical it is, and they don’t like it.

There, the state board of education last year adopted a plan that, like Florida’s, scraps high school graduation requirements and instead establishes statewide outcomes for student performance and requires districts to develop their own plans for how they will attain the outcomes.

In January, after some delays and protests, the board adopted 55 learner outcomes that all students must attain in order to graduate from high school. They include: “All students demonstrate knowledge of basic concepts and principles of physical, chemical, biological, and earth sciences’'; “all students develop and defend a position on current issues confronting the United States and other nations’'; and “all students explore and articulate the similarities and differences among various cultures and the history and contributions of diverse cultural groups, including groups to which they belong.’'

But last month, the state House voted overwhelmingly to nullify the rules.

In part, the vote reflected the view that the system is untried and potentially disastrous.

“Those making a radical change have the burden of proof to show it works,’' says Bill Sloan, an aide to State Rep. Ron Gamble, the sponsor of the amendment to kill the rules. “Unless and until the proponents produce research showing it to be effective, I don’t think the state government ought to be doing anything.’'

“If they want to experiment in a couple of places, that’s one thing,’' Mr. Sloan continues. “But don’t mandate an experiment statewide in 501 school districts.’'

‘A Real Shame’

While critics in Pennsylvania questioned the system as a whole, they particularly objected to a number of the outcomes adopted by the state board. To many critics, these outcomes imposed “values’’ on students by measuring their cooperation and tolerance, not simply their reading and writing skills.

“We believe the purpose of the public schools is to teach academics. Period,’' Mr. Sloan says.

Susan H. Fuhrman, the director of the Consortium for Policy Research in Education at Rutgers University, calls the Pennsylvania House action “a real shame.’'

“You shouldn’t throw out the idea because of a disagreement over the outcomes,’' she says. “In a sense, that’s what we’ve been doing for 200 years.’'

Ms. Fuhrman acknowledges that the outcomes are controversial, and lead to “culture wars.’' But, she says, “that’s a healthy discussion. We ought to decide as a society what we want.’'

While states such as Florida and Pennsylvania have pursued broad outcome goals for students that cut across disciplines, other states, such as California, have chosen to define what students should know and be able to do along more traditional subject-area lines. In the coming years, which approach is more productive--as well as the details of what goes into such outcomes--will continue to be the topic of heated discussion.

Outcome Systems Demand New Methods of Assessment

The debates over outcomes in Pennsylvania and elsewhere have also focused attention on another potential problem in developing the new accountability systems: whether states will be able to measure the performance goals they have set.

Unlike traditional testing systems, which tend to measure students’ knowledge of facts, the new outcomes generally demand that students demonstrate their abilities to use their knowledge, as well as other abilities that are seldom tapped by conventional tests. These demands require a new testing technology, which as yet hasn’t been perfected.

“We know how to assess how kids are doing in math,’' Mr. Kirst of Stanford says. “It’s not clear how to assess kids’ management of time or if they work well in groups.’'

Eva L. Baker, the co-director of the National Center for Research on Evaluation, Standards, and Student Testing at the University of California at Los Angeles, says the new forms of assessment raise a number of issues that must be resolved if they are to be used in accountability systems.

One significant problem relates to equity. To demonstrate that they can understand historical events or solve math problems, different students may write different essays or submit a wide variety of materials in a portfolio. But how can states hold schools accountable for vastly different student work that is based on different background knowledge and interests?

“We’re caught in a dilemma when we try to contextualize [assessment] and make things relevant, and, at the same time, make things uniform and comparable,’' Ms. Baker says.

‘Bad Inferences’

Another problem is insuring that the new assessments are technically sound so that valid inferences can be drawn from them.

A recent RAND Corporation study of Vermont’s pioneering portfolio-assessment program shows the technical problems that can arise in shifting to new forms of assessment.

Vermont’s program, the first statewide program to include portfolios, measures 4th and 8th graders’ math and writing performance on a range of criteria, such as students’ writing “voice’’ and their understanding of math tasks.

But in analyzing the results from the first full year of implementation, the RAND researchers found that the “rater reliability’'--the extent to which raters agreed on the quality of a student’s work--was low. They recommended that the state report the results of the portfolio assessment at the state level only, where results were sufficiently reliable.

Ms. Baker notes that, if states report unreliable results, they can draw improper conclusions about the quality of schools--a serious problem if states intend to hold schools accountable for the results.

“When the technical stuff isn’t close, people make bad inferences,’' she says. “They think schools are worse than they are, but maybe [that’s because] the measures are straining to provide information people thought they wanted.’'

“What policymakers want,’' Ms. Baker says, “sometimes they can’t have.’'

Florida officials, for their part, say they continue to want such information.

Ms. Holmes, the Pensacola teacher, says the new assessments would provide a much better gauge of student abilities than conventional tests.

“My students could do science, they could write, they could do math and work out problems,’' she says. “What they couldn’t do was bubble in answers on a test.’'

Mr. Fowler, the Sarasota superintendent, also says that the new measures would help improve teaching by breaking down disciplinary boundaries in schools.

“Real life for students is not organized by academic disciplines,’' he says. “If you are hired by a bank, you are not going to, in the first hour, do math; in the second, economics; and in the third, write reports.’'

Interim Measures

But while states wait for the new measures to be perfected, officials are building the accountability systems by using existing measures that they admit don’t match the goals of the new systems.

When the Florida accountability commission selected the measures that will be used until the new assessments are ready, for example, members debated which test to use as a measure of student achievement--the basic-skills High School Competency Test, which is required for graduation, or a higher-level Grade 10 Achievement Test.

In the end, the panel decided to use both, even though members acknowledge that they do not match the standards in Blueprint 2000.

“Are these perfect measures? No,’' says Michael C. Biance, the executive director of the accountability commission. “That’s why it’s a transition.’'

Mr. Biance notes, though, that the panel kept its eye on the higher standards. As evidence, he points out that it agreed to drop a proposed requirement that schools attain minimum proportions of students passing the competency test on their first attempt.

Although the panel’s motivation was primarily to permit schools greater flexibility, its action also has the advantage of focusing schools’ attention on the long-range goal, rather than the interim, lower standard, Mr. Biance says.

“This gets us away from ‘dumbing down,’ '' he says.

Mr. Kirst of Stanford also dismisses the concern that the interim measures would send a mixed signal to schools.

“In the 70’s, [minimum competency] was driving the whole system,’' he says. “Now, we’ve got the view that there is a difference between minimum competency and what we need to aspire to.’'

Loosening Regulations Also Stirs Controversy

In creating the new accountability systems, states have pledged to loosen regulations on schools to allow them flexibility to meet the standards for performance. But in practice, eliminating regulations has proved almost as contentious as setting outcomes for student learning.

Part of the tension has arisen because lawmakers, accustomed to keeping tabs on the dollars they send to schools, are reluctant to cut the strings attached to those funds.

“We have to let people at the local level earn their way out of that deal,’' Mr. Lewis, the former president of the Florida Senate, says. “Is the legislature going to send $8 million to $10 million to the local level and say, ‘Go forth and do good’? That ain’t gonna happen.’'

Educators are also quick to point to the reasons the rules were adopted in the first place, and are loath to abolish them.

“You can focus more on outcomes, but it’s not necessarily desirable to think about getting rid of the process regulations,’' says Susan H. Fuhrman, the director of CPRE, the policy-research consortium. “There are equity concerns.’'

Adds Pascal D. Forgione Jr., the state superintendent of public instruction in Delaware: “One has to remove them with purpose and direction. We’re not here just to sweep the minefield.’'

The tension over eliminating rules reached a breaking point last year in New Hampshire, where the state board set off an outcry by proposing to abolish its standards for school approval.

Board members argued that, by eliminating numerical caps on class sizes and a mandated ratio of guidance counselors to students, for example, schools could have greater flexibility in providing services to students. Although elementary schools had been free under a previous regulation to petition to waive the regulations, few schools actually took advantage of that freedom.

But the move to abandon standards sparked a furor from education groups. “If you gut the regulations, what do you need a state department of education for?’' asks Mel Myler, the executive director of the National Education Association-New Hampshire. “It’s the only bastion within the state to provide some equality of educational opportunity.’'

In the end, the board agreed to reinstate some numerical requirements, which made the revisions “palatable’’ to groups like the N.E.A. affiliate. Nevertheless, a legislative committee voted to oppose the rule, which takes effect this June.

Tension Palpable

Even in states that have agreed to relax rules, the tension between state control and local flexibility remains palpable.

In South Carolina, for example, where the legislature in 1989 adopted a plan to free high-performing schools from most regulations, the program isn’t reaching all schools that can benefit from it, a study by CPRE found. In fact, the report states, it may be the low-performing schools that most need deregulation.

“Unlike currently eligible schools,’' it points out, “they have not flourished under the prevailing rules.’'

And in Florida, teachers warn that the state standards themselves may restrict local schools. Although schools are theoretically free to come up with their own plans for improvement, the state measures may, in fact, direct what such improvements must address.

“I understand the need to set standards,’' says Brenda Emerson, a teacher of visually impaired students at Princeton Elementary School in Orlando, Fla. “But attaching standards to every goal area defeats the purpose of allowing everyone to create what they need.’'

Will 1,000 Flowers Bloom?

Indeed, simply setting standards and measuring student progress won’t, in itself, insure that schools improve. Schools must also have the wherewithal to bring about changes.

“What if you come up with some innovative things, such as technology, that cost a lot of money?’' Ms. Emerson asks. “If we come up with a great plan, how will we fund it?’'

Many schools, moreover, may be unable even to dream up innovative ideas, Mr. Kirst says. Without the proper professional development that would introduce teachers to new practices and assist them in teaching in new ways, schools will fall back on old methods.

“The assumption is, you let off input regulations, and 1,000 flowers will bloom,’' he says. “Our findings are that 1,000 flowers do not bloom without help.’'

As a result, the new accountability systems may reinforce the inequalities that exist among schools. Those with the capacity to innovate may thrive, while those that lack such a capacity will languish.

Noting such equity concerns, Ms. Baker of U.C.L.A. says, “What worries me is the mentality that tests, pretty much by themselves, will create educational nirvana.’'

Educators Considering Standards for Delivery, Too

To address such problems, some educators have begun to think about a new set of “school delivery standards’’ to accompany the standards for student performance. The additional standards would insure that schools have the capacity to deliver the type of instruction needed to bring all students up to the performance standards.

Such standards also reflect the legal requirement the Florida court imposed in issuing its ruling allowing the use of a test to deny diplomas. The test could only be used, it ruled, if students had been exposed to the instruction the test measured.

In her report, Ms. Darling-Hammond of Teachers College outlines a dozen “standards of practice for learner-centered schools.’' They include: all students should have access to the school funding necessary to fulfill standards of excellence; all students should have access to well-prepared, fully qualified teachers; and all students should have access to a rich and challenging curriculum.

“A focus on outcomes with unequal allocations of resources won’t enable students to achieve the outcomes, or schools to organize themselves to give students a fair shot,’' she says.

In a similar vein, the National Council on Education Standards and Testing, a Congressionally chartered panel that called for national standards and a related system of assessments, also urged that delivery standards be developed to measure schools’ abilities to provide high-quality instruction.

Debate in Congress

But that recommendation sparked a fierce debate in Congress. Although the council had recommended that states could select their own delivery standards from among those developed by groups of states, Democrats insisted on national delivery standards to hold all states equally accountable. But Republican governors and the Bush Administration resisted the national standards, arguing, in part, that a system of school-delivery standards smacked of the same kind of input standards the new accountability systems were trying to get away from.

The debate over delivery standards is likely to recur this year, when Congress again considers the issue of national standards and assessment systems.

But whatever the outcome of the debate, those involved in setting up the new accountability systems see them as perhaps their best chance of improving schools. And they plan to take full advantage of this opportunity.

“We are going to have one chance to do this,’' Ms. Holmes, the Pensacola teacher, says. “If we don’t [succeed], we’ll see more and more mandates. We can’t fail.’'