Keeping Track

From the day children enroll in a Florida public school to the moment they start a career, the state is keeping tabs on them.

In mammoth sets of computerized data collected by several government agencies, the Florida Department of Education knows what courses they take, what grades they get, and the scores they post on state and national tests. It knows who taught them, where those teachers attended college, and how they scored on state certification tests.

For the students who stay in the state—and the vast majority of them do—Florida also knows where they work or go to college, and the grades they earn there, too.

And all of that electronically stored information is a crucial ingredient in setting educational policies and validating instructional practices in the state.

“Data should drive policy,” Gov. Jeb Bush says. “In Florida, we’re obsessed with that.”

Educators say the state’s obsession has paid off in their own ability to make decisions, whether it’s to hold students back in 3rd grade or enroll them in Advanced Placement classes in high school. And researchers say the computerized data system helps them determine whether the state’s policy decisions have been effective.

“Florida is just about the best data system that I have seen,” says Jay P. Greene, an associate professor of education reform at the University of Arkansas at Fayetteville and a senior fellow of the Manhattan Institute, a conservative think tank based in New York City. While many states collect a lot of the information Florida does, few assemble it in a centralized data system and ensure its accuracy in the way that the Sunshine State does, he adds.

What’s more, Florida leaders have figured out how to use the data to influence policy to an extent that few other states have, says Chrys Dougherty, the research director of the National Center for Educational Accountability. That nonprofit research group, based in Austin, Texas, advocates improvement in the quality of K-12 data.

“Other states are not taking the initiative and are sitting on their data,” Dougherty says. “There’s a mind-set in [Florida] of: ‘We’re going to go out and use this data to improve schools.’ ”

20-Year Effort

The Florida data system has been more than 20 years in the making. In the 1980s, in an effort to evaluate vocational and training programs, the state created a computerized database to track its high school graduates as they entered the workforce and enrolled in college.

Today, the state’s massive computerized data warehouse includes information on more than 10 million current and former students and almost 1 million current and former school staff members, dating back to the 1995-96 school year. It has 366 tables with more than 5,000 columns of data in them. The data are stored on a network server that holds 1.5 terabytes of data—equivalent to 15 personal computers with 100 gigabytes of memory.

Since 1998, the central component of the state’s data warehouse has been the scores on the Florida Comprehensive Assessment Test, or FCAT. Since 2002, the FCAT has been given to all students in grades 3-10 in reading and mathematics, as well as those in grades 4, 8, and 10 in writing.

With such a wealth of numbers, the state education department produces reports to inform policymakers’ decisions for the state as a whole.

And by the end the year, state officials say, they will provide teachers with a computerized tool that will help them individualize instruction for students, particularly those who are performing below grade level.

Every year, the department publishes a performance report for all of the legislature’s 40 senators and 120 representatives, detailing the test scores, school budgets, graduation rates, and other data for the schools and districts in each lawmaker’s legislative district.

The agency also produces performance reports for every high school and community college in the state. In addition to FCAT scores and graduation rates, the reports include the percentage of students taking college-entrance tests, the percentage of students completing Algebra 1 before graduating, and the percentages of students taking Advanced Placement, International Baccalaureate, or college courses. It compares each school’s data against the statewide and districtwide averages.

Making Decisions

In addition to evaluating schools, the state scours the data before making major policy decisions.

In 2003, the state started retaining students in the 3rd grade if they failed to score at grade level on the FCAT reading assessment. The policy included an exception that allowed children who rated below grade level to be promoted on the basis of teacher recommendations and other factors. About 40 percent of the 3rd graders who failed to score at or above grade level were promoted based on teacher recommendations.

Before instituting the retention policy, the state also started tutoring and other out-of-school learning experiences for K-3 pupils considered at risk of failing the 3rd grade reading test.

By tracking test-score data from the early grades, schools were able to identify youngsters who needed the extra help and prepare them to advance from 3rd grade to 4th grade, says John Winn, the state education commissioner.

Florida officials see signs that such interventions helped prevent retention. In 2001, Winn says, 40 percent of African-American 3rd graders and 29 percent of Hispanic 3rd graders scored at level 1 on the FCAT, the lowest of five performance levels. By 2004, those proportions fell to 26 percent of African-Americans and 18 percent of Hispanics.

Children who repeated 3rd grade also showed signs of success once they reached 4th grade, Winn says. Of the students who were held back in 3rd grade in 2002-03 because they scored at the lowest performance category on the FCAT, 41 percent scored at level 3—which represents grade level—in reading once they completed 4th grade two years later. That’s more than twice the passing rate of pupils who weren’t retained even though they scored at level 1 in 3rd grade.

“We have absolutely no doubt that the interventions that are occurring are helping students,” Winn says.

Still, some say the state doesn’t always use its data fairly. A case in point is a plan the state board of education approved this year to give bonuses to up to 10 percent of teachers, based on the progress their students made on the FCAT.

“It’s a one-day snapshot,” says Mark Pudlow, a spokesman for the 129,000-member Florida Education Association, an affiliate of the National Education Association and the American Federation of Teachers. “You might be able to use data if everybody agreed that that measure made sense in judging how a teacher does his or her job and how students learn. But I’m not sure the FCAT can do all of those things.”

Student Adviser

Another use for Florida’s electronic database is to help in giving students career and college counseling. Through a Web-based software program, Florida Academic Counseling and Tracking for Students, which is online at www.facts.org, students can access all of their academic records and map out the courses they’ll need to take to meet educational or career goals.

Whether students want to be doctors or auto mechanics, the Web site will tell them the high school courses they need to take to qualify for appropriate colleges or trade schools. The site also provides information on public and private universities and recommends potential sources for financial aid. Students can even apply to Florida colleges via the site.

“It’s literally a personal advising tool,” says Hanna Skandera, the state’s deputy commissioner for accountability, research, and measurement. “The reason we want these data systems is so students are empowered to make decisions.”

School officials use similar data to make decisions about student coursetaking. In the 188,000-student Hillsborough County public schools, for example, students are now assigned to Advanced Placement classes based on FCAT and other test scores, says Michael A. Grego, the assistant superintendent for curriculum and instruction for the Tampa-based district. Previously, AP assignments were based on teachers’ recommendations, an imprecise measure.

“I know if that student can be successful,” Grego says of his reviews of student test-score data. “I have data to show that the student can be successful.”

‘Connections’ Coming

Even though the state has a trove of academic and other data on every student, teachers and principals must do lots of work to put it into a form in which they can use it.

At Ernest E. Just Elementary School—one of three elementary schools in Hillsborough County to score an F on the state’s accountability report card in 2004-05—first-year Principal Tricia McManus individually reviews FCAT scores with students. On her computer printout of the test results, she has used a pen to mark up pupils’ scores and subscores. The marks are reminders to emphasize points in the one-on-one conferences she holds with every student in the predominantly African-American school.

In the 2004-05 school year, only 16 percent of Just Elementary School’s 3rd graders scored at or above grade level in reading. The same percentage ranked at or above grade level in math. For 5th grade, those percentages were 15 percent in reading and 9 percent in math.

With her printouts, McManus can point to specific reading skills—such as distinguishing facts from opinion or understanding cause and effect—that a child is struggling to master. With that knowledge, she and her teachers can track a student’s progress on such subcategories throughout the school year by administering a set of district-developed assessments that it calls “mini-FCATs,” which students can take on computer or with paper and pencil.

While McManus finds that the best way to use the data is to study her paper reports, she also will enter those numbers into spreadsheets for further analysis. She’s looking forward to using a data tool that the district is planning to have ready by the end of the 2005-06 school year that will help her create individualized plans to address every student’s weakness.

“Some of the tedious stuff won’t be as tedious for us,” she says.

The state will soon make its own data-analysis tool available. By this summer, the state education department will unveil Sunshine Connections, a Web site designed to equip principals and teachers to perform much of the analysis McManus now does on paper.

With Sunshine Connections, teachers will be able to review individual students’ scores and create plans for addressing their particular needs. The Web site will even refer teachers to specific lesson plans that target a student’s poorest-performing areas, the state says on a Web site describing the project.

Teachers won’t have to do their own research and analysis, cumbersome work many can’t do until the school year starts, says Winn, the education commissioner. “You can individualize instruction from day one,” he says. All teachers in Florida are due to have access to Sunshine Connections by this coming fall.

While Florida appears to be ahead of most states in how it amasses and uses data, it doesn’t see its data collection as complete. Even as the state prepares for the launch of Sunshine Connections, the education department has a contract with the Microsoft Corp., the vendor creating the Web tool, to keep refining and improving the tool for four additional years.

State officials say that other parts of the data system also will continue to be updated. “It’s never complete,” says Skandera, the deputy commissioner for accountability. “There’s always something more to be done.”