| Statistics can be a powerful tool for those who use them well. |
Alan Greenspan and the Federal Reserve Board meet regularly to analyze the economy, pore through the latest data, and consider adjusting interest rates in response to a range of shifting performance indicators. Although the Fed hasn’t always received the plaudits for its actions it is now generally enjoying, no one in the financial community would rest easy if the managers of our national monetary policy were not regularly sifting through the most reliable data they could amass. Contrast this behavior with what typically transpires in our schools, where buckets of data are collected but rarely receive close inspection by teachers and administrators for clues to what is going right and what is amiss in our classrooms.
Like school data, data on the economy are far from perfect. Managing the economy doesn’t readily yield to cookbook recipes, and neither does managing schools or classrooms. Nevertheless, Mr. Greenspan and his colleagues pay attention to the best data they can get their hands on, seeking to understand how the system is functioning.
Schools should do no less, but such habits are not often found in our schools--how ironic for institutions that are supposed to foster inquiry and inquisitiveness and teach self-criticism, problem-solving, and the fundamentals of mathematics, including data analysis.
Of course, there are good explanations for schools’ aversion to data about operations and performance. Educators have often been bludgeoned with the data they are periodically forced by others to collect. They also have seen data, taken out of context, lead to ill-conceived policy decisions. Equally significant, collecting data is time consuming and arduous, particularly when critical information is not readily available or is expensive to collect. And, its final use is not always clear. As Albert Einstein once noted: “Not everything that counts can be counted. And not everything that can be counted, counts.”
Consequently, it is not surprising that educators do not usually rush to quantify critical aspects of schooling, nor do they particularly trust those who do or the results of such efforts. This suspicion is natural and is probably quite healthy, as education data often imply a degree of certainty that is not well deserved. However, ignoring such information and blinding oneself to the underlying truths that are often captured by quantitative measures is shortsighted.
Precisely because the mysteries and complexities of education often mask both positive and negative occurrences in schools, quantitative analyses can be a powerful tool. If applied well, such analytical work can alert us to unexpected triumphs and to unforeseen problems. It can tell us if new initiatives are having their desired effects, are provoking a set of unintended consequences, or both. We can learn whether implementation of promising ideas is proceeding as planned or is being compromised by unexpected changes in the school district. And, as we seek to educate our children in a manner that emphasizes both excellence and equity, we can develop measures that assure we do no harm.
| Schools willing to critically examine their core operations will increase the odds that their students will succeed. |
|
At a time when it is popular to distrust data and adopt the view that statistics can be made to support any stance, we believe that ignoring the hard realities that education statistics often present does a tremendous disservice to our children and our communities. At a time when education is widely understood to be key to a healthy and prosperous future, those who lead our schools and are responsible for educating the nation’s young people, as well as the public at large, need the best tools available to gauge accurately how well schools are functioning and how well our children are learning.
How can school communities systematically marshal valid, reliable, and trustworthy data? How can they use data to think clearly about how they might most effectively dedicate their knowledge, skills, and energies to educating young people?
They can begin with the presumption that much of the data needed to make this happen already reside nearby in school and district offices, in teachers’ own records, and in readily accessible public databanks. Such resources are frequently overlooked and undervalued because they have been created to satisfy someone else’s agenda. Schools regularly receive orders to produce what seem like great mounds of data, be it in the form of student tests, attendance records, or budgets. However, while school people left to their own devices might not choose to devote their scarce time to such pursuits, this information, when organized well and joined with other useful data, can often be turned to a school’s advantage.
School transcripts, for example, are typically used more by college admissions offices than school officials. Through careful examination, these everyday records can help educators understand whether students are generally following and succeeding in a coherent sequence of challenging academic coursework or are wandering aimlessly through a smorgasbord of electives or soft courses.
School grades, regularly sent to parents and postsecondary institutions, can be an immediate indicator of how well schools and students are doing. Typically, educators know little about trends in grading or of the grade-distribution patterns across their school. Today, grades are generally not based on specific standards about what constitutes highly accomplished work--rather, they are determined by independent teacher judgments. There is generally no discussion about whether grading policies and practices are sensible, constructive, or based on particular skills and competencies. Presently, about one-quarter of all high school grades are D’s and F’s. While this may be an accurate reflection of student performance, there is little evidence available to reassure us that such is the case.
Similarly, school leaders are often in the dark about attendance patterns. They may know that the daily attendance rate for their school is 92.7 percent, the national average. This sounds respectable--an A grade. But once confronted with the fact that this means that students, on average, are missing 13 days of school each year, the complacency associated with an A usually evaporates. School people start asking questions that can lead to constructive action, such as: What days of the week and times of year does the rate go up or down? What percentage of students have absentee rates double the average or more? Who are these students?
|
| Sound data analysis can “bring statistics to life” and help educators have a richer appreciation of how well schools are satisfying their missions. |
To address such matters and related issues that surround the implementation of new curricular initiatives, we have developed a comprehensive, practical process for school leaders to use to establish a performance-indicator system. It includes:
- Establishing goals. Local education goals must be meaningful, realistic, complementary, measurable, and reflect clear priorities.
- Identifying related outcomes, inputs, and practices. Many indicator systems focus solely on measuring outcomes, but our research on school performance indicates that schools that also examine data on instructional practices, teacher education, student demographics, and other key factors will learn a great deal more.
- Defining the data that will provide information on those factors. Schools not only can make judicious use of data they collect for others, but also can fashion a powerful information system by augmenting these data with other sources of their own design that are well aligned with their values, philosophy, and objectives.
- Examining and interpreting the data. One does not have to be a statistician or a computer whiz to analyze data effectively. Developing experience with a small repertoire of statistical tools can go a long way.
- Setting performance targets. Targets are often set when managers and policymakers establish goals. But our research has shown that this often results in the establishment of arbitrary targets that lack meaning to those who must achieve them.
- Monitoring performance over time. Creating an effective performance-indicator system helps schools interpret trends and decide whether to continue, expand, or modify their programs.
These steps are common sense and build on research on the use of performance measures and standards for accountability.
Schools willing to critically examine their core operations will increase the odds that their students will succeed. These schools will also discover a new and effective means of communicating in a very powerful way with parents, others in the system, and the broader community. In fact, just sharing information about how a school functions is such an unusual act that it will often yield a measure of goodwill that is invaluable to efforts to test new curricular and instructional approaches.
Internally, the availability of such information can lead to a new and healthy conversation among the faculty--one that promotes reflection on practice, healthy skepticism about trendy ideas, and a school culture that values professional knowledge and expertise and finds ways to channel and use it to yield the greatest good for the greatest number.
Sound data analysis can “bring statistics to life” and help both educators and the general public have a richer appreciation of how well schools are satisfying their missions. The news will not always be good, but bad news is better than ignorance.
Gary Hoachlander is the president of the Berkeley, Calif.-based think tank mpr Associates Inc. Karen Levesque is the project director for mpr‘s book, At Your Fingertips: Using Everyday Data To Improve Schools, and David R. Mandel is the director of its Center for Curriculum and Professional Development, based in Washington.
