Defining Teacher-Prep Accountability

Much has been written and discussed of late about the debate over the best method of assessing teacher-preparation programs. As the dean of the school of education at Indiana University Bloomington, I understand that meaningful assessment of teacher preparation requires a multifaceted approach based on a robust research methodology and focused on program outcomes. A sound study, as researchers know, begins with a viable research question. The design and method of data collection then flow from that question. Moreover, the scientific validity of conclusions reached on the basis of the data depends on the ethical application of research principles and, where appropriate, validation of results through peer review and replication.

For more than 10 years, I have steered the public-reporting results of our university alumni follow-up studies and surveys of the principals who hire our graduates. Our accreditation self-studies and accreditation decisions are also posted routinely on the school of education’s website for all to see. In addition, we systematically and proudly report on the performance of our graduates where such information is available. For instance, we publicly celebrate the fact that the last four Indiana teachers of the year and seven of the last 10 are Indiana University Bloomington graduates. These educators are selected annually through a peer-review process based on nominations from superintendents and vetted by the Indiana Department of Education.

While it is true that I have openly criticized the research methodology of the National Council on Teacher Quality on the issue of teacher-preparation rankings, I want to emphasize, at the outset, that the decision of our faculty at the school of education at Indiana University Bloomington not to participate in the NCTQ studies has nothing to do with fear of accountability or concern about how our programs might be ranked.

In fact, the first national NCTQ teacher-preparation study in 2013, in which our school participated as a result of a public-records request, ranked our university’s secondary education program among the top 10 percent of such programs nationally, earning it a place on NCTQ’s “honor roll.” Upon learning of the results, however, I issued a statement categorically rejecting the rankings because I thought they would be misleading. I am of the opinion that the NCTQ methodology is fatally flawed, but that is a discussion for another day.

The National Academy of Education recently proposed a set of principles for making decisions about teacher-preparation program evaluation. At the center of the proposal was the principle of validity, meaning “the requirement that an evaluation system’s success in conveying defensible conclusions about a teacherpreparation program should be the primary criterion for assessing its quality.” As the national academy’s report points out, finding imperfections in evaluation methods is easier than fixing them. But that’s no excuse for making indefensible claims based on faulty methods.

As the report makes clear, evaluation is a complex process requiring careful alignment between the methods used and the purpose of the evaluation, be it for or ensuring accountability, providing information to consumers, or enabling program improvement. If the purpose of an evaluation is accountability, then the National Academy of Education insists that data “need to be collected scrupulously and interpreted rigorously, according to defined and accepted professional standards.” In other words, a high-quality assessment system is necessary to draw valid conclusions about program effectiveness.

If I were to design a study to hold preparation programs accountable for their graduates’ performance, as the group Teach Plus Indianapolis has challenged me to do, I would start with the question of whether a given teacher-preparation program produces graduates who can work effectively in school classrooms to increase student learning and achieve other valued educational outcomes. Then, I would select or create appropriate measures of student learning and related educational outcomes, as well as ways to assess teacher effectiveness on the impact of those measures.

Although the research community is making progress on assessment of both student learning and teacher performance, significant scientific challenges remain in creating valid and reliable measures that can control extraneous influences and minimize random error. Nevertheless, it is now possible to utilize results of student test scores, expert observation of teaching, teacher self-reflection, student-teacher evaluations, work products of students and teachers, and other indicators to address specific questions on teacher effectiveness. As the reliability and validity of such assessment systems improve, it should be possible to link aggregated results to the programs that prepare teachers.

Some states, such as Louisiana and Tennessee, are experimenting with the application of these types of assessment systems to teacher-preparation accountability. Early results of these studies have been published in peer-reviewed journals and are being replicated or otherwise examined by the research community.

Currently in Indiana, legislation is moving through the General Assembly to create a teacher-preparation accountability system that, among other things, would link required annual teacher evaluations to the programs that prepare teachers. Various deans of education, members of our faculties, and I have testified in favor of the legislation. In addition to the aforementioned link of district-level teacher evaluations to teacher preparation which include evidence of measurable gains in student test scores, one attractive feature of the legislation is that it recognizes the role of the standards of the Council for the Accreditation of Educator Preparation in teacher-preparation accountability. The CAEP standards focus on selection criteria for applicants to teacher-preparation programs; clinical experiences in schools; and measurable improvement against desired student outcomes. The council’s methods include review of course syllabi, observations of student-teaching experiences, measures of growth in student achievement, results of surveys of program graduates and their employers, and other information collected through both examination of documents and program visits.

In a Sept. 18, 2013, Education Week Commentary, “Why the New Teacher Ed. Standards Matter,” Mary Brabeck and Christopher Koch pointed out that the CAEP standards were not created “by policymakers in Washington, but by a broad cross section of stakeholders, including university and P-12 officials, representatives of nontraditional programs, chief state school officers, critics, union officials, and others who put aside their differences to create new expectations for the field.” There’s really no need for me to take the lead in developing a framework that holds preparation programs accountable for their graduates’ performance. The field has already done that through the development of the CAEP standards and accreditation procedures.

Teach Plus Indianapolis issued a challenge to me that I’ve tried to address here. Now, I’d like to present Teach Plus Indianapolis with my own challenge: Insist that any studies you endorse or accountability measures you propose are based on valid data. At a minimum, that means using an appropriate methodology for the research question under investigation and, where warranted, credible Institutional Review Board approval, peer-reviewed publication of results, and replication. It does not suffice to call for a tough conversation about teacher preparation if it is driven by ideology and devoid of meaningful data to inform decisions.

The education and scientific community has a duty to provide a critical review of the evaluation methods used and insist that conclusions reached as a result of the “conversation” on teacher education accountability are based on valid data. Otherwise, it’s just empty rhetoric.