Teacher-Quality Group to Revamp Education School Review

The organizer of a plan to review all 1,400 schools of education and publish the findings in U.S. News and World Report is altering several key aspects of the study’s methodology—an attempt, officials of the group say, to respond to mounting complaints.

The National Council of Teacher Quality posted its grading criteria online Feb. 9 to address concerns among education school deans that the review wouldn’t be transparent or accurate. It also plans to supplement the content-based analysis at the heart of its methodology with information on candidate classroom performance culled from “value added” data.

Finally, the organization will not, as previously planned, issue a failing grade to institutions that don’t provide the information requested.

Much about the ambitious study, due out next year, is in flux—including whether such changes will be enough to persuade schools of education to participate.

The situation underscores the tensions brewing as conversations about teacher accountability that proliferate in school districts to establish new teacher-evaluation systems gradually permeate higher education.

“I think it’s good that they are addressing some of these methodological issues,” said Mary Brabeck, the dean of the Steinhardt School of Education at New York University. Ms. Brabeck, along with dozens of other deans, signed letters protesting the study.

“Nobody’s against rankings, nobody’s against evaluation, nobody’s even against high-stakes evaluations,” she said. “But if the methodology is flawed, how does that serve the public?”

NCTQ’s president indicated that the changes were meant to encourage participation, but she was adamant that the study continue.

“These are folks for whom this is their life’s work. ... We totally understand why people are inclined to get upset about this,” Kate Walsh of the Washington-based advocacy institution said. “But, on the other hand, we’re asking folks to put that to the side and recognize what we all recognize, that there are many institutions in the U.S. not preparing teachers adequately, in addition to many doing a great job.”

Robert Morse, the director of data research for U.S. News and World Report, confirmed that the outlet will continue the review.

“We’re used to people having negative opinions about what U.S. News does,” he said.

Review Details

The two organizations officially launched the project Jan. 18, but it has been in the works for nearly a year. Its main purpose, according to the NCTQ, is to provide school districts and other education consumers with more information about teacher-candidates’ training.

The report’s methodology is largely based on pilot reviews conducted last year of education schools in Illinois and Texas, and as such, it differs markedly from that used in other U.S. News projects.

Rather than rankings, the council will rate all programs on an A-to-F scale on up to 17 standards. They include whether teachers are adequately trained in the science of reading and math content; the length and coherence of student-teaching experiences; and whether their programs include training on working with English-language learners.

After the council sent initial requests to education schools in late January, two distinct groups of deans organized and publicly objected to the review in letters to Brian Kelly, the editor of U.S. News.

One was signed by about 35 education school deans belonging to institutions within the Association of American Universities, an organization of elite research institutions.

A smaller group of education school deans at institutions that have received teacher-training grants from the Spencer Foundation sent another complaint. Among other complaints, the deans alleged incomplete standards, a flawed research methodology, and scoring criteria that were not transparent.

The AAU deans compared NCTQ’s standards unfavorably with teacher standards crafted by the National Council for Accreditation of Teacher Education and the Council of Chief State School Officers.

Both groups of deans were galled at the NCTQ’s plan to give institutions an F on certain standards if reviewers could not obtain information on which to base a rating.

“We worry that this implied coercion will cast doubt on the results of the entire evaluation,” the AAU deans wrote in the Feb. 3 letter.

Ms. Walsh said that the council will address some of the criticism. Instead of issuing failing ratings, it will give an “incomplete” grade while seeking the information through other means. Ultimately, it will base its rating on the information available, as per U.S. News policy.

In addition, she said, the council has now placed its scoring criteria on its website so it is accessible to all. “We’re making those changes because we want institutions to feel they can trust this process. It will work so much better,” Ms. Walsh said.

She defended her group’s standards, saying the alternatives presented in the deans’ letters were too vague to serve as guidelines.

“Previous reviews [in Texas and Illinois] have shown high rates of failure against these standards, and that creates nervousness.”

Bad Blood?

In a Feb. 9 webinar, NCTQ representatives said that, once the ratings were computed, the council would show how its scorers arrived at their determinations and allow schools to provide additional evidence to contest them, possibly leading to a ratings change.

Such details would be publicly posted on the council’s website.

Previous NCTQ reports have been poorly received by education schools, perhaps explaining why their appeals have been directed at U.S. News and not the NCTQ.

A 2006 study of reading preparation in a sample of teacher colleges, for instance, led to complaints that the reviews were flawed or incomplete. (“Teacher Ed. Faulted on Reading Preparation,” June 7, 2006.)

Officials at the American Association of Colleges for Teacher Education, a national membership group based in Washington, have expressed concerns about the review to their members, referring to the council’s “questionable methodologies.”

AACTE President Sharon P. Robinson said the changes were overall a “very gratifying” response to members’ concerns. Still, some of the indicators, especially the ones on content training, could prove difficult to respond to because of education schools’ varying approaches to delivering such courses, she cautioned.

Not all education schools have condemned the study. The NCTQ’s technical advisory panel includes some from inside education schools. One member, the dean of Southern Methodist University’s education school, in Dallas, said he didn’t agree with all the standards and results in his school’s review last year, but the content analysis provided some useful information overall.

“I believe there should be other ways to evaluate the effectiveness of our programs as well, but that doesn’t negate NCTQ’s efforts to review programs for their content,” David J. Chard wrote in an e-mail.

As a result of the review, Mr. Chard added, SMU will put more emphasis on math-content instruction and special education.

Some of the deans’ objections are also rooted in the tricky issue of how to balance the value of the content of education school programming with measures of what teacher-candidates have learned and can do. The AAU institutions contended in their letter that the review was too focused on teacher-preparation “inputs,” rather than the classroom effectiveness of teachers coming out of the programs. It suggested several alternative methodologies.

Ms. Walsh said the NCTQ will now also examine information generated from value-added systems where available. Florida, Louisiana, and Tennessee have already begun to examine their education schools based on such systems, which track newly placed teachers and gauge their ability to raise their students’ achievement scores.

The AAU letter also suggested that the council consider a project by institutions in more than a dozen states to devise a performance-based licensing test. (“State Group Piloting Teacher Prelicensing Exam,” Sept. 1, 2010.)

But Ms. Walsh said that pass rates on the California Performance Assessment for Teachers, on which the new test is based, are too high to provide useful information for distinguishing program quality.

Tough Sell

Education school deans acknowledged that their protests are coming at a time in which public perception of teacher education’s strengths is tenuous.

“I fear that if anyone protests or refuses to participate in the review, they are sending a message that they are not responsive to the public,” said Mr. Chard of Southern Methodist University.

Deborah L. Ball, the dean of the University of Michigan’s education school, said in a statement that protesting the study would distract from conversations about how to improve the quality of teacher training.

“Claiming over and over that we know what we are doing and that we should control training looks foolish to our critics and, in the face of weak or nonexistent evidence, only discredits our claim to expertise,” she said.

Many deans are in the position of Richard De Lisi, the dean of the education school at Rutgers University, in New Jersey, who is still weighing whether to take part in the review.

“It’s unfortunate that this can be painted as shying away from evaluation,” Mr. De Lisi said. “But that’s not what it’s really about. You have to have judgment criteria that are clear and evidence everyone can see.”