Teacher-candidates who passed a performance-based licensing test on their first try tended to boost their students’ reading test scores more in their first year of teaching than those who didn’t, according to new research released this afternoon.
While that finding is good news for supporters of the Teacher Performance Assessment, or edTPA as it’s known in the field, the rest of the study is a bit of a mixed bag. In contrast to the reading findings, passing the exam didn’t bear any relationship to students’ math scores. It’s also less clear whether incremental improvements on the exam—such as improving one’s score by three or four points—translate into student learning gains.
“This is a study where middle-ground findings make it harder to interpret,” said Dan Goldhaber, one of three researchers who conducted the study for the National Center for Analysis of Longitudinal Data in Education Research.
Studying the edTPA
The findings are part of the first major independent research study on the edTPA.
The exam differs from most other licensing tests because it hinges on a demonstration of a teacher-candidate’s classroom instruction, rather than a bunch of multiple-choice questions, as is the case with other popular teacher-testing series. Some 18,000 teacher-candidates took the edTPA in 2014.
There have been a few smaller-scale studies on edTPA and on its precursor. But as a footnote in the new study notes, one of them isn’t published and the scholars who conducted the other, including Stanford University’s Linda Darling-Hammond, who was instrumental in creating the edTPA, say they no longer have their data.
It’s unclear how the new findings might shape edTPA policy moving forward. Some 13 states now use or are planning to use it for licensing or to gauge the quality of preparation programs.
The CALDER study takes a stab at the important question of “predictive validity"—whether candidates who achieve a certain score on the edTPA help their students learn more than those who don’t.
The researchers looked at teacher-candidates in Washington state. In all, they examined scores from some 2,300 teacher-candidates who took the exam in 2013-14, looking at their first submission. Then they examined a subset of 280 of these teachers who taught reading and math in grades 4-8 in 2014-15, using a “value added” methodology to gauge their impact on student performance. (Candidates did not have to actually pass in order to teach until January 2014.)
They found a significant association between candidates who achieved the state cutoff score—35 out of 75 in most fields—and students’ test scores in reading. Washington is set to increase its cutoff score to 40, and the reading findings hold up at that cutoff score, too, though they are not as strong. (States that require the edTPA each set their own scoring standards, so it’s easier to pass in some states than others.)
But in math, there was no consistent relationship between teachers’ who had passing scores on the edTPA and student test-score gains. (In econometric studies such as this, researchers usually use a variety of different controls or specifications to see if they hold up across multiple ones, a way of improving confidence in the results.)
Score Increases Over Time?
It’s unclear why the link seems to show up only in reading, Goldhaber said.
“It falls into the realm of speculation, but I think some of what edTPA is picking up is your ability to communicate, either in written form or orally. And those are skills sets that may be more important to teaching reading,” he said.
It’s a reasonable hypothesis because the exam measures candidates’ grasp of discipline-specific academic language used in instruction.
Because looking at edTPA as a “pass/fail” mechanism provided only limited insight into the exam, the researchers also looked at whether there was a “continuous” relationship between edTPA scores and student achievement. In other words, as candidates’ scores go up, do their students do better?
This is an important question because groups like the American Association of Colleges for Teacher Education have billed the exam not just as a hurdle for entering the profession, but as a developmental tool that can be used by preparation programs to help improve their curriculum and to help candidates progress towards mastery of teaching skills. And there are other potential such uses: Districts might want to use edTPA scores as one element in hiring decisions.
But the study found that the results were decidedly mixed in this connection. There was no association between edTPA score distribution and students’ reading scores. In math, there was only modest evidence that a higher score meant better teaching.
Teacher-Policy Implications
The findings are likely to be closely analyzed, in part because the exam has proved to be controversial.
Although it was designed by Darling-Hammond—one of the country’s most influential teacher-educators—and her team at the Stanford Center for Assessment, Learning, and Equity, some teacher-educators say the edTPA takes away their responsibility to determine when someone is ready to lead a classroom. Others question whether it’s too easy to cheat, or at $300 a pop, too expensive for teacher-candidates.
Ray Pecheone, the executive director of SCALE, noted that value-added estimates can be unstable based on which years of data are used and which tests supply the underlying data. But he praised the study overall.
“I find the results, while mixed, encouraging,” he said.
He would like to see future research look at the link between edTPA scores and teachers’ evaluations. (That was not possible in this instance because Washington state does not collect individual teachers’ observation scores.) And he would like to see whether the results look different once teachers are tracked over time.
“I think we do want to take a closer look and hope we can do more longitudinal work,” he said. “The first year of teaching is really a struggle for most teachers and they’re just trying to get their sea legs on becoming effective teachers, and it takes certainly more than a year for them to really show powerful results, so I’d love to see this study continued over multiple years.”
See also:
Save
Save