It’s a question that can stump even the most conscientious and skilled teachers: Why isn’t students’ achievement improving?
Teachers study the state academic standards, learn as much as they can about the state tests for their grade levels, and get grounded in the district curriculum. Then they carefully craft lesson plans to cover the topics and skills their students will be expected to know.
In too many classrooms, though, achievement levels off, and some students continue to fail. After giving their best, many teachers conclude that other factors are undermining their efforts.
“They say, ‘You gave me the wrong class,’ or ‘I don’t get enough support from my principal,’ or ‘My teaching partners aren’t working together,’ ” said Stan Hill, the assistant superintendent for instruction in the 47,000-student Winston-Salem, N.C., district. “They think they are doing all the right things.”
Districts like Winston-Salem have turned to research to peer into classrooms and help teachers carefully examine their practices. Using research tools to analyze what is taught and map how closely instruction follows the expectations outlined in state academic guidelines and what is tested each year, teachers can step back and scrutinize the effectiveness of their work.
“It is like being able to hold a mirror up to someone” to help them view their own practice, said Mr. Hill. “When teachers see what they are doing, it is often not what they think they are doing.”
Standards-Based Instruction
Throughout the development of national and state academic standards in the 1990s, debates raged over what children should know and be able to do at each stage of their academic careers. But as difficult as it was coming to consensus on the content of the standards, and the assessments that have followed, many who struggled through that process knew the real test of the standards movement would be in changing what teachers did behind closed doors.
A decade into the push for rigorous academic standards, many districts are still working to align curriculum and teaching with those expectations.
“Classroom teachers are the ultimate arbiters of what is taught, and how,” Andrew C. Porter wrote in a 2003 research brief for the Wisconsin Center for Education Research, at the University of Wisconsin-Madison, where, until this academic year, he served as director.
Mr. Porter, now a researcher at Vanderbilt University in Nashville, Tenn., and his colleagues have been producing and refining tools for analyzing course content for more than two decades. Now, through projects directed by the Council of Chief State School Officers—involving 11 states, the districts in Winston-Salem and Charlotte-Mecklenburg, N.C., Chicago, Miami- Dade County, and Philadelphia, and a federal education laboratory—that research is being used to fine-tune state programs of standards and assessments.
Other organizations, including Achieve and the Council on Basic Education, both in Washington, are also working with states and districts to enhance standards-based instruction using other alignment models.
“Regardless of what a state policy requires or what a district curriculum spells out,” Mr. Porter said in an interview, “the classroom teacher ultimately decides how much time to allocate to particular school subjects, what topics to cover, when and in what order, to what standards of achievement, and to which students.”
Through his research and the help of UW colleague John Smithson, Mr. Porter came up with a template for quantifying the content topics and cognitive demands in standards, assessments, and local curricula for mathematics and science, and a way to illustrate that information. A similar matrix for reading/language arts is due out this month.
First, standards and assessment items are coded to measure how much emphasis is given to specific topics—in math, for example, topics include inequalities, operations on polynomials, and quadratic equations—and to types of tasks required, from memorizing to performing procedures to communicating understanding.
Using the same language to describe content, teachers complete detailed surveys that outline how much time and emphasis they dedicated to those topics and tasks over the course of a school year. Instructional materials and professional-development sessions are also analyzed to gauge subject matter covered in class.
And, in some districts, including Winston-Salem, administrators or researchers observe classrooms to check the validity of the teacher reports. Maps and graphs of instructional content are then devised that show where it overlaps with and diverges from standards and assessments. Statistical analyses are then computed to determine levels of alignment.
Inadequate Alignment
Mr. Porter has used those tools in his research into the rigor of high school courses in the 1990s, and in a national evaluation of the federal Dwight D. Eisenhower Professional Development Program for math and science teachers.
The 11 states and five urban districts participating in the CCSSO alignment projects generally found that instruction is not adequately aligned with state assessments and standards, according to Rolf K. Blank, the director of the Washington-based CCSSO’s Education Indicators Program.
Those data—as well as professional development and technical assistance from the CCSSO have enabled district administrators and teachers to re-evaluate the local curriculum and change instruction as necessary, according to Mr. Blank.
“The schools that really use the data the most are going to see some change in moving toward alignment,” he said. The state chiefs’ council is planning to study the effect of the alignment data on instruction and student achievement.
“The payoff will be in looking at the teachers who have aligned instruction to see if achievement is higher,” Mr. Blank added.
Having clear indicators of the match between actual content and state or district expectations of what students should be taught is the first step toward improving curriculum and teaching, Mr. Hill of the Winston-Salem district says.
While the districts have yet to quantify the changes in instruction undertaken as a result of the alignment analyses, or to attribute achievement gains to changes that have been made, they are making some of the requisite first steps toward meeting those goals.
“This gives us three legs of the stool to stand on,” Mr. Hill said. They are, he said, “what teachers say and perceive they are doing, how children are performing, and then, what someone observes them doing.”
Critical Measures
In Ohio, officials are using the tools as a yardstick of how well standards-based measures have taken hold.
In trying to think about school reform and school improvement,” said Mitchell Chester, an assistant state superintendent there, “we felt there’s nothing more critical than being able to get a handle on how teachers are spending their time, what they’re emphasizing in terms of curriculum, what they weren’t emphasizing, and the kinds of understandings and expectations they have of students.”
After piloting the project in Cleveland, state officials hope the resulting data will spur teachers to reflect more deeply on their work, and might shed light on whether students have equal access to learning opportunities.
“We are so heavily invested in an outcomes-driven system, where we measure school success, or lack thereof, based on test scores,” Mr. Chester said. “It’s critical to have some intermediate measures of things we know have a strong relationship to eventual achievement.”
For Mr. Porter, the scholar at Vanderbilt, the alignment tools can help answer questions about academic content that cannot be gleaned from test data.
“What gets taught is the strongest single predictor of gains in student achievement,” he said. “We developed this model as a research tool, but now it can be used to improve practice.”
