Three Teachers, One Test Question: Will Their Responses Differ?

A common argument in favor of project-based and portfolio assessments is that even high-quality standardized tests do not evaluate students' work as do teachers who work with the students.

To get a feel for how much teacher grades might differ, Assistant Editor Sarah D. Sparks asked teachers from district, charter, and independent middle schools to share their perspectives on real student responses to open-ended questions from the 8th grade National Assessment of Educational Progress. Cossondra George, a middle school teacher of math, language arts, history/social studies, and special education at Newberry Middle and High School in Newberry, Michigan; Jonathan Gold, a middle school history teacher and academic team leader at the Moses Brown School, an independent school in Providence, R.I.; and Ariel Sacks, a teacher of grades 7-9 English/language arts in the Renaissance Charter School in Jackson Heights, N.Y., took up the challenge. (Sacks also writes opinion essays for edweek.org.)

Special Report: Projects, Portfolios, and Performance Assessments

---

Student Response 1

“The vast open space would bring settlers who wanted to own land. The peaceful lake would bring settlers who wanted peace and quiet. The train would bring settlers who wanted an easier way to travel.”

Student Response 2

“the open land
the transportation
and the colony seemed good.”

Teachers' Takes

Cossondra George: “Question 1 is an example of a question I might actually use with my own classes. I would be looking for well-thought-out responses with reasons behind their answer choices.

Student 1’s responses would have been very close to what I was looking for. Not only did they give three reasonable answers but backed up those answers with logical reasons for choosing them. The answers were not as in-depth as I would have liked for an average 8th grade student to have answered. For example, the answer about the lake being peaceful does not indicate the student has thought beyond the obvious for reasons settlers would have chosen this location. On a 1-4 scale with 4 being perfect, I would score Student 1 a 3.

Student 2’s answers offer no reasoning behind the choices and are superficial. On a 1-4 scale, with 4 being perfect, I would score Student 2 a 2/2.5.”

Jonathan Gold: "I would also use this question, probably more as an in-class journal prompt than an assessment. I would also rate the first response higher for using more 'historical imagination' to demonstrate that the student was thinking about what people at the time would have been thinking about, which we talk about often as part of the goal of learning how to think like a historian. I would be looking for responses that ground their logic in more historical reasoning, so I would agree with Cossandra about the inadequacy of the 'lake' point, as well as the lack of quality in the second response."

Ariel Sacks: "As an English teacher, I'm always considering both the content and the style demonstrated in student writing, especially on formal assessments. I do use a scoring rubric for student writing at various points throughout the year, but the practice I find more helpful is descriptive, rather than evaluative. I have students look at examples from their classmates and discuss 'what's working' and 'what needs work' for each one. We work to create a supportive, nonjudgmental environment as we build the practice as a whole class, then in small groups and pairs. I like to give this same kind of descriptive feedback on student writing directly on their papers and in one-on-one conferences. I also give students many chances to apply the process to their own writing as they set goals for revisions or for skills they want to work on in future assignments.

What's working for Student 1: I appreciate the complete sentences and the clarity of the student's response. It shows an understanding of the question and the painting, and some of the history surrounding it. It's helpful that the student not only named the items in the painting, but also gave reasons as to why these would cause settlers to move there. Land ownership and the use of the train both seem historically accurate to me.

What needs work: This would be stronger if the student structured it as a formal paragraph. It could begin with a topic sentence that 'echoes' the question, making it clear what the paragraph is about: "This painting of a town on the Western frontier would entice people to want to settle there in several ways," for example. In a paragraph, the student could offer more in the way of describing these items, which the prompt is asking for. What was alluring or exciting about a train for settlers, for example? Finally, I am wondering about the second item—the peaceful lake for settlers wanting 'peace and quiet.' I don't recall this being a significant motivation for settlers, nor is it something that they couldn't find in Eastern landscapes. This should either be supported better or replaced with another item.

For Student 2, what's working: This response shows a basic understanding of what the question is asking and what the painting depicts. 'Open land' and 'transportation' both seem to be historically accurate. What needs work: This would be stronger if the student explained why open land and transportation would be attractive to settlers. The third item, 'the colony seemed good,' is too vague to be convincing. What in the painting suggests that the colony would seem good to settlers? Good in what way? Describe! Finally, I would like to see this response written in paragraph form. Unless the prompt explicitly asks for a list, I would expect 8th grade students to use the writing tools they have acquired to construct the strongest response they can on a formal assessment."

---

Student Response 1

"The United States sent soldiers to Germany because of World War II. Japan had attacked the U.S. in Pearl Harbor and got us involved in the war because they were Germany's allies. Also, we had got angry that the Germans were taking over, so the U.S. sent soldiers to germany to stop them.

The United States sent soldiers to Iraq to stop the terrorist Sadam Husane. We had believed that he had nuclear weapons hidden in his country, so the U.S. soldiers went there to stop him. Now, soldiers are still in Iraq so they can help its citizens become a democracy."

Student Response 2

"In Iraq we sent people to fight because we thought they had weapons of mass destruction

We sent soldiers to Germany because they attacked us first"

Teachers' Takes

Cossondra: "Question 2 is a broad question with many possible choices of answers. While both students correctly identified two logical, correct-answer choices, again, Student 1 was more in-depth with their explanation of why those countries were chosen by them. From the responses, I would feel fairly confident Student 1 had an understanding of why the U.S. actually went to war against both countries. On a 1-4 scale with 4 being perfect, I would score Student 1 a 3.5. Student 2, while correct, did not give more than just a 'memorized' answer. I would expect a more in-depth explanation from a student in my class. On a 1-4 scale with 4 being perfect, I would score Student 1 a 2/2.5."

Jonathan: "This question is probably a bit too open-ended (my course is more unit- and region-driven). But in evaluating the responses, I would score Student 1 higher for using more historical information and more accurate information. I would also look for more specifics in both responses, and I would be looking to connect the prompt to a broader reflection in U.S. foreign policy.

I use a variety of assessment strategies, but both of these questions would fit naturally on the kinds of quizzes I give. I'd be looking for students to make a clearer connection between the claim and evidence they were using, as well as to emulate historians' writing. I'd probably have a rubric that would break out the content and style, evaluating how well students understand the facts of the situation and how well they articulate those conclusions in historians' language."

Ariel: "For the first response, here's what's working: The student shows an understanding of the question and identifies two places the U.S. has sent troops to fight a war. The student writes in complete sentences and separates the two examples with indentation. The student accurately connects Germany with WWII and knows that a central issue was Germany trying to take over. The student adds the detail of the bombing of Pearl Harbor which 'got us involved in the war,' which helps the explanation. The student explains that the U.S. 'believed' Iraq had weapons of mass destruction, which gestures at the complexity of the situation. The student adds that troops are still in Iraq to 'help its citizens become a democracy.' The chronology in this second example makes it easy to understand. Although there are some minor grammatical errors, they don't impede understanding. The sentences read clearly and the student uses commas correctly.

But what needs work: In the first example, it's not very clear what Germany was up to in 'taking over,' and why it was a problem. Taking over where? What else was going on? Why did the U.S. get angry about it? Mentioning Adolf Hitler and/or Nazis would add a key layer to this explanation. In the second example, it would add a level of complexity to include that later it was discovered there were actually no weapons of mass destruction, and that this caused criticism among U.S. citizens of the Bush administration's motivations in attacking Iraq.

In the second student response, the student accurately identifies two countries where U.S. troops were sent to fight, accurately states a reason for sending troops to Iraq, and separates each reason with an indentation. What needs work: The reason mentioned for sending troops to Germany, 'they attacked us first,' sounds inaccurate to me. At the very least, it needs explanation. Both examples could use more explanation, since the prompt asks for that. I would like to see a paragraph for each example, with more detail, rather than a single sentence. Both sentences need end punctuation! Review comma usage with a conjunction!"

---