Special Report
Artificial Intelligence Quiz

Who’s the Tougher Grader: ChatGPT or Teachers? Quiz Yourself to See

By Sarah Schwartz & Hyon-Young Kim — February 14, 2025 1 min read
Photo collage of man working on tablet computer.
  • Save to favorites
  • Print

Grading “assistants” that use generative artificial intelligence offer teachers a tantalizing promise: You can spend less time poring over hundreds of student essays while still giving kids the thoughtful feedback they need to make their writing better.

But how do these tools actually stack up against experienced educators? Education Week wanted to find out.

We asked two teachers to grade the same piece of writing—a short persuasive essay from a 7th grader in Missouri—against a set of criteria for organization, clarity, and mechanics. Then, we gave the same instructions to ChatGPT, the generative AI chatbot developed by OpenAI.

See also

Photo collage of woman working on laptop computer.
Education Week + Getty
Artificial Intelligence Is It Ethical to Use AI to Grade?
Sarah Schwartz, February 14, 2025
9 min read

The teachers—Heather Van Otterloo, a middle school English teacher in Joplin, Mo., and Chad Hemmelgarn, a high school English teacher in Bexley, Ohio—have both used AI tools to help them give feedback on student work in their classes. Even so, both say that they’re always involved in the process, reviewing AI’s comments or making the final decision about how to score a piece of writing.

This student writing sample, from one of Van Otterloo’s middle schoolers, is from an assignment she might run through her AI grading platform, she said.

The prompt—“Should people ever wear pajamas in public? Why?”—asks students to defend their position on this issue as a formative measure of their ability to take a stance and support it with evidence. Students were directed to make a claim, provide reasons and evidence, offer a counterexample, and include a conclusion. They received the set of criteria that Van Otterloo would use to evaluate their papers and some general guidance on writing for organization and clarity.

Education Week provided all this information to Hemmelgarn, and to ChatGPT. All three respondents’ answers are below. Take the quiz to see if you can tell whose response is whose—and whether AI can match teachers’ insights.

Student writing sample

Pajamas should never be worn in public. First of all, they are not a good representation that you would want to be seen as in public. Pajamas state a message that you didn't put in effort to make yourself presentable or simply just don't care. Pajamas are seen by many as something that should only be worn to sleep in, so wearing them in public may have a negative affect on your reputation.


Second of all, pajamas are not made to be worn out in public anyways. They are made to be slept in and not to be worn out of the house. If they are not used for their original purpose then there isn't a point in having them. If all you have is pajamas in your wardrobe then you will not have anything to wear to particularly serious events, like a nice dinner or a program.


Some may say that pajamas are acceptable in public because they are comfortable and relaxing, however there are brands that make comfortable outfits that are still more presentable to be worn out of the house, such as Lululemon and Vuori.


In conclusion, pajamas should never be worn in public because they do not give off a positive representation of yourself and are not made for that purpose but to specifically be worn to bed.

Scroll down to see Rubrics A, B, and C. Click the tabs to view each graded rubric.

Meets Expectations Approaching Expectations Needs Improvement Unsatisfactory
Claim (Thesis Statement) X
Reasons (Explanation of Opinion) X
Support with Examples X
Counter-Argument X
Conclusion X
Organization and Clarity X
Grammar and Mechanics X
Meets Expectations Approaching Expectations Needs Improvement Unsatisfactory
Claim (Thesis Statement) The claim is clearly stated at the beginning of the response and is easy to identify. The stance is strong and unambiguous. The claim is stated, but may not be as clear or obvious. The stance is present but might need further emphasis. The claim is somewhat unclear, or the stance is not fully defined. The claim is missing or unclear, and the stance is unidentifiable.
Reasons (Explanation of Opinion) Provides multiple well-developed reasons supporting the claim. The reasoning is clear, thoughtful, and strongly related to the topic. Provides at least one solid reason, but might lack depth or fully developed explanation. Provides a reason that is somewhat vague or underdeveloped. Lacks sufficient reasoning or explanation.
Support with Examples Includes at least two specific examples that directly support the reasons given, such as personal experiences, facts, or cultural references. Examples are strong. Includes one strong example or two examples that are somewhat relevant but could be expanded upon. Includes vague or generic examples that don't strongly support the reasons. Lacks examples or uses examples that don't support the reasons.
Counter-Argument Acknowledges the opposing view thoughtfully, with a clear explanation of why the writer's stance is stronger. Acknowledges the opposing view but does not fully explain why the writer's stance is stronger. Mentions the opposing view, but the response is weak or unclear. Does not acknowledge the opposing view.
Conclusion Provides a clear, thoughtful summary that reinforces the claim and reasons. Includes a final reflection that ties the argument together. Summarizes the claim and reasons, but the conclusion may lack depth or new insight. Includes a weak or generic conclusion. The summary of reasons is unclear. Conclusion is missing or does not adequately summarize the points made.
Organization and Clarity The essay is well-organized, with clear paragraphs that each support a specific point. Transitions between ideas are smooth, and the essay is easy to follow. Organization is clear, but some paragraphs may lack clear focus or transitions may be awkward. The essay lacks organization or clear transitions, making it harder to follow. The essay is disorganized, making it difficult to understand the argument.
Grammar and Mechanics Few or no errors in spelling, grammar, punctuation, and sentence structure. A few minor errors in grammar or punctuation, but they do not distract from the content. Some errors that may make the essay harder to read or understand. Frequent errors in gramma or mechanics that interfere with readability.

Related Tags:

Coverage of education technology is supported in part by a grant from the Siegel Family Endowment, at www.siegelendowment.org. Education Week retains sole editorial control over the content of this coverage.

Events

School & District Management Webinar Fostering Productive Relationships Between Principals and Teachers
Strong principal-teacher relationships = happier teachers & thriving schools. Join our webinar for practical strategies.
Jobs Virtual Career Fair for Teachers and K-12 Staff
Find teaching jobs and K-12 education jubs at the EdWeek Top School Jobs virtual career fair.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Artificial Intelligence Webinar
Promoting Integrity and AI Readiness in High Schools
Learn how to update school academic integrity guidelines and prepare students for the age of AI.
Content provided by Turnitin

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Artificial Intelligence More Teens Than You Think Have Been 'Deepfake' Targets
A growing number of teenagers know someone who has been the target of AI-generated pornographic images or videos.
4 min read
A photograph of a 13-year-old girl using her smartphone in a dark room. The content she is browsing from a social media feed projects over her face and on the wall behind her and shows a partial view of a pillow and mattress.
E+
Artificial Intelligence Why Teachers Should Talk to Students Before Accusing Them of Using AI to Cheat
Software showing how a student interacted with a document can shine a light on AI cheating, but it shouldn't be the final word, experts say.
3 min read
A student returns a tablet computer to a charging cabinet.
A student returns a tablet computer to a charging cabinet.
Allison Shelley for All4Ed
Artificial Intelligence How These Schools Are Getting Parents on Board With AI
Schools should give parents a primer on how AI is used in education, emphasizing its strengths and weaknesses.
1 min read
The school principal addresses parents during a monthly meeting.
The school principal addresses parents during a monthly meeting.
Allison Shelley for All4Ed
Artificial Intelligence How District Leaders Use AI to Save Time, Help Teachers, and More
Here are 20 ways district leaders are using AI to help them do their jobs.
1 min read
Business people supporting and giving the priority to the robot.
iStock/Getty