Newnan, Ga.
Second grader Brooks Rudnik hunches over a screen, guiding a purple, sunglass-wearing character through an imaginary world called “Keenville.” His mission: help save the planet’s animals, in part by showing off his knowledge of phonics. As he answers questions correctly, virtual jelly beans pile up in a corner of his monitor.
Brooks doesn’t realize it, but he’s taking a test.
Brooks and his classmates here at Newnan Crossing Elementary School are early adopters of the state’s newly developed game-based assessment system for 1st and 2nd graders, a group that tends to get antsy about pen-and-paper tests. Georgia hopes the game—which is also known as Keenville—will eventually act as a formative assessment, giving teachers a real-time picture of how well their students understand math and reading skills like grouping, graphing, and reading comprehension.
Keenville, which was only available in many schools at the beginning of this school year, is still a work in progress. The Georgia education department has released only about a third of the games that will eventually be available. And although the games currently spit out some data for teachers on how their students performed, Newnan’s teachers are anxious to get their hands on more-sophisticated student-data “dashboards” that will give more detailed information on student performance on a range of skills and standards.
All Brooks is interested in, though, are those jelly beans, the game’s digital reward for correct answers. Students can exchange them for accessories for their avatar in the game—called a “Keen"—or to buy accessories for the Keen’s house. Possibilities include a purple lava lamp, an electric guitar, or a fluffy couch.
Brooks’ teacher, Brandi Cook, sees a lot of potential in Keenville.
“They love Keenville. They think decorating those houses and their [character] is so much fun and they have no idea that those problems they’re solving are really assessing them at all. That’s amazing, to have them actively engaged in taking a test and them not know it.”
Brooks, she said, told her that he thought there would be “lots of girls around” if he did well in the game and created a snazzy house for his avatar.
‘Something That Would Be Ours’
The Keenville test grew out of recently passed legislation which called for Peach State districts to develop a system of formative assessments for K-5. After getting feedback from district leaders, Georgia’s education department decided to create a game-based test to help districts meet that requirement for the early grades. The state collaborated with the Georgia Center for Assessment at the University of Georgia and FableVision Studios on the project. Districts that opt not to participate in Keenville must use another formative assessment with their 1st and 2nd graders.
So far, about 200 schools are using the games across 76 districts, roughly half the districts in the state. More districts and schools will likely jump on board as more games are released, said Jan Reyes, the director of assessment development for the state education department.
Right now, there are 10 online assessment games that are part of the Keenville suite available to schools. The state’s vision is to eventually have 31 games by next fall, including 16 for English/language arts and 15 for math.
There are plenty of off-the-shelf options for game-based assessments. Yet Georgia chose to build its own in part to avoid costly annual licensing fees, making the game more financially viable over the long term.
“We wanted to do something that would be ours and we would own it and we could sustain it over time,” Reyes said. She hopes the game can also be more easily customizable and responsive to teachers’ needs.
Eventually, Keenville will incorporate games set in make-believe regions that look like different parts of Georgia—mountains, a coastal area, a big metropolis, and even a swamp modeled on the state’s signature Okefenokee. The idea is for students from all over the state to see their own community reflected in the game, Reyes said.
One Keenville game—based at a carnival—helps students tell time. Another takes place in a cave where students can unlock a “treasure chest” by reading nonfiction passages on subjects like “air” and then answering questions. Another, called “Peachling Cafe,” asks students to figure out how much food the planet’s animals need, in part by using a counting strategy.
“They’re solving real-world problems,” Cook, the teacher, said.
That’s by design, said Scot Osterweil, the research director of Comparative Media Studies at the Massachusetts Institute of Technology, who is consulting with Georgia on the design of Keenville.
“What a good game does, in general, is give you an interesting challenge and give you fairly open-ended ways of solving that challenge,” Osterweil said. “What we try to do with all these standards is come up with what are the interesting ways in which kids interact with these ideas and can you give the kid an interesting way of working through this stuff, not just here’s a math ... question and did you get it right or wrong.”
‘All About Engagement’
Interest in game-based assessment is “definitely burgeoning,” said Valerie Shute, an education professor at Florida State University who has done extensive research on formative assessment. But she hasn’t heard of another state attempting a game-based approach at this scale.
She said there are still “tons and tons of unanswered questions” about game-based tests like Keenville, but she’s personally a believer. “It’s all about engagement. Right now, when people take tests, it’s so boring and tedious; it’s a turnoff for a lot of people,” Shute said.
But she said she would encourage Georgia to run checks to make sure the test can gauge students’ skills accurately. “For any assessment to unfold on such a large scale, we need to ensure that validation studies are conducted to make sure that the assessments are measuring what they are intended to measure,” she said. (The state plans to do validation studies, said Meghan Frick, a spokeswoman for Georgia’s education department.)
To be most useful, the test should give teachers a clear picture of how their students perform on specific skills and an understanding of why they missed certain questions. That would be “something that teachers can grab and go with,” Shute said.
For now, the state is in the early stages of implementation, still figuring out where the kinks in the program are and how to grapple with them. The state began piloting Keenville last spring and added additional schools this fall. Although teachers can already get some information about how many questions their students answered correctly, they don’t yet have a detailed picture of their students’ performance.
That will change later this school year, when Georgia is set to release more-detailed dashboards that will give teachers a better idea of how their students are mastering the math and literacy skills the game measures, Reyes, the state director, said. The state also plans to roll out broader dashboards for school and district leaders.
Eventually, each student will receive both an overall measure of his or her reading comprehension that’s used in part to help teachers find books that will be understandable but challenging for the child, and an overall rating of math content knowledge.
At this point, though, Newnan teachers haven’t gotten much training on how to use Keenville as a formative assessment, in part because the state is waiting for the more-detailed dashboards to become available.
“Admittedly, it is very basic right now because we didn’t want to hold up the release of the games until we had everything that we wanted in the dashboard,” Reyes said. “We didn’t do explicit dashboard training because we didn’t have all the features, so we wanted to wait until we had more to train on.”
Georgia’s districts have broad leeway in incorporating the game into their classroom instruction. For now, Cook and Tonya Copeland, another 2nd grade teacher at Newnan, said they have used Keenville to reinforce skills they teach in class and to adjust instruction a bit.
Down the road, they’d like to assign students a game instead of using a more traditional test to check their mastery of a skill.
“Instead of giving them the benchmark or a milestone test or something, I could be giving them a formative assessment that way, with whatever standard I assign them,” Cook said. “You would know what to reteach and who to put in what group.”
If Keenville is successful, the state may consider expanding the game-based assessments to other subjects and grade levels.
“We see it as a model for where we might want to go with some other things,” said Allison Timberlake, the state’s deputy superintendent for assessment and accountability. “We could eventually think of some really neat things we can do for upper grades and science. A traditional assessment model may not be able to fully measure what they are capable of.”
