Back in 2009, Joel Rose and his team used spreadsheets to manually produce customized “playlists” for 70 New York City middle school students each day. The goal was to always give each child the best possible content, delivered in the optimal manner, at the best possible time.
Figuring out how to make that happen took 11 hours every night.
Now, Rose’s New York-based nonprofit organization, New Classrooms, performs those same functions each day for 11,000 students in 38 district, charter, and independent schools spread across 10 states and the District of Columbia.
Thanks to complex algorithms, the process is now complete by 4:30 p.m. each day.
“The basic concept of using data to match each student with the best possible lesson every day has remained consistent,” Rose, a former middle school math teacher, said. “But on the tech side, we’ve made quantum leaps.”
Belief in the potential of “curriculum playlists” is central to the K-12 personalized-learning movement. The premise is that in any given math classroom, for example, some students will need help with basic arithmetic, while others should be practicing solving equations. And some students might need to work with a teacher, while others might be better off practicing online. Even the best teachers can’t consistently differentiate at that level for 30 students every single day. Technology can help.
The concept has led to both new opportunities and new challenges for schools.
Scores of publishers and educational technology companies now purport to use algorithms to deliver personalized content to students. As a result, educators looking for new digital classroom resources must not only sift though a growing menu of curricular options, but also determine how customizable each lesson truly is and how well vendors actually match students with the right material.
The stakes are even higher with groups such as New Classrooms, which does not operate its own schools, but sells an instructional model known as Teach to One. Along with California charter network Summit Public Schools and the Bay Area startup AltSchool, New Classrooms takes a more comprehensive approach to curriculum playlists, treating them as a catalyst for rethinking the basic organization of school.
The Teach to One model has garnered considerable attention, including plaudits from Microsoft founder and philanthropist Bill Gates and the U.S. Department of Education.
But some have found that implementing a playlist-based instructional model is harder—and more expensive—than it looks: On top of facilities renovations and technology upgrades, yearly software licenses cost $225 per student, and schools typically pay New Classrooms between $90,000 and $160,000 per year for other supports. More than a quarter of the 53 schools that have started with Teach to One: Math are no longer using the program.
Research on Teach to One’s impact, meanwhile, is mixed. One major study found no significant effects on achievement, with many students saying they felt they learned more when they worked directly with a teacher in a more traditional manner.
That combination of hype and uncertainty has led some observers to voice concern that algorithm-driven playlists are just another technology that K-12 schools are embracing without adequate scrutiny or regard for possible unintended consequences.
“People don’t really demand evidence when technology and algorithms are involved, because they’re bewildering,” said Cathy O’Neil, a data scientist and activist and the author of Weapons of Math Destruction.
“But you don’t need to understand what’s in the black box to know if something works,” O’Neil said.
Some innovators want to use playlists to link young people to their passions, to each other, and to opportunities to make an impact in the world.
Curriculum playlists are often thought of as algorithm-driven systems for matching students with content and learning experiences that can best help them master prescribed learning standards as efficiently as possible.
But proponents of the “connected learning” movement are taking playlists in a different direction: They want to connect young people to their passions, to each other, and to opportunities to make a real impact on the world.
“My concern about the current approach to algorithms and personalized learning in schools is that we’re still so tied to traditional notions of curriculum and what content needs to be taught,” said Constance Yowell, the CEO of Collective Shift, a nonprofit organization spun off by the MacArthur Foundation in 2015.
“If you put four healthy foods in front of a young person, but they hate all of them, which one they choose is not all that interesting,” Yowell said.
Collective Shift’s biggest project is LRNG, a platform intended to provide students with tailored opportunities to pursue their own interests through a wide variety of often-informal learning experiences, ranging from museum trips to wearable-technology projects to online coding tutorials.
It’s still early days for the project. Some of the challenges Yowell and her team are wrestling with:
Who gets to design the playlists?
Right now, the answer is mostly adults and experts at various groups that LRNG partners with, including a new set of grantees financed to develop playlists on such topics as digital literacy and career preparation. But eventually, Yowell said, the goal is for young people to be able to create and share their own playlists.
How do you make playlists matter?
The connected-learning movement is heavily focused on out-of-school and informal-learning opportunities. Proponents are generally not interested in measuring effectiveness with standardized-test results. So what exactly do young people get out of completing an LRNG playlist? Right now, the answer is mostly digital badges, with some additional real-world opportunities, such as an internship with a local partner organization, sprinkled in.
How do you get young people to access and use the playlist platform?
Currently, it happens mostly through mentors, such as teachers or program staff at a local Boys & Girls Club. A mobile app is on the horizon. But even after LRNG manages to draw young people in, questions remain. One biggie: how to strike the right balance between recommending learning experiences to users versus letting them discover such experiences for themselves?
LRNG is still in beta, and there’s a long way to go.
But Yowell said the vision is clear.
“We want LRNG to be an infrastructure that enables in-school and out-of-school learning to work together,” she said.
Remaking the Classroom
Peek into a 6th grade math classroom at Nathan Hale Middle School in Norwalk, Conn., and you might see as many as 67 children in a single room, each pursuing his or her own Teach to One playlist.
Some might be conducting a “live investigation,” working with a teacher to solve a problem. Many will be using Chromebooks, working independently through exercises that algorithms have selected specifically for them. Part of the room will look almost like a traditional classroom, with another teacher leading students through a group lesson.
“When I first heard about Teach to One, I was taken aback. I didn’t see how it could work,” said Albert Sackey, the school’s third-year principal. “But when I saw it in action, I thought it was incredible.”
Undoubtedly, switching to a playlist-driven instructional model has been a major lift at Hale. Sheetrock had to be torn down, so that multiple classrooms could be converted into one large learning space. The school’s wifi network had to be upgraded. Sackey and his team had to rework Hale’s entire bell schedule—teachers in the Teach to One program can’t plan ahead, because each day’s lessons are driven by algorithmic recommendations created the previous evening.
Still, eight months into the transition, Sackey said the upheaval has been worth it.
“Good teaching is always good teaching,” he said. “Teach to One is just a tool that helps teachers be more targeted.”
Personalizing Learning
Here’s how the technology works.
At the beginning of the school year, each student in the New Classrooms network takes a series of diagnostic assessments. Based on the results, and with the help of technology, the group then maps out the specific skills and concepts that each student needs to learn over the year. For 2016-17, that meant 11,000 different skills maps for 11,000 different students.
From there, New Classrooms’ algorithms begin identifying the optimum learning sequence for each student, so that the skills he or she encounters in the playlist will build on each other in a natural, coherent way.
Then the algorithms start identifying the specific lessons or curricular content that will ostensibly best help each student master the next skill they encounter. New Classrooms staff members have vetted and approved roughly 9,000 lessons from more than two-dozen content partners.
During extended class periods that last 80 to 90 minutes, students work through the material their playlist serves up to them. Three students sitting side by side in the “independent learning zone” might work on three entirely different skills, using three different problems, from three separate vendors.
At the end of each day, each student completes a short quiz known as an “exit ticket.” The results go to New Classrooms, where the algorithms go to work.
What skills did students master? Are they struggling with solving equations because they don’t understand variables or because they don’t know how to multiply? Would the gaps in each student’s understanding be best addressed via a live investigation or with some solo practice using the free online math website Khan Academy? What do each student’s past experiences say about what type of content will be most engaging now?
After figuring out what each student needs, the algorithms then work to figure out what’s most feasible for the whole classroom. Maybe there’s not a critical mass of students who need help with dividing fractions, so a small-group lesson isn’t an option.
The algorithms’ final daily output consists of a “preview schedule” that fits all the pieces of the puzzle together. New Classrooms’ human scheduling team reviews those algorithmic recommendations, makes tweaks as necessary, and sends each school suggestions on how its classrooms should be organized the following day.
Teachers on the ground get final say—what if the algorithms want to put two feuding students into the same small group?
When students walk back into the classroom the next day, their playlists have already been updated, and large flat-screen TVs point them to the section of the room where they’ll spend the period.
Algorithmic Bias?
Teach to One has come a long way, but there’s still plenty of room to grow, said Rose, the New Classrooms co-founder and CEO.
On the technology side, other companies, such as New York City-based Knewton, are already leveraging the power of machine learning to help make their own playlist-creation systems smarter.
Logistical challenges also remain. When the District of Columbia school system tried Teach to One at one of its middle schools, for example, experienced teachers ended up feeling limited by the technology, because they wanted the freedom to plan more than one day ahead.
“When the rubber met the road, there were just so many little practical challenges to making it work,” said John Rice, the district’s director of educational technology, who helped make the decision to drop the program last year.
Then there are the growing number of questions about the role algorithms play in society more broadly.
In fields such as criminal justice, concerns have been raised about “algorithmic bias.” Formulas used to help determine criminal sentences, for example, have been found to produce unfair outcomes for African-American defendants.
One fear for K-12 schools is that as the algorithms behind playlists become “smarter” and more efficient, students with special needs—say, dyslexia—might find that their atypical mental processes are not reflected in the software that shapes their instruction.
“Algorithms are inherently optimized for some people,” said O’Neil, the author and activist.
For his part, Rose turns the question around: Students with learning disabilities are often poorly served in traditional classrooms, he said, and internal metrics show the 600 or so special-needs students in Teach to One classrooms have made significant academic growth.
But for the time being, at least, New Classrooms does not have any built-in checks to ensure that its technology is free of bias.
Figuring out how to ensure public trust by making that happen is just one of the hurdles the group will have to overcome if it wants to take its curriculum playlists from the margins to the mainstream.
“Our long-term vision is we’d love to see personalized learning be the way kids learn,” Rose said. “And we’d like to be one of the models.”
