Push for ‘Learner Profiles’ Stymied by Barriers

For many proponents of personalized learning, the notion of a digital “learner profile” is something of a Holy Grail—tantalizingly powerful to imagine, but stubbornly difficult to actually find.

Just ask Jose P. Ferreira, the CEO of Knewton, a New York City-based company that is currently amassing hundreds of millions of bits of data on the learning histories of about 2 million American schoolchildren.

The goal is to generate ever-more comprehensive portraits of each student’s strengths, weaknesses, and preferences in order to provide them with customized academic content.

But to date, efforts by Knewton and other vendors to build a truly holistic portrait of any one child have been stymied. The long list of barriers includes technical hurdles, turf battles among educational publishers, lack of clear demand from schools, concerns about data privacy, and criticism that their work is based on a fundamental misunderstanding of how children learn.

“One day the world’s going to wake up and say, ‘Well, of course I’ve got a learner profile that’s hosted in the cloud and goes from book to book and school to school and country to country and gets smarter and smarter the more I use it,” Mr. Ferreira said. “We know that’s obviously going to happen. But it’s not obvious to anybody outside of Knewton. So we’ve got our work cut out for us.”

Harvesting Student Data

Although far from a mainstream idea, the concept of digital learner profiles has been around for years. Its clearest roots are in the health-care sector, where electronic patient records have in recent years become the norm.

The basic idea is to harvest all the data students generate while interacting with the dizzying range of software and apps now found in many classrooms in the United States—every minute spent on a given problem, every answer to every question, even every click and hover—then store them in a single place.

From there, the data can be analyzed for signs that students have mastered a given standard or concept, as well as for evidence of the optimal ways in which each student learns.

In theory, the resulting profile can then be used by teachers to target instruction based on individual students’ needs and preferences. Software can also analyze the information in order to recommend the specific piece of content in the exact format at the precise time that algorithms determine is most likely to help an individual student master the specific concepts at hand. Some evidence suggests that such analyses may yield new and unexpected insights into how different topics are best taught.

“It’s the type of information that parents and teachers want to know, and frankly, should know,” said John Bailey, the executive director of the Tallahassee, Fla.-based nonprofit Digital Learning Now, who co-authored the 2012 report “Data Backpacks: Portable Records & Learner Profiles.”

Online adaptive tests, now in widespread use, rely on similar principles, but on the much smaller scale of an individual assessment.

Likewise, private-sector developers of educational software such as Amplify, Dreambox, and Khan Academy now use proprietary personalized learning platforms to generate robust profiles of individual students, but those portraits are generally confined to a particular course or suite of content.

And makers of K-12 learning management systems are increasingly trying to integrate the student-learning data generated from a variety of sources, but progress has been halting, and most LMS software lacks the sophisticated predictive algorithms that actually put all that data to work.

So for now, the ideal learner profile—a digital dossier that stitches together everything a student does across multiple textbooks, inside school and out, over the course of multiple years, and feeds software that uses the data to make recommendations for individual students—lives mostly in the imaginations of those who believe the concept has the power to transform education, for better or worse.

Few companies are searching as aggressively as Knewton for a way to make the vision a reality.

“If we have not just a semester’s worth of your math [learning] history, but every semester, and we can tie that to your science history, we can tell you things like, ‘You’re struggling in physics because of your math skills,’ ” Mr. Ferreira said. “In the whole history of education up until now, those have been invisible failures.”

Critiquing the Vision

But for Yong Zhao, such grand pronouncements would mostly be annoying, if they weren’t so dangerous.

The professor and presidential chair in the college of education at the University of Oregon, in Eugene, is among those who believe that the notions underlying Knewton’s work—that the bulk of what students should learn in school amounts to a series of discrete concepts; that mastery of those concepts is best “acquired” in a linear, orderly fashion; and that algorithms can analyze digital data to predict the optimal path for students to follow—are, at best, unsupported by evidence.

“Children have feelings. They’re social beings. And they learn best by interacting with others,” Mr. Zhao said. “By putting too much blind faith in this data, we risk treating learning as a mechanical process and not a human endeavor.”

The potential costs, he said, include inaccurate or misguided judgments about children, a lack of appreciation for how mistakes and mental detours can actually aid the learning process, and time and money diverted from “the things that parents and educators actually care about,” such as small class size.

That’s probably the highest-order critique of the kind of comprehensive learner profile for which Knewton and others advocate. But it’s far from the only one.

Even some proponents of big data in K-12 schools say that many software companies purporting to personalize education by profiling students are going about it in the wrong way.

“There are Silicon-era snake-oil vendors that slot people into very crude pre-existing categories instead of learning from patterns that emerge in the data,” said Viktor Mayer-Schönberger, a professor of Internet governance and regulation at England’s Oxford University.

And the most vocal concerns about learner profiles come from those worried about the privacy and security of students’ data—especially when it comes to vendors like Knewton, which boasts about collecting orders of magnitude more data about its users than consumer companies such as Google and Facebook.

“Data collection is just out of control in the education space,” said Khaliah Barnes, a lawyer with the Electronic Privacy Information Center, or EPIC, a Washington-based advocacy group. “Children and families are really left out of the process and have no control over their information or how it will be used.”

Part of the problem, Ms. Barnes said, is the sheer complexity and opacity of what a company like Knewton actually does.

When trying to visualize what the company’s learner profiles actually look like, it’s easy to imagine a spreadsheet or a Facebook page or a manila folder with a paper dossier.

But in reality, the profiles that Knewton currently builds amount to constantly updating, mind-bogglingly long streams of alphanumeric code that are tied to an anonymous identifier. Much of that data is not directly about the individual student, but instead pertains to how his or her experiences contribute to a virtually endless series of probability calculations that also consider the experiences of millions of other students.

Even if it were possible to hit a button and receive the complete data record Knewton has amassed on an individual child—company officials say it’s not—the resulting information would be so voluminous and come in such a format that it would be unreadable by a human being.

Knewton’s data only becomes accessible to real people, company officials say, after it is whittled down and the relevant bits are presented through a dashboard created by one of its publishing partners. (Knewton doesn’t produce content or sell anything directly to schools, but gets paid by publishers to “power” their content with its learning-analytics technology.)

At no point in this process does Knewton hold any personally identifiable information on a child, company officials stressed. The company’s anonymized data are only connected back to specific children by publishers or schools, they said.

In the future, however, the company hopes to allow students and parents to create accounts directly with Knewton, opting to tie their identities to their learning data in order to gain access to a portable learner profile they could choose to “carry” with them from school to school or district to district.

Ms. Barnes of EPIC is skeptical.

If the hundreds of millions of records Knewton collects and the analyses it generates are at some point being tied back to an individual student, she said, then that student’s privacy is at stake—especially if the information is being used as the basis for important decisions about what the student knows, how he or she should be taught, and what his or her educational trajectory should look like.

And that student’s privacy is not being adequately protected, Ms. Barnes maintains, if he or she will never be able to access or contest the legitimacy of the information behind the analyses, will never be able to see or understand the algorithms performing the analyses, and will never know for sure what all that information he or she generated is ultimately being used for.

While such big-data-driven technologies are now widely accepted in the consumer world, she said, they deserve harsher scrutiny inside schools.

“If Amazon recommends a book you don’t want, there’s no lasting impact on your life,” Ms. Barnes said.

“But if a company says, ‘You’re not good at math’ or ‘Let me steer you away from this subject,’ that can be crippling to a student.”

Making matters worse, many in the ed-tech community believe current student-data-privacy laws are not up to the task of resolving other knotty questions presented by companies like Knewton, such as how long a third-party vendor should be allowed to maintain the information it gathers on students.

The constellation of concerns around data privacy were enough to topple the Atlanta-based nonprofit inBloom earlier this year, abruptly ending the most ambitious attempt to date to construct the type of data repository that might make comprehensive learner profiles possible on a large scale.

Beneath his hyperbole, Mr. Ferreira acknowledges—and even agrees with—some of the concerns that critics raise.

On the issue of data privacy, he said, it would be “suicide” for a vendor like Knewton to attempt to sell students’ sensitive information. “We sleep well at night,” he said, because the company does not gather any information that it can tie directly to an individual student.

But Mr. Ferreira also acknowledged that many school districts continue to do a terrible job of writing strong protections into their contracts with vendors who will be harvesting students’ data. And he conceded that parents often have little insight or say into the vast amounts of information on their children that is already being shared with third parties such as Knewton.

Both issues, he said, need to be resolved in a manner that gives parents more control over their children’s information.

Recognizing Limitations

And when it comes to the fundamental nature of schooling, Mr. Ferreira’s philosophy is actually quite similar to that of many of his critics.

“There’s too much testing, and too much teaching to the test,” he said. “Schools are reacting to [accountability-driven education policies] by drilling students on basic skills, and it’s just ruining education.”

In his view, Knewton should be allowed to take over that menial work, freeing educators to do the kinds of teaching that he believes really matters: building students’ critical-thinking skills, spurring their imaginations, and helping them understand difficult concepts, make sense of ambiguous information, and solve big problems.

Algorithm-powered adaptive software, he said, “doesn’t work if there isn’t a right answer” and an effective means of measuring student proficiency. Knewton doesn’t even attempt to “power” courses in fields that involve a high degree of subjectivity, complexity, and creativity, such as art, philosophy, and advanced math and science.

“We’re very careful to limit ourselves,” Mr. Ferreira said. “We know exactly where the borders are, where what we do doesn’t work anymore.”

Business Barriers

Conceptual and policy problems aside, Knewton—which has a significant presence in the U.S. higher education and international K-12 sectors—has been comparatively slow to grow its footprint inside K-12 schools in the United States for business reasons.

The company has partnerships with Triumph Learning, a small educational publisher based in New York City, as well as two of the largest educational publishers working in the United States, Houghton Mifflin Harcourt and Pearson.

But to date, Knewton is powering only very limited slices of each group’s content offerings. Getting its partners to focus on the business and technical hurdles associated with stitching together Knewton’s learner profiles across multiple textbooks, subjects, and years has mostly been a nonstarter.

Such companies also have a tremendous incentive to not provide a third party such as Knewton with the means to build learner profiles that cut across content from multiple publishers.

“This is a land grab, with many different commercial stakeholders going into the fray and trying to get a piece” of the growing market for big-data solutions in education, said Mr. Mayer-Schönberger, the Oxford professor.

For the time being, at least, the resulting balkanization in the marketplace will likely be reflected in schools, where unified, comprehensive learner profiles are likely to remain an aspiration, while smaller, segmented profiles for individual courses or publishers become an increasingly widespread reality.

Ultimately, only demand from parents and schools is likely to change that equation.

While there are stirrings in that direction among a handful of districts, the complexity of the technology, concerns around data privacy, and wariness over the direction of K-12 schooling still stand in the way.

Mr. Ferreira, however, remains undeterred in his conviction that the Holy Grail will eventually be found.

“Right now, the market doesn’t understand it,” he said. “But we’ve got to be ready for when the day comes when people do.”