Short-Cycle Efficacy Trials Key to Personalized Learning

We're launching an ed-tech Innovation Hub in Chicago, and a key element is our pilot network, comprised of a diverse range of learning environments that will pilot and research the best existing and emerging tech learning solutions. Our initial focus is on K-12, and we received a Gates grant to support the first cohort of schools in our pilot network. The grant allows us to begin multi-phased evaluations of tech solutions and their effect on student outcomes using nationally normed assessments, as well as qualitative reviews. Our goal is to build an industry standard for quality products, helping ensure that effective tech tools solving critical learning gaps become available -- and scalable. The pilots also expose educators to promising solutions, open companies' eyes to real-world challenges facing teachers, provide a low-risk environment for trying innovation, and build a pipeline of innovative educators and schools to impact the greatest number of students.

The iZone's Innovate NYC Schools initiative received a development grant to explore the incorporation various kinds of efficacy metrics into the test bed model we validated for the GapApp prototyping work. Our emphasis was on building capacity through collaboration—the capacity of developers to understand better the realities of the urban classrooms that are the market for their work, and the capacity of educators to incorporate aspects of design thinking, prototyping, and agile/lean methodology into their pedagogy. The Gates grant will allow us to incorporate an efficacy component into our lessons-learned. Both of these elements—capacity-building and an understanding of what works in what contexts—are crucial to improving the ecosystem. Real-world, organic deployments of new technology and new pedagogy require more real-world organic indicators of impact: Not just the logistics but the premises of classic RCT work against real ROI for promising new approaches and ignore the realities of real-world implementation. It's very helpful that this new portfolio is paving the way for better-faster-cheaper evaluation and we're looking forward to contributing the benefits of our experience.

CFY received the largest grant from the Gates Foundation's RFP on evaluating the efficacy of digital courseware. CFY's proposal focused on short-cycle "A/B testing" of granular pieces of digital content—such as games, interactives, video, short simulations—to help software developers improve their products and know "what works." To conduct short-cycle A/B tests, CFY has developed a win-win process that plugs in to what teachers are already teaching (rather than being an "add on" that wastes classroom time), and provides valuable mastery feedback so students and teachers can see where help is needed. CFY will enlist teachers and students from its network of PowerMyLearning® users to participate in this process. PowerMyLearning is CFY's free K-12 platform, which has received major support from national foundations and is currently in use in more than 14,000 school communities nationwide.

eSpark Learning is embarking on an ambitious project to measure the impact of specific educational apps on student achievement and engagement. With the support of the Gates Foundation, we will evaluate hundreds of apps using real-time feedback from thousands of students all over the country. eSpark has been collaborating with school districts to increase student achievement for over three years. No other organization has more expertise in helping teachers and school leaders leverage iPads for student learning, and no other organization has a comparable infrastructure to analyze data on the impact of educational apps. eSpark's data on student achievement and engagement provides a previously unattainable view of the educational quality of iPad apps. This data will help us rigorously identify which content is truly best of breed, as revealed to us by thousands of students. We believe that student voice is central to building the leading digital curriculum. The aim of this project is to continually improve the digital curriculum. Specifically, we will link standard-level data on student achievement to apps and learning pathways, then use A/B testing techniques to determine how to restructure our curriculum to produce bigger learning gains. We already do much of this app-level optimization with student engagement, so the major step here is incorporating detailed student learning outcomes. We will report back to teachers, administrators, and app developers about what works in the modern classroom.

The three goals of the work are to (1) empower students to provide learning-experience data to teachers and school administrators; (2) enable teachers to access outcome-based application feedback from students and other teachers, in order to make better choices when selecting learning applications; and (3) establish a feedback channel between the teachers/students and vendors enabling, quicker and constant improvement throughout the system and its constituents. The application will be an extension of the widely popular Clarity platform, a SAAS-based tool used by thousands of schools globally. The tool collects feedback in a fun and engaging manner using gamification, while focusing data collection on learning outcomes and factors that influence them.

The grant funds concept testing and piloting of a process viaCommon Sense Media's Graphite ™ that links developers with expert educators for rapid, iterative testing and feedback on early stage EdTech products. We will consider both immediate (like teacher satisfaction, student learning and engagement) as well as more downstream (such as integration into technological and pedagogical infrastructures) measures of efficacy at schools/district, teacher/classroom, and student levels.