Opinion Blog


Rick Hess Straight Up

Education policy maven Rick Hess of the American Enterprise Institute think tank offers straight talk on matters of policy, politics, research, and reform. Read more from this blog.

Classroom Technology Opinion

Do Educational Apps Actually Help Kids Learn?

For whom and under what conditions are important to consider, too
By Rick Hess — October 23, 2023 4 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
  • Save to favorites
  • Print

The last few years have made clear that education may be increasingly infused with technology but that much ed. tech is disappointing and distracting. Heck, I spent a whole chapter in The Great School Rethink on this very point and on what it’ll take to do better. Well, that’s what makes a new meta-analysis by Jimmy Kim and Josh Gilbert so intriguing. Kim, a professor at the Harvard Graduate School of Education, and Gilbert, a Ph.D. student at Harvard, dug deep into the research literature to figure out what we know about when and why apps actually help students learn. I think it’s fascinating stuff and asked if they’d mind sharing some of the takeaways. They were good enough to oblige. Here’s what they had to say.

—Rick

Can educational apps really help children learn? It’s a question that has become especially relevant since the COVID-19 pandemic, when millions of students were forced by school closures into online learning. With the abundance of educational apps available, parents and teachers are wondering whether they are truly effective in improving student learning outcomes such as math or reading test scores.

Researchers often conduct randomized controlled trials, RCTs, to test whether an educational app is effective in promoting student learning. In these studies, a group of students is randomly divided into two groups: one that practices their skills with the educational app and another that does not. Individual RCTs provide valuable insights, but they often involve small, homogeneous samples—which makes it challenging to draw broader conclusions about the overall effectiveness of educational apps.

This is where “meta-analysis” comes in. Meta-analysis is a statistical method that combines the results of multiple studies to provide a more comprehensive understanding of the overall research landscape. In meta-analysis, the data points are not the test scores of individual students, as in a single study, but rather the results of many studies. This lets us study the consistency of results and when or why they vary.

Our meta-analysis sampled 36 studies focusing on educational apps designed to improve math and reading skills in children aged 3 to 9. The findings bring good news: The overall effect of these apps is positive. But before parents and teachers rejoice and hand over tablets to their kids for the entire day, it’s important to note that the average positive effect hides significant variation in app effectiveness. Our study revealed that the effects of the apps ranged from slightly negative to hugely positive. With such a wide range of outcomes, it’s crucial to explore the characteristics of the studies and apps that may explain these differences.

We identified three key variables that accounted for the variation in app effectiveness. First, the effects were larger in studies that used tests specifically designed to measure the skills targeted by the app, rather than relying on externally developed standardized tests. Second, apps focusing on “constrained” skills, such as letter names or counting, showed larger effects compared with those targeting “unconstrained” skills like vocabulary or arithmetic. Third, the effects were more pronounced in preschool-aged children compared with those in kindergarten to 3rd grade. Interestingly, our findings also highlight what does not explain the differences in app effectiveness. The “dosage” of the app study—including the number of sessions, time spent per session, and duration of the study—did not predict variations in effectiveness. Additionally, there were no significant differences between apps aimed at improving literacy outcomes versus math outcomes.

So, what does this all mean for parents and teachers? Before they make any drastic decisions based on the latest study suggesting positive effects of an educational app, it’s crucial to consider critical questions about how the study was conducted. These questions will help evaluate the study’s findings and their relevance to individual situations.

First, it’s important to determine whether the study measured success using a standardized test or a researcher-developed test. Standardized tests are widely recognized and provide a clear benchmark for comparison, while researcher-developed tests may be more tailored to the specific skills targeted by the app. In general, studies that use standardized tests are more likely to be generalizable to other contexts than those that use researcher-developed tests.

Second, it’s essential to assess whether the study measured a constrained or unconstrained skill. Constrained skills refer to specific abilities like letter names or counting, which are more easily quantifiable and targeted by educational apps. Unconstrained skills, on the other hand, encompass broader abilities such as vocabulary or arithmetic, which may be more challenging to improve solely through app usage. Knowing which types of skills were evaluated will help determine the potential benefits of the app in question.

Finally, it’s valuable to consider whether the study focused on preschool-aged children or those in kindergarten to 3rd grade. Children at different developmental stages may respond differently to educational apps, and age-specific factors can influence their learning outcomes. Understanding the age group involved in the study will assist in gauging the relevance of the findings to the specific age range of interest.

Parents, teachers, and researchers should remember that the measures used in a study matter significantly. By critically evaluating the methodology and considering these important factors, we can make more informed decisions regarding the use of educational apps for children. The essential question isn’t simply whether apps work but rather, for whom and under what conditions they are effective.

The opinions expressed in Rick Hess Straight Up are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.

Events

School & District Management Webinar Crafting Outcomes-Based Contracts That Work for Everyone
Discover the power of outcomes-based contracts and how they can drive student achievement.
School & District Management Webinar EdMarketer Quick Hit: What’s Trending among K-12 Leaders?
What issues are keeping K-12 leaders up at night? Join us for EdMarketer Quick Hit: What’s Trending among K-12 Leaders?
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Artificial Intelligence Webinar
Teaching Students to Use Artificial Intelligence Ethically
Ready to embrace AI in your classroom? Join our master class to learn how to use AI as a tool for learning, not a replacement.
Content provided by Solution Tree

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Classroom Technology Typing Is Still a Foundational Skill. Do We Teach It That Way?
As more high-stakes testing goes digital, educators see a need to teach keyboarding skills in younger grades.
6 min read
Close cropped photograph of a child's hands on the proper computer keys of a white keyboard as they learn to type
Getty
Classroom Technology Opinion Why School Cellphone Bans Are a Bad Idea
We cannot ignore the powerful relationship between students and their phones—and what they mean for equity in our most challenged schools.
Brandon Cardet-Hernandez
4 min read
Trendy halftone collage. Hand holding and using cell phone.
Natalya Kosarevich/iStock
Classroom Technology From Our Research Center How Strict Are School Cellphone Policies?
New survey data show that schools are trying a variety of approaches to curb students’ cellphone use.
2 min read
Young student using on smartphone in classroom
Leonardo Patrizi/iStock/Getty
Classroom Technology From Our Research Center How Students Are Dodging Cellphone Restrictions
Schools’ efforts to restrict cellphone use have set up a battle of wits between teachers and students.
1 min read
A ninth grader places her cellphone in to a phone holder as she enters class at Delta High School, Friday, Feb. 23, 2024, in Delta, Utah. At the rural Utah school, there is a strict policy requiring students to check their phones at the door when entering every class. Each classroom has a cellphone storage unit that looks like an over-the-door shoe bag with three dozen smartphone-sized slots.
A 9th grader places her cellphone into a holder as she enters class at Delta High School in Delta, Utah, in February. The rural school has a strict policy requiring students to check their phones at the door when entering every class.
Rick Bowmer/AP