Deepak Agarwal, Principal Scientist, Khan Academy India

India has made remarkable progress in getting children into school. But learning outcomes—especially in mathematics—remain a challenge. Many students move from grade to grade without mastering foundational concepts, and the gaps keep widening over time.

Over the past few years, education technology has emerged as a potential solution, but evidence from schools across India and globally shows that its impact depends heavily on how it is used. 

A randomized controlled trial (RCT) led by University of Toronto researcher Philip Oreopoulos shows that when Khan Academy’s technology-based learning platform is implemented as intended, students learn significantly more than otherwise comparable students.

What is a randomized controlled trial (RCT)?

A randomized controlled trial is the most rigorous way to answer the key question: Did this program cause an increase in student learning or would the same thing have happened anyway? Imagine you want to test whether a new math program causes students to learn more. You take a group of schools that are broadly similar—same grades, similar backgrounds, same syllabus—and then randomly assign some schools to implement the program. These schools are called “Treatment” schools. Other schools continue with business as usual and are called “Control” schools. The program is implemented in the Treatment schools during the school year and also within the school’s timetable. If at the end of the program, on average, students in the Treatment schools outperform students in the Control schools, the difference can be attributed to the program.

The intervention: Regular use of Khan Academy for math practice, with the support of lab in-charges

In partnership with the Uttar Pradesh Department of Social Welfare, Khan Academy was implemented in 74 residential schools for students in Grades 6–8. Classroom teaching continued as usual in all schools. Khan Academy was used as a supplement to support practice, not to replace instruction. Twice a week in the computer lab, teachers and students were recommended to use Khan Academy for lesson-aligned practice and remediation using syllabus-matched exercises and videos. Students used individual devices (computers or tablets) for independent practice in online mode during designated timetable slots and under teacher supervision. This allowed for personalized pace and progression.

Out of the 74 total schools in the study, 24 schools were randomly selected to be part of the Treatment group, and the remaining 50 schools made up the Control group. The schools in the Treatment group received additional implementation support in the form of non-instructional facilitators who visited treatment schools twice a week, referred to in this study as lab in-charges. Their responsibilities were to ensure two Khan Academy sessions per week, troubleshoot technical- and program-related challenges, monitor student progress data, and work with school leadership to integrate Khan Academy practice into mandatory curriculum time. The Control schools had access to Khan Academy, but the Khan Academy program was not actively promoted in the Control schools during the intervention period. 

Over the course of seven months, the study included approximately 2,000 students in the Treatment condition and 3,500 students in the Control condition. Students’ learning outcomes were measured through baseline and endline mathematics assessments administered to students in grades 6-8. An independent assessment agency developed grade-specific tests aligned with both CBSE and UP Board curricula and administered in Hindi.

Clear evidence of learning gains

Students in Treatment schools scored 0.44 to 0.47 standard deviations higher than students in the control group on the end-of-year mathematics assessment. To contextualize these magnitudes, a 0.44 to 0.47 standard deviation gain represents moving an average student from the 50th percentile to approximately the 67th or 68th percentile.

Why did students learn more? Because they practiced more. Students in the Treatment group used Khan Academy for an average of ~47 minutes per week to practice math content that was closely integrated with the classroom curriculum. When there is clear ownership, monitoring, and motivation to use the program, usage increases. Many edtech platforms do not see students achieving meaningful levels of practice time. This study shows one means to achieve that.

More practice led to more learning—even 15 minutes per week made a difference

During the study, students in the Treatment group used Khan Academy twice per week for an average of 47 minutes, but the amount of time they spent was not uniform across the board. Some students used Khan Academy more while others used it less. We analyzed this variation in time spent by each student and correlated it with the associated learning gains (see figure below). This analysis shows that even 15 minutes of usage each week can help students make considerable progress. What’s more, learning gains are proportional to the amount of time students spend practicing on the platform. In other words, more practice means more learning.

Learning gains were consistent across grades and genders, as well as for students at different starting points

Another highly important and desirable feature of Khan Academy implementation is that its benefits were not limited to a small subset of students. Learning gains for the Treatment group were consistent across grades, genders, and performance levels, indicating that students of all backgrounds and ability levels can benefit from using the Khan Academy (see figure below). 

What this means

This randomized controlled trial shows that when there is support for implementation, students can reach desired levels of usage and realize significant learning gains. Digital learning tools can deliver large gains when schools are given enough support to use them efficiently—even in challenging real-world settings.

When thoughtful structures are put in place to ensure regular practice and logistical challenges are addressed, it creates an environment in which technology usage translates to improved skill mastery and ultimately results in test-score gains. In such a setting, students engage more deeply with their studies and learn more effectively. The technology becomes a vehicle for activating the processes necessary for facilitating deep learning.

Khan Academy supported such an intervention by providing mastery-based, curriculum-aligned practice in Hindi, enabling students to learn at their own pace while giving teachers and lab in-charges visibility into student progress. The combination of content delivered in their native language, regular practice time, sustained engagement, and efficient practice time successfully converted platform access into meaningful skill development. The results demonstrate that when digital learning tools are thoughtfully integrated into the curriculum, they can deliver large and equitable gains. 

**

Read Dr. Oreopoulous’ working paper with the findings here. 

¹ Standard deviation is a common way researchers compare learning gains across different tests and settings. It measures how much scores shift relative to the typical spread of student performance. Reporting results in standard deviation units allows learning gains to be compared across studies, even when assessments differ.