QuizBot: A Dialogue-based Adaptive Learning System for Factual Knowledge

Smart Primer Research Group, Computer Science Department, Stanford University
May 2018 - November 2018






Two Studies


Study Procedures




What is QuizBot?

QuizBot is an AI-powered chatbot to help college students review questions through natural-language conversations.


Advances in conversational AI have the potential to enable more engaging and effective ways to teach factual knowledge.


To investigate this hypothesis, we created QuizBot, a dialogue-based agent that helps students learn factual knowledge in science, safety, and English vocabulary. We compared QuizBot against a Flashcard app, the traditional medium for learning factual knowledge.


We evaluated QuizBot with 76 students through two within-subject studies against the Flashcard app. Though both systems used the same algorithm for sequencing materials, QuizBot led to students recognizing (and recalling) over 20% more correct answers than when students used the flashcard app. Using a conversational agent is more time consuming to practice with; but in a second study, of their own volition, students spent 2.6x more time learning with QuizBot than with flashcards and reported preferring it strongly for casual learning. Our results in this second study showed QuizBot yielded improved learning gains over Flashcard on recall.


Our results suggest that educational chatbot systems may have beneficial use, particularly for learning outside of traditional settings.


• As the lead developer, together with five other programmers, developed the front and back ends of QuizBot, a dialogue-based conversational agent embedded in Facebook Messenger app, via Python3, AWS and SES Flask
• Devised a full-stack Apache Cordova Flashcard app via HTML and JavaScript, deployed to the iOS and Android platforms
• Iterated the design and implementation of the two apps via a pilot study with 38 users
• Together with two other researchers, conducted two within-subject studies with 76 users to compare the learning engagement and outcomes using QuizBot and Flashcard
• Submitted a paper that describes our work and results to CHI 2019 as the second author


QuizBot: A Dialogue-based Adaptive Learning System for Factual Knowledge

Sherry Ruan, Liwei Jiang, Justin Xu, Bryce Tham, Zhengneng Qiu, Yeshuang Zhu, Elizabeth Murnane, Emma Brunskill, and James A. Landay

CHI: ACM Conference on Human Factors in Computing Systems, 2019 | Paper | Slides | Video | Press | BibTeX

© Copyright Liwei Jiang Template by Colorlib