BookBuddy: Turning Digital Materials to Interactive Second Language Learning Lessons Through a Voice Chatbot

Smart Primer Research Group, Computer Science Department, Stanford University
February 2019 - June 2019

BookBuddy is greeting the child and gathering user information for book recommendation


Image

BookBuddy is quizzing the child on topics related to the book Oh, Raccoon shown in the left window


Image

The interaction flow of the BookBuddy system


Image

The chatbot system architecture in BookBuddy


Image

Image

What is BookBuddy?

BookBuddy is a scalable virtual reading companion that can turn any reading material into an interactive conversation-based English lesson.


Hypothesis

Children enjoy speaking English with a virtual tutoring chatbot and will be highly engaged during the interaction.


Who use BookBuddy?

BookBuddy is designed for Children who learn to improve their reading ability. The difficulty of the content of the lessons can be customized for children with different ages and reading ability.

Systems

BookBuddy is a web application that can be accessed with a computer. The interface has two windows side by side. The left window is a reading window that delivers stage-dependent content including the chatbot’s avatar, the cover of the recommended book, and the book page that the child is currently studying. The right window is a chat window that allows the child to have both spoken and typed conversation with the chatbot. The BookBuddy interaction flow has four stages as demonstrated: introduction, recommendation, learning, and assessment. The child can browse the conversation history by scrolling the chat window. A synthesized voice accompanies all text messages sent by the chatbot to the child.


Study

We recorded five 6-year-old Chinese children, who all spent fewer than seven months abroad, using BookBuddy to read one book in our database. We used screen-capturing technology and positioned webcams to record the students’ faces while using BookBuddy. Following the protocol in the study of the educational software AutoTutor, we coded users’ emotions in each 20-second segment in the videos. The coding was done by two independent coders, both native speakers of Chinese born and raised in China, in order to avoid cultural differences in displays of emotions. Each 20-second segment was coded as one of seven possible emotions: engagement, boredom, confusion, curiosity, happiness, frustration, or neutral.


Implication

Pilot study results indicate that children were highly engaged while interacting with the system and preferred speaking English with our chatbot over human partners. Our work is a first step toward creating conversation-based intelligent tutoring systems at scale for improving children’s foreign language speaking skills.

Roles


• Conducted a study with five 5 and 6-year-olds for using the BookBuddy app to practice reading abilities
• Processed the recorded videos sessions and coded emotions of the five kids while they used the app to study
• Coauthored a paper that describes our work and results at L@S 2019

Resources


BookBuddy: Turning Digital Materials to Interactive Second Language Learning Lessons Through a Voice Chatbot

Sherry Ruan, Angelica Willis, Qianyao Xu, Glenn Davis, Liwei Jiang, Emma Brunskill, and James A. Landay

L@S WIP: Proceedings of the Sixth Annual ACM Conference on Learning at Scale, 2019 | Paper | Video | BibTeX

© Copyright Liwei Jiang Template by Colorlib