From gestures to robots: two new SCIoI studies reveal how AI enhances learning and teacher-student connections

How can AI tools be used to foster learning and to improve our understanding of effective nonverbal communication? Two new publications provide insights into how researchers from the Cluster Science of Intelligence (SCIoI) address these questions. One study, published in the British Journal of Educational Psychology, shows how computer vision methods based on classroom video data enhance our understanding of teachers’ adaptive nonverbal social behaviors in class. The second study, published in npj science of learning, reveals that nonverbal behaviors matter when interacting with social robots in learning situations. Students enjoyed learning more with embodied robot companions than with non-embodied AI-based agents.

Paper British Journal of Educational Psychology: https://doi.org/10.1111/bjep.12732
Paper npj science of learning: https://doi.org/10.1038/s41539-024-00293-z

How nonverbal communication shapes classroom engagement and how computer vision methods help us understand it

Understanding what motivates students to engage in learning is a key challenge in education research. The new study “‘No words’—Machine-learning classified nonverbal immediacy and its role in connecting teacher self-efficacy with perceived teaching and student interest”, first-authored by Rebecca Lazarides and co-authored by Olaf Hellwich, Uroš Petković and Jonas Frenkel, all members of SCIoI, offers insights into the role of teachers’ nonverbal communication in shaping student interest and engagement, highlighting how subtle cues like gestures, facial expressions, and posture can make a significant difference.

The research focuses on teacher self-efficacy, which refers to teachers’ confidence in their ability to motivate students, and its connection to nonverbal immediacy (NVI). NVI encompasses behaviors that convey warmth and approachability, such as maintaining eye contact, smiling, using expressive gestures, and reducing physical distance. These actions help create a sense of closeness between teachers and students, fostering a positive learning environment.

Using video recordings and computational analysis, researchers found that teachers who reported higher confidence in their ability to engage students displayed more pronounced nonverbal behaviors. These behaviors were perceived by students as enthusiasm, leading to increased interest immediately after the lesson. However, the impact on long-term student interest was less pronounced, suggesting additional strategies may be needed to sustain engagement over time.

This study highlights the importance of nonverbal communication in teaching, showing how subtle cues can enhance classroom interactions. By improving awareness of nonverbal immediacy, educators can foster stronger connections with students and create more engaging learning experiences.

How physical robots spark curiosity in classrooms and how AI systems reveal their educational impact

Could the presence of a physical robot in the classroom make learning more engaging? Research into the use of artificial intelligence in education suggests it might—at least at first. The second study, titled “Physical embodiment and anthropomorphism of AI tutors and their role in student enjoyment and performance” and first-authored by Helene Ackerman and co-authored by Rebecca Lazarides, Verena Hafner, Niels Pinkwart, Anja Henke, Johann Chevalère and Hae Seon Yun, another SCIoI team,  explores how physically embodied AI tutors, combined with virtual agents, impact students’ enjoyment and learning outcomes in an Intelligent Tutoring System (ITS) called “Betty’s Brain.”

The findings showed that students initially reported higher enjoyment when interacting with a physical robot alongside a virtual tutor, but this excitement faded over time and had no measurable effect on their task performance. Surprisingly, perceptions of the robots’ human-like qualities, such as sociability, were sometimes linked to poorer performance, while students who found the robots distracting enjoyed the experience less.

This research highlights the challenges of designing robotic tutors to complement rather than distract from learning. Far from suggesting that robots could replace human teachers, the study points to their potential as tools for personalized, adaptive learning, when thoughtfully designed. However, sustaining engagement and addressing individual student needs remain key hurdles for integrating these technologies into classrooms effectively. And while technology was used to analyze these behaviors, the goal is not to replace teachers but to better understand and support effective teaching practices.

Advancing Learning Through AI and Nonverbal Communication

These two studies demonstrate how research within SCIoI uses AI tools to address important questions about education and communication. By applying computer vision to analyze nonverbal behaviors and exploring the role of physical robots in classrooms, the findings shed light on how both human and artificial intelligence can enhance learning experiences. This research aligns with SCIoI’s interdisciplinary focus, combining insights from psychology, education, and technology to better understand and improve human-AI interactions in practical settings.