Exploring Emotion Recognition of Students in Virtual Reality Classrooms Through Convolutional Neural Networks and Transfer Learning Techniques
Date
2024-01-15
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
In contemporary educational settings, understanding and assessing student engagement through non-verbal cues, especially facial expressions, is pivotal. Such cues have long informed educators about students' cognitive and emotional states, assisting them in tailoring their teaching methods. However, the rise of online learning platforms and advanced technologies like Virtual Reality (VR) challenge the conventional modes of gauging student engagement, especially when certain facial features become obscured or are entirely absent. This paper explores the potential of Convolutional Neural Networks (CNNs), specifically a customized training approach model adapted from the ResNet50 architecture, in recognizing and distinguishing subtle facial expressions in real-time, such as neutrality, boredom, happiness, and confusion. The novelty of our approach is twofold: First, we optimize the power of CNNs to analyze facial expressions in digital learning platforms. Second, we innovate for the context of VR by focusing on the lower half of the face to tackle occlusion challenges posed by wearing VR headsets. Through comprehensive experimentation, we compare our model's performance with the default Residual Neural Network 50 (ResNet50) and evaluate it against full-face and VR-occluded face datasets. Ultimately, our endeavor aims to provide educators with a sophisticated tool for real-time evaluation of student engagement in technologically advanced learning environments, subsequently enriching the teaching and learning experience.
Description
Keywords
emotion recognition, facial expression, VR, virtual reality, virtual learning environment, machine learning, deep learning, CNN, Convolutional Neural Networks, transfer learning, ResNet50, VGG19, MobileNet
Citation
Shomoye, M. A. (2024). Exploring emotion recognition of students in Virtual Reality classrooms through Convolutional Neural Networks and transfer learning techniques (Master's thesis, University of Calgary, Calgary, Canada). Retrieved from https://prism.ucalgary.ca.