Bi-Modal Deep Neural Network for Gait Emotion Recognition

Abstract

Emotion Recognition systems can be used for autonomous tasks such as video gaming experiences, medical diagnosis, adaptive education, and smart homes. Several biometric modalities, including face, hands, and voice have been successfully used for emotion recognition tasks. Gait Emotion Recognition (GER) is an emerging domain of research that is focused on identifying the emotional state of a person from gait biometric, which represents the person’s manner of walking. In comparison to the other modalities, gait provides a non-intrusive method to collect data remotely without an expert’s supervision. Moreover, unlike facial expression-based emotion recognition, it does not require high-resolution data for inference. Early works in GER produced limited feature sets and used classical machine learning methodologies to infer emotions, but could not achieve high performance. This thesis proposes powerful architectures based on deep-learning to accurately identify emotions from human gaits. The proposed Bi-Modal Deep Neural Network (BMDNN) architecture utilizes robust handcrafted features that are independent of dataset size and data distribution. The network is based on Long Short-Term Memory units and Multi-Layered Perceptrons to sequentially process raw gait sequences and facilitate feature fusion with the handcrafted features. Lastly, the proposed Bi-Modular Sequential Neural Network (BMSNN) has a low number of parameters and a low inference time, hence making it suitable for deployment in real world applications. The proposed methodologies were evaluated on the Edinburgh Locomotive MoCap Dataset and outperformed all recent state-of-the-art methods.