Object-Subject Relationship Detection using Deep Learning and Decision Support Systems for Safety and Security Applications
Date
2022-01
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
This thesis develops the theoretical foundations and practical realization of state-of-the-art computer vision methodologies for safety and security applications. Deep neural networks have shattered standard benchmarks on popular computer vision datasets in recent years, outperforming classical digital image processing methodologies by significant margins. Unfortunately, research on useful applications of these state-of-the-art computer vision models to real-world scenarios related to safety and security are currently lacking. The first portion of this thesis consists of several manuscripts presenting the development of a state-of-the-art deep neural network models which address the object and relationship detection problem. The object and relationship detection problem involves predicting the visual relationship triplet, <subject, predicate, object>. The subject and the object detections are localized regions on the image which contain the identified subject and object. The predicate is the description of the relationship between the subject and the object. We demonstrate that convolutional neural network-based approaches to object detection with a soft-attention mechanism for visual relationship detection is effective on several computer vision problems involving subject-object weapon relationships, face mask relationships, and workplace personal protective equipment relationships. The second portion of this thesis contains manuscripts which approach interpretation of deep neural network outputs through meta-analysis of validation results to improve testing results and real-world applicability and trustworthiness. Methodologies utilizing score-level fusion methods and Bayesian networks are demonstrated to improve model performance and interpretability in computer vision problems involving person identification and object detection. The third and final portion of this thesis covers the development of a graphical user interface and deep neural network-based backend model which combines the developments in this thesis into a stand-alone tool for researchers to use. The tool provides a foundational base for future research and development related to visual relationship detection and its applications to safety and security.
Description
Keywords
computer vision, object detection, visual relationship detection
Citation
Truong, T. (2022). Object-subject relationship detection using deep learning and decision support systems for safety and security applications (Doctoral thesis, University of Calgary, Calgary, Canada). Retrieved from https://prism.ucalgary.ca.