Browsing by Author "Zhao, Richard"

Now showing 1 - 19 of 19
  • Thumbnail Image
    ItemEmbargo
    A Case of One: An Autobiographical Design Approach to Explore a Personal Informatics Preparation Stage Procedure
    (2024-09-20) Zhang, Xinchi; Schroeder, Meadow; Ringland, Kathryn; Zhao, Richard; Wang, Mea
    This thesis exploration was started as a personal design endeavor to have a system that can support realistic task arrangement during my graduate school. This exploration landed to the often-overlooked area — the preparation stage in the Stage-Based Model (SBM) — in the personal informatics (PI) field. Personal informatics supports people to gain self-understanding through reflection on their relevant personal data. The preparation stage, which can involve many decision-making processes such as understanding the motivation of collecting personal information, deciding the information to collect, and choosing the appropriate tools, is where prior PI research focused significantly less on. This thesis aims to narrow this gap by introducing a procedure and an accompanying artifact, Qubio. I took an autobiographical design approach. Autobiographical design offers many advantages such as close use to allow rapid iteration whenever needed (fast tinkering). Then, combining with reflection, diligent documentation (46+ hrs recordings, 262 reflection entries), and long-term usage (47 months), I established a personal reflective procedure to determine what data I might track. The procedure includes 1) externalization of obligations and interests, 2) mapping (for goal choices), and 3) task arrangement, which is supported by the token-based artifact, Qubio. This exploration bridges the preparation stage of the Stage-Based Model in PI and the Integrated Model of Goal-Focused Coaching (Integrated Model) in psychology. I conclude this thesis by discussing research opportunities in connection to the Integrated Model for the preparation stage in PI and suggesting collaboration between PI and personal information visualization to support visualization agency in PI practices. I further suggest revisiting established PI models to potentially integrate the field’s expanded understanding of PI related activities. Finally, I reflect on how an autobiographical design approach produced a personalized procedure and artifact.
  • Thumbnail Image
    ItemOpen Access
    A Deep Learning Based Method for Fast Registration of Cardiac Magnetic Resonance Images
    (2024-04-29) Graham, Benjamin; Jacob, Christian; Alim, Usman; Zhao, Richard; Garcia Flores, Julio; Hoyer, Peter
    Image registration is used in many medical image analysis applications, such as tracking the motion of tissue in cardiac images, where cardiac kinematics can be an indicator of tissue health. Registration is a challenging problem for deep learning algorithms because ground truth transformations are not feasible to create, and because there are potentially multiple transformations that can produce images that appear correlated with the goal. Unsupervised methods have been proposed to learn to predict effective transformations, but these methods take significantly longer to predict than established baseline methods. For a deep learning method to see adoption in wider research and clinical settings, it should be designed to run in a reasonable time on common, mid-level hardware. Fast methods have been proposed for the task of image registration but often use patch-based methods which can affect registration accuracy for a highly dynamic organ such as the heart. In this thesis, a fast, volumetric registration model is proposed for the use of quantifying cardiac strain. The proposed Deep Learning Neural Network (DLNN) is designed to utilize an architecture that can com-pute convolutions incredibly efficiently, allowing the model to achieve registration fidelity similar to other state-of-the-art models while taking a fraction of the time to perform inference. The proposed fast and lightweight registration (FLIR) model is used to predict tissue motion which is then used to quantify the non-uniform strain experienced by the tissue. For acquisitions taken from the same patient at approximately the same time, it would be expected that strain values measured between the acquisitions would have very small differences. Using this metric, strain values computed using the FLIR method are shown to be very consistent.
  • Thumbnail Image
    ItemOpen Access
    An Elicitation Study on Multi-modal Interactions with Immersive Data
    (2024-05-17) Farajian, Samin; Maurer, Frank; Zhao, Richard; Far, Behrouz; Maurer, Frank
    In the rapidly advancing domain of Extended Reality (XR), technological progress has introduced a broad spectrum of input and interaction methods. This thesis investigates user preferences for interacting with immersive data via a head-mounted display (HMD). We introduce two task types based on the existing task taxonomy in the context of data interaction in an immersive environment. The results of an elicitation study are presented by examining user preferences and measuring interaction agreements based on these two task types. Specifically, the study aims to determine which body-based interactions users naturally prefer when engaging in various tasks to interact with data visualizations. Additionally, it seeks to gain insights into how the distance from immersive data influences users' experiences and preferred interactions. Our goal is to gather insights to develop guidelines that contribute to designing interactions with immersive data in single-user applications.
  • Thumbnail Image
    ItemOpen Access
    Anti-Freeze: High-Quality Adaptive Live Streaming with Real-time Transcoder
    (2023-01-05) Mehmuda, Asif Ali; Wang, Mea; Krishnamurthy, Diwakar; Drew, Steve; Zhao, Richard
    Video streaming constitutes more than 80% of Internet traffic [1], and the demand continues to rise as interactive video applications like virtual conferencing/collaboration as well as augmented/mixed reality emerge. Such interactive video applications pose a challenge for real-time video transcoding and streaming. Transcoding is a computationally intense process and if not performed efficiently it can result in unwanted delays, which further limit the Quality-of-Service (QoS) delivered by the streaming protocol. In this thesis, we aim to address the real-time challenge and propose Antifreeze, a complete end-to-end solution for real-time transcoding and streaming. We propose a machine-learning solution to estimate the transcoding time and resource requirement, which is used to direct our real-time transcoder to transcode video segments at the rate matching their frame rate. We complete the Antifreeze design with a novel quality adaptation algorithm that not only considers visual quality and bandwidth dynamics, but also transcoding time and necessary computing resource allocation. Our results show that Antifreeze significantly reduces the playback stalls and substantially improves the visual quality in interactive video streaming sessions under various bandwidth profiles.
  • Thumbnail Image
    ItemOpen Access
    Computational Media Design: Using Graph Data to Improve Non-Player Character Acting in Games
    (2023-09-14) Brierley, Owen Douglas; Finn, Patrick; Jacob, Christian; Aycock, John; Sengupta, Pratim; Zhao, Richard; Leblanc, Jean-Rene; Guzdial, Matthew
    This thesis uses graph databases to enhance non-player character (NPC) behavior in computer games. The approach is tested in three discrete projects by developing and using the Neo4jConnector, a custom toolkit enabling novel bi-directional communication between real-time simulation data and server-based graph database long-term storage. The first project demonstrates the Neo4jConnector’s use in recording player movement as graph data and facilitating NPC movement playback from this data. The second project explores improving NPCs’ non-deterministic behaviors through q-learning, a reinforcement learning algorithm, and storing the resultant graph data in the server-based database. The third project investigates dynamically loading 3D geometry from a graph database in multi-scale cellular simulations, supporting intricate bioinformatics simulations necessitating non-deterministic agent actions. Emphasizing an artist’s approach to Computational Media Design, the research targets animators, game makers, bioinformatics researchers, and the growing field of games research. Through an elaborative design process, the thesis showcases the value of incorporating graph databases in game production environments, offering new opportunities to create more realistic and adaptive NPC behaviors that boost player engagement and improve the quality of simulations for future research.
  • Thumbnail Image
    ItemOpen Access
    The effects of sharing viewpoints on task performance in collaborative VR applications
    (2023-02-10) Aminbeidokhti, Amir; Maurer, Frank; Far, Behrouz; Zhao, Richard
    While virtual reality applications allow for face-to-face collaboration and the ability to see each other’s avatars, having different graphical viewpoints can hinder task performance due to the confusion caused by left-right ambiguities and text orientation. In this thesis, we investigate the effect of altering collaborators’ viewpoints and sharing one collaborator’s viewpoint with the other without moving the avatars’ positions in a collaboration between two people in Virtual Reality. We created a Virtual Reality application and twelve scenarios to understand this approach’s effect. Two terms were defined: Shared Viewpoint, which is when users’ graphical viewpoints are decoupled from their avatars and put next to each other, and Independent Viewpoint, in which users’ graphical viewpoints are where their avatars are located. A user study is conducted to gather qualitative and quantitative data. We calculated task completion time and users’ preferences when they have a shared graphical viewpoint from the virtual world and object. Our findings showed that users prefer a shared viewpoint rather than independent viewpoints. Additionally, we discovered that having a shared viewpoint can either increase or decrease task completion time, depending on the relative positions of avatars around the table.
  • Thumbnail Image
    ItemOpen Access
    Expanding the User Interactions and Design Process of Haptic Experiences in Virtual Reality
    (2023-08) Smith, Christopher Geoffrey; Sharlin, Ehud; Somanath, Sowmya; Suzuki, Ryo; Sharlin, Ehud; Somanath, Sowmya; Suzuki, Ryo; Zhao, Richard
    Virtual reality can be a highly immersive experience due to its realistic visual presentation. This immersive state is useful for applications including education, training, and entertainment. To enhance the state of immersion provided by virtual reality further, devices capable of simulating touch and force have been researched to allow not only a visual and audio experience but a haptic experience as well. Such research has investigated many approaches to generating haptics for virtual reality but often does not explore how to create an immersive haptic experience using them. In this thesis, we present a discussion on four proposed areas of the virtual reality haptic experience design process using a demonstration methodology. To investigate the application of haptic devices, we designed a modular ungrounded haptic system which was used to create a general-purpose device capable of force-based feedback and used it in the three topics of exploration. The first area explored is the application of existing haptic theory for aircraft control to the field of virtual reality drone control. The second area explored is the presence of the size-weight sensory illusion within virtual reality when using a simulated haptic force. The third area explored is how authoring within a virtual reality medium can be used by a designer to create VR haptic experiences. From these explorations, we begin a higher-level discussion of the broader process of creating a virtual reality haptic experience. Using the results of each project as a representation of our proposed design steps, we discuss not only the broader concepts the steps contribute to the process and their importance, but also draw connections between them. By doing this, we present a more holistic approach to the large-scale design of virtual reality haptic experiences and the benefits we believe it provides.
  • Thumbnail Image
    ItemOpen Access
    Exploring Adaptive MCTS with TD Learning in miniXCOM
    (2022-10-24) Saadat, Kimiya; Zhao, Richard
    In recent years, Monte Carlo tree search (MCTS) has achieved widespread adoption within the game community. Its use in conjunction with deep reinforcement learning has produced success stories in many applications. While these approaches have been implemented in various games, from simple board games to more complicated video games such as StarCraft, the use of deep neural networks requires a substantial training period. In this work, we explore on-line adaptivity in MCTS without requiring pre-training. We present MCTS-TD, an adaptive MCTS algorithm improved with temporal difference learning. We demonstrate our new approach on the game miniXCOM, a simplified version of XCOM, a popular commercial franchise consisting of several turn-based tactical games, and show how adaptivity in MCTS-TD allows for improved performances against opponents.
  • Thumbnail Image
    ItemOpen Access
    Exploring Comfortable Coexistence with Autonomous Pods in Pedestrian Spaces
    (2024-06-26) Luchak, Iryna; Sharlin, Ehud; Zhao, Richard; Finn, Patrick
    The rapid advancement of autonomous vehicles (AVs) marks a significant milestone in transportation technology, revolutionizing mobility and urban planning. AVs promise to improve road safety, reduce traffic congestion, and contribute to environmental sustainability. As AVs become increasingly adept at navigating complex environments, challenges arise in integrating them into existing infrastructures, making it crucial to study their coexistence with people and how they are perceived. Pods, also known as low-speed autonomous transport systems (L-SATS), are emerging in pedestrian areas like airports and malls to help solve the last-mile problem. Nonetheless, their introduction into pedestrian-centred spaces presents new challenges for researchers in ensuring the comfortable sharing of these spaces with people. This thesis explores and investigates factors contributing to comfortable coexistence between pods and incidentally copresent persons (InCoPs) in pedestrian spaces. Through our exploration, we make several contributions. We begin by examining autonomous vehicles in urban spaces, including a preliminary exploration of situated visualizations, proxemics, and technology acceptance, which leads us to the notion of pods in pedestrian spaces. We then propose a design space for pods in pedestrian spaces, reflecting on various aspects of coexistence between InCoPs and pods. Our dimensions describe the pedestrian space, pod interactions, and the physical design of pods. Using the initial design space dimensions as a foundation and inspiration in design, we build a virtual reality (VR) testbed to facilitate research on the coexistence between pods and InCoPs. We design ten scenarios in the pedestrian space testbed. We conduct a user study, analyzing the significance of different variables, including pod quantity, pod group formation, passenger presence, and InCoP position. We provide insights on factors enhancing InCoPs' comfort, emphasizing the importance of an improved sense of control, space and freedom to move, passenger awareness, and the social behaviours of other pedestrians. By examining key factors that contribute to InCoP coexistence alongside pods, this thesis aims to offer initial research insights into the future integration of pods in pedestrian spaces to improve the comfort of InCoPs.
  • Thumbnail Image
    ItemOpen Access
    Exploring Convolutional Neural Networks and Transfer Learning for Oil Sands Drill Core Image Analysis
    (2021-08-24) Anzum, Fahim; Costa Sousa, Mario; Alim, Usman; John Jacobson Jr., Michael; Osvaldo Trad, Daniel; Zhao, Richard
    An accurate permeability estimate is crucial for effectively characterizing the McMurray oil sands for in situ recovery. Such an estimate is critical to inform the best locations for placing wells and pads and accurately forecast future oil production rates. This fact is becoming significantly important as in situ development moves to areas of increasingly complex geology. The traditional methods of estimating permeability largely do not work well in oil sands because of the core disturbance or the fact that the core is filled with immobile bitumen. Moreover, it is expensive to get physical samples from many different depths at many wells, and the experiments carried out in the labs to measure permeability sometimes are not representative. However, permeability can be estimated from different parameters such as mean grain size (MGS), median grain size, and particle size distribution (PSD). This thesis investigates how convolutional neural networks (CNNs) and transfer learning perform when estimating MGS from the oil sands drill core photos. Three preliminary approaches are explored for classifying core photos based on the facies, including (1) the application of transfer learning on the pre-trained VGG-16 CNN model, (2) fine-tuning a few top layers of VGG-16, and (3) the combination of VGG-16 and traditional machine learning (ML) algorithms. Experimental results achieved by these classification models reveal opportunities to extend these approaches for predicting MGS from core photos. Therefore, the three approaches are then investigated using a library of core photographs with known MGS calculated from PSD to see which one works best. Experimental results exhibit good performance in estimating MGS from core photos using the explored approaches. Overall, the investigation supports that the application of CNNs, and transfer learning is feasible in different oil sands drill core image analysis workflows and more advanced research outcomes can be achieved by further exploration of these techniques in the oil sands research domain.
  • Thumbnail Image
    ItemOpen Access
    Exploring Emotion Recognition of Students in Virtual Reality Classrooms Through Convolutional Neural Networks and Transfer Learning Techniques
    (2024-01-15) Shomoye, Michael Abidemi; Zhao, Richard; Farhad, Maleki; Usman, Alim
    In contemporary educational settings, understanding and assessing student engagement through non-verbal cues, especially facial expressions, is pivotal. Such cues have long informed educators about students' cognitive and emotional states, assisting them in tailoring their teaching methods. However, the rise of online learning platforms and advanced technologies like Virtual Reality (VR) challenge the conventional modes of gauging student engagement, especially when certain facial features become obscured or are entirely absent. This paper explores the potential of Convolutional Neural Networks (CNNs), specifically a customized training approach model adapted from the ResNet50 architecture, in recognizing and distinguishing subtle facial expressions in real-time, such as neutrality, boredom, happiness, and confusion. The novelty of our approach is twofold: First, we optimize the power of CNNs to analyze facial expressions in digital learning platforms. Second, we innovate for the context of VR by focusing on the lower half of the face to tackle occlusion challenges posed by wearing VR headsets. Through comprehensive experimentation, we compare our model's performance with the default Residual Neural Network 50 (ResNet50) and evaluate it against full-face and VR-occluded face datasets. Ultimately, our endeavor aims to provide educators with a sophisticated tool for real-time evaluation of student engagement in technologically advanced learning environments, subsequently enriching the teaching and learning experience.
  • Thumbnail Image
    ItemOpen Access
    FrAG: Framework for the Analysis of Games
    (2023-12-19) Ganesh, Sankarasubramanian; Aycock, John; Henry, Ryan; Zhao, Richard; Aycock, John
    Historical games or retrogames ran on constrained systems that required programmers to use various techniques and optimizations. To learn about these techniques, we often study their source code. However, when the only remaining information about the games is their binary image, conventional analysis methods are time-consuming and do not scale. One way to study these techniques is to reverse engineer the binary images. However, conventional approaches to reverse engineer the images often do not provide accurate results as programmers often blurred the lines between code and data, because unlike most modern platforms, these platforms do not distinguish between code and data. We present FrAG, a Framework for the Analysis of Games that dynamically analyzes games at scale with no human interaction using artificial intelligence. FrAG’s design allows it to be ported to other platforms. To demonstrate FrAG’s capability and to evaluate its efficacy, we use a test suite of eight Atari 2600 games with ground truth available. We also present a novel way of disassembling game ROMs using data collected from the framework. Furthermore, FrAG can also be used as a platform for training artificial intelligence agents for several platforms.
  • Thumbnail Image
    ItemOpen Access
    Large-surface Passive Haptic Interactions using Pantograph Mechanisms
    (2024-01-17) Friedel, Marcus Kenneth Ernst; Suzuki, Ryo; Sharlin, Ehud; Nittala, Aditya; Zhao, Richard
    Dexterous and natural haptic interaction with the environment in Virtual Reality promises a new era of embodied and intuitive computing. But among the remaining challenges stands the difficulty of natural wall interactions. Personal haptic devices for natural wall interaction in virtual reality should be portable and should provide passive, body-scale interactions. However, existing techniques fall short: Room-scale proxies lack portability, wearable robotic arms are energy-intensive and induce friction, and existing hand-scale passive interaction techniques are unsuitable for continuous large-scale renders. In this thesis, we introduce PantographHaptics, a technique which uses the scaling properties of a pantograph to passively render body-scale surfaces. A pantograph is a classical linkage mechanism which can enlarge or shrink designs by coordinating nodes to move in scaled, geometrically similar paths. To our knowledge, no prior work has applied the pantograph mechanism to large-scale immersive haptics. PantographHaptics is a novel method for passively achieving body-scale haptics which uses a pantograph to scale up a small positional constraint into an encounterable midair render. We present the conceptual foundation underpinning of PantographHaptics by describing the operation of the pantograph mechanism and detailing how we apply it for haptics. Then we verify the PantographHaptics technique through two prototypes: HapticLever, a grounded system, and Feedbackpack, a wearable device. We detail the designs, implementations, and technical evaluations of both prototypes, and we highlight the challenges and solutions involved in their development. We evaluate these prototypes with user evaluations, which contribute assessments of their interaction fidelity, investigations of their usability, comparisons of their performance against other haptic modalities, and recorded participant experiences of using the devices. By introducing and verifying PantographHaptics, we show that this novel technique is a viable and promising approach for interactions with large surfaces. By documenting the development of our prototype artifacts and reporting user experiences with the devices, we contribute a foundation for future research.
  • Thumbnail Image
    ItemOpen Access
    Online User Recognition using Social Behavioral Biometric System
    (2022-04) Tumpa, Sanjida Nasreen; Gavrilova, Marina; Zhao, Richard; Ullyot, Michael
    Online Social Networking (OSN) platforms have become an integral part of the daily life of individuals around the world. The uniqueness of social interactions on online social networks draws the attention of cybersecurity research. Social Behavioral Biometric (SBB) systems extract unique patterns from online communication trails and generate digital fingerprints for user identification. This research investigates the impact of users’ vocabulary to conclude whether such features contribute to user authentication. This thesis combines for the first time the textual, contextual and interpersonal communicative information of users in online social networks to develop a social behavioral biometric system. This thesis also pioneers the comparative study of fusion methods in SBB systems. The proposed system achieves 99.25% recognition accuracy and outperforms all prior research on SBB. In addition, the effects of template aging on the individual SBB traits and the overall system have been analyzed first-ever. The experimental results on permanence evaluation demonstrate that the developed system can perform remarkably well despite the template aging effect.
  • Thumbnail Image
    ItemOpen Access
    Person Identification From Audio and Visual Aesthetics
    (2021-11) Sieu, Brandon; Gavrilova, Marina; Ai He, Helen; Zhao, Richard
    In recent years, the trend that allowed for person identification based on behavior rather than physical traits to emerge as a growing research domain. Its application spans areas such as online education, e-commerce, e-communication, human-computer interaction, robotics, and biometric security. The expression of opinions is an example of online behavior, that is commonly shared through the liking of images or music. A person's aesthetic preference involves using a person's sense of fondness to experienced stimulus. This thesis examines for the first time aesthetic preference as a biometric trait. It then establishes the efficacy of a combined multi-modal approach to person identification using aesthetic preference. The potency of this approach is tested on a proprietary dataset.
  • Thumbnail Image
    ItemOpen Access
    Quantifying the Role of Prosody in the Perception of Deception
    (2021-05-05) Rey, Lyndon Thomas McIntosh; Winters, Stephen; Zhao, Richard; Darin, Flynn
    This work investigates the relationship between inflection and perceived honesty in Canadian English, specifically testing whether a terminal rising inflection is perceived as more dishonest than a falling terminal inflection. Canadian English listeners heard pairs of sentence stimuli which differed only in terms of a final falling, neutral, or rising intonation contour and judged which sentence in each pair sounded more “honest”. I found that speech with a rising intonation is perceived as significantly less honest than speech with either flat or falling intonation. Then, I trained an Exemplar model (Johnson, 1997) and a neural network model, which were both able to match listener performance with roughly 60% accuracy. This result is significantly better than chance, but leaves much room for improvement. It provides a realistic view into how intonation clearly influences the perception of honesty, but with it being just one of many factors playing a role in this judgment.
  • Thumbnail Image
    ItemOpen Access
    Single-player to Two-player Knowledge Transfer in Atari 2600 Games
    (2024-11-18) Saadat, Kimiya; Zhao, Richard; Abou-Zeid, Hatem; Aycock, John
    Playing two-player games using reinforcement learning and self-play can be challenging due to the complexity of two-player environments and the potential instability in the training process. It is proposed that a reinforcement learning algorithm can train more efficiently and achieve improved performance in a two-player game by leveraging the knowledge from the single-player version of the same game. This study examines the proposed idea in ten different Atari 2600 environments using the Atari 2600 RAM as the input state. The advantages of using transfer learning from a single-player training process over training in a two-player setting from scratch are discussed, and the results are demonstrated in several metrics, such as the training time and average total reward. Finally, a method for calculating RAM complexity and its relationship to performance after transfer is discussed. Results show that in most cases transferred agent is performing better than the agent trained from scratch while taking less time to train. Moreover, it is shown that RAM complexity can be used as a weak predictor to predict the transfer's effectiveness.
  • Thumbnail Image
    ItemOpen Access
    Theory-guided machine learning in geophysics
    (2021-10-18) Niu, Zhan; Trad, Daniel Osvaldo; lnnanen, Kristopher A.H.; Karchewski, Brandon Anthony James; Zhao, Richard
    Machine learning has become a popular topic in the past decade thanks to the booming in computer hardware and the tools invented. Many successful applications have been made in various subjects in geophysics, including salt body detection, facies recognition and inversion etc. However, the fact that most geophysical theory is well-established sometimes contradicts the black box theory in machine learning when combining methods in the two fields. This thesis will discuss several ways of incorporating well-established knowledge into machine learning by giving a few applications and experiments in geophysics. We will also discuss the limitations and challenges machine learning is facing.
  • Thumbnail Image
    ItemOpen Access
    Using Active Probing by a Game Management AI to Faster Classify Players in Online Video Games
    (2021-06) Eidelberg, Arkady; Jacob, Christian; Denzinger, Jorg; Aycock, John Daniel; Zhao, Richard
    A Game Management AI is a framework to classify players based on their interest of the game. It is different from other work in this area by the fact that it actively manipulates the game state. This encourages the players to act in a certain way (or not), indirectly providing data currently missing to achieve the classification. This is called “Active Probing". The Game Management AI uses two sets of rules. The first contains rules that are intended to represent the knowledge allowing a classification and the second contains rules that indicate which game events can contribute to triggering conditions used in the first rule set. The Game Management AI was evaluated on the role playing game “Realm of Dreams”, a game that was created for this purpose. The experimental evaluation showed that using the active probing by the Game Management AI allows dentification of players highly interested in the game four times faster than such players were identified without active probing.