Dynamic Locomotion for Humanoid Robots Via Deep Reinforcement Learning

Garza Bayardo, Rodrigo Alberto

Dynamic Locomotion for Humanoid Robots Via Deep Reinforcement Learning

dc.contributor.advisor	Ramirez-Serrano, Alejandro
dc.contributor.author	Garza Bayardo, Rodrigo Alberto
dc.contributor.committeemember	Lee, Jihyun
dc.contributor.committeemember	Nittala, Aditya Shekhar
dc.date	2022-11
dc.date.accessioned	2022-09-28T15:19:25Z
dc.date.available	2022-09-28T15:19:25Z
dc.date.issued	2022-09
dc.description.abstract	A longstanding goal in legged-mobile robotics is to enable robots to learn robust control policies capable of expanding their locomotion abilities to be competent and efficient when moving on a priori unknown uneven terrains. Traditional control techniques lack the generalizability required for legged robots to locomote outside a controlled lab environment in terrains with diverse difficult-to-model uncertainties (e.g., friction coefficients, compliance/deformation, etc.). The approach presented in this thesis combines Deep Reinforcement Learning (Deep RL) with motion capture (MoCap) data to train a biped simulated agent (i.e., humanoid robot) to perform a rich repertoire of diverse skills (e.g., walking, crawling, running, climbing steps, etc.). The method begins by creating reference motion clips with the robot’s morphology for desired locomotion gaits. This is achieved by applying motion retargeting techniques to adapt MoCap clips (taken from humans) to the humanoid system. Subsequently, these adapted reference clips are used as inputs to a new Deep RL architecture. Such architecture uses the Proximal Policy Optimization (PPO) algorithm to train the simulated agent to perform the gait of interest on randomized domains that vary with each iteration. After training, the generated control policy is transferred to a real robot for testing and fine-tuning. The benefits of this approach include i) reducing the training time typically required in multi-legged systems, ii) avoiding exposing the robot and the trainer (human operator) to the tedious and time-consuming initial iterations of the learning process where mistakes are likely to happen, and iii) generalizability – the ability to employ the training model on virtually every available legged mobile robot. Thus, the proposed approach enables robots to locomote in real-world settings. The results are demonstrated through simulation and experimentally tested on a Robotis’ THORMANG 3.0 humanoid robot.	en_US
dc.identifier.citation	Garza Bayardo, R. A. (2022). Dynamic locomotion for humanoid robots via deep reinforcement learning (Master's thesis, University of Calgary, Calgary, Canada). Retrieved from https://prism.ucalgary.ca.	en_US
dc.identifier.uri	http://hdl.handle.net/1880/115311
dc.identifier.uri	https://dx.doi.org/10.11575/PRISM/40317
dc.language.iso	eng	en_US
dc.publisher.faculty	Schulich School of Engineering	en_US
dc.publisher.institution	University of Calgary	en
dc.rights	University of Calgary graduate students retain copyright ownership and moral rights for their thesis. You may use this material in any way that is permitted by the Copyright Act or through licensing that has been assigned to the document. For uses that are not allowable under copyright legislation or licensing, you are required to seek permission.	en_US
dc.subject	Dynamic Locomotion	en_US
dc.subject	Reinforcement Learning	en_US
dc.subject	Humanoids	en_US
dc.subject.classification	Education--Technology	en_US
dc.subject.classification	Artificial Intelligence	en_US
dc.subject.classification	Robotics	en_US
dc.title	Dynamic Locomotion for Humanoid Robots Via Deep Reinforcement Learning	en_US
dc.type	master thesis	en_US
thesis.degree.discipline	Engineering – Mechanical & Manufacturing	en_US
thesis.degree.grantor	University of Calgary	en_US
thesis.degree.name	Master of Science (MSc)	en_US
ucalgary.item.requestcopy	true	en_US