This thesis presents a comprehensive exploration of autonomous exploration and mapping in unknown environments, emphasizing the importance of minimizing exploration time while ensuring high-quality mapping results. It introduces a classical coordinated autonomous exploration strategy inspired by Market-Based task allocation and Ant Colony Optimization techniques. Compared to simpler Market-Based methods, this approach improves efficiency by reducing overlap between robots. The study utilizes a 2D simulation environment to collect training data efficiently, facilitating the evaluation of the proposed methods’ efficiency, adaptability, and generalizability through various experiments. Furthermore, the thesis develops intelligent single-robot autonomous techniques using advanced Deep Reinforcement Learning (DRL) algorithms. Notably, it adopts risk-sensitive strategies, in contrast to traditional risk-neutral approaches in DRL, aiming to enhance exploration efficiency, defined as the time required to complete 95% of the map. The best intelligent policy achieved a significant improvement in exploration efficiency compared to the utility-based classical method. Despite selecting the appropriate DRL algorithm and fine-tuning it accordingly, a comprehensive study and series of experiments are designed to investigate the effects of various state, action, and reward spaces. Moreover, different external features, such as incorporating the Intrinsic Curiosity Module (ICM) or integrating Long-Short Term Memory (LSTM) layers, are considered to further enhance autonomous exploration efficiency. The results demonstrate a significant improvement in autonomous exploration efficiency compared to well-known classical single-robot exploration policies, validating the effectiveness of the suggested novel autonomous exploration strategies. The codes developed for this thesis are available in the Intelligent Dynamics and Control Lab GitHub repository: https://github.com/IDCL-UCalgary