Geometry and radiometry based enhancement for 3D reconstruction from imagery
Date
2020-05-01
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
3D reconstruction from imagery is making significant contributions in a wide array of applications, including but not limited to heritage documentation, 3D modelling of the human body, and digital elevation model (DEM) generation. The importance of imagery-based 3D reconstruction is a result of being a low-cost alternative to active sensors, such as laser scanners. However, it is unfortunate that the point cloud generated from active sensors most likely will be more accurate than a point cloud generated from stereo imagery or structure from motion (SFM). Besides the scale ambiguity, which can be resolved using the associated navigation sensors like the inertial measurement unit (IMU) and the GPS receiver, there are other issues that limit the quality of imagery-based 3D reconstruction. Reconstruction from imagery involves several stages; and in most of these stages, the outcome is based on estimation procedures, which means that the whole process is more probabilistic than deterministic. Furthermore, noise and radiometric errors in the input images can lead to errors in feature detection and matching and can dramatically affect the quality of the orientation solution. The effects of noise and radiometric errors are not limited to the deterioration of the orientation parameters, however, and also can have a significant impact on dense matching, the stage at which the point cloud is generated. In this dissertation, an enhancement workflow for imagery-based 3D reconstruction is proposed. The proposed workflow consists of three approaches, each of which tackles one common problem in the process of 3D reconstruction. The first approach aims at enhancing the feature detection and matching in image pairs in which both the geometric and radiometric information of the image pairs are fused to find an accurate and robust set of matches. In the proposed approach, a small subset of matches is used to estimate an initial solution for the homography and fundamental matrix. These two entities constrain the point correspondence to a specific area. Then, a radiometric correlation is applied to find the desired set of matches in a recursive way. In the second approach, the noise in disparity maps is removed while image segmentation is being performed at the same time. The main target of the proposed approach is to segment both the disparity map and the original images based on both the geometric and radiometric constraints. First, the disparity map is roughly segmented using its grey-level histogram and basic region growing/thresholding algorithms. Then, a homography relation between the image pairs, based on the segmented disparity map, is constructed. This allows labelling each connected region in the image pair as a segment. The segmentation process is further enhanced using colour edge detectors and spatial and frequency domain filters. The third approach enhances the dense matching process using the segmented disparity map. The segmented disparity map obtained from the second approach is considered to be almost noise-free as a result of applying the homography-based segmentation and that all invalid disparity points have been detected and rejected. Therefore, the proposed approach aims to replace those points with more accurate points based on the local homography and the colour constraints imposed on the image pair. This approach is considered a post-processing approach, and it is therefore assumed that the disparity map is already known. The outcome of the proposed approaches is a segmented, more accurate, and well-defined 3D point cloud with less noise compared to the current 3D reconstruction approaches.
Description
Keywords
3D reconstruction, Segmentation, Feature matching, Dense matching, Computer vision, Photogrammmetry
Citation
Mohammed, H. M. M. (2020). Geometry and radiometry based enhancement for 3D reconstruction from imagery (Doctoral thesis, University of Calgary, Calgary, Canada). Retrieved from https://prism.ucalgary.ca.