The development of object detection algorithms utilizing convolutional neural networks (CNNs) has led to the proliferation of these methods in daily life, from autonomous vehicles and astronomy to countless applications on mobile devices. Commercialization of unmanned aerial vehicles (UAVs) has resulted in CNNs applied to aerial-based imagery which present unique challenges to modern detection architectures. Aerial-based images typically consist of small objects in large, complex environments that continually change in scale and orientation— challenging deep learning models despite CNNs dominance of object detection competitions. Two-stage detectors such as Mask R-CNN have consistently demonstrated high detection accuracy on benchmark tests by first extracting regions of interest (RoI) likely to contain objects before predicting objects within the RoI; thus avoiding performing predictions across an entire image at once as in single-stage models—an attractive characteristic for detection of objects dwarfed by their backgrounds. Single- and two-stage methods continually underperform on aerial-based data, limiting their applicability to these applications. This thesis evaluates the means of failure of both single- and double-stage detectors on the challenging characteristics of aerial imagery and introduces a training augmentation for both architectures. Mask R-CNN is further improved upon with two modifications to the region proposal network (RPN) to increase the accurate extraction of RoIs when applied to small objects and objects of changing scales. Augmentations and improvements are demonstrated on two challenging aerial-based datasets.