The field of adult-centric brain magnetic resonance (MR) image analysis is well-established, yet research on the pediatric population remains limited. Pediatric brains undergo rapid development and present complex structural and functional attributes, necessitating a customized approach for MR processing and machine learning (ML) analysis. This study introduces a comprehensive processing framework tailored for pediatric (2 to 8 years old) T1-weighted structural brain MR scans, considering several processing steps including skull stripping (SS), registration, bias correction, normalization, resizing, and tissue segmentation, utilizing neuroimaging tools and Python-based libraries (FSL, Freesurfer, SimpleITK, SciPy, etc.). For SS, FSL-BET and SynthStrip are recommended. For registration, an age-specific template selection is recommended to correlate the characteristics of pediatric brains. Bias correction and normalization are desirable for non-uniform and distinct voxel intensities respectively. Depending on the post-processing requirements, resizing and tissue segmentation can be optional, where FSL-FAST was preferred for segmentation under non-artifact conditions. This study incorporated unexposed controls to develop the framework, while prenatal alcohol exposure (PAE) scans were utilized to validate the generalizability of the framework, confirmed by visual inspection as a qualitative approach. Furthermore, deep learning (DL) is applied by utilizing the pre-trained simple fully convolutional network (SFCN) as a transfer learning (TL) approach for extracting features from the processed scans and a newly trained classifier to distinguish between unexposed and PAE participants. The classifier secured the highest sensitivity of 88.47% with 85.04% average accuracy on testing data when considering a balanced dataset with augmentation for both classes. Moreover, this study performed a preliminary explainability analysis using the Grad-CAM method, highlighting various brain regions including corpus callosum, cerebellum, pons, and white matter (WM) as the most important features in the model’s decision-making process. Despite the challenges of constructing DL models for pediatric populations due to the brain’s rapid development, motion artifacts, and insufficient data, this work highlights the potential of TL where data is limited. Furthermore, this study emphasizes preserving a balanced dataset for fair classification and using explainability to clarify the model’s predictions.