Convolutional neural network (CNN) can predict chronological age accurately based on MRI. However, most studies use single feature to predict brain age in healthy individuals, ignoring the adding information of multiple sources. Here, we developed an interpretable 3D CNN model to predict brain age based on a large, heterogeneous dataset (N = 1464). Comparing with state-ofthe-art methods, our prediction framework has the following improvements. First, our model utilized multiple 3D features derived from T1w data as inputs, and reduced the mean absolute error (MAE) of age prediction to 3.32 years and improved Pearson's r to 0.96 on 154 healthy controls (HCs). Strong generalizability of our model was also validated across different centers. Second, network occlusion sensitivity analysis (NOSA) was adopted to interpret our model and capture the age-specific pattern of brain aging. Regions contributing significantly to brain age were different for HCs and patients with mild traumatic brain injury (mTBI) in different life stages but all within the subcortical areas throughout the lifespan. Left hemisphere was confirmed to be more contributed in the brain age prediction throughout the lifespan. Our research showed that increased brain predicted age gap (brain-PAG) in 98 acute mTBI patients was highly correlated with cognitive impairment and higher level of plasma neurofilament light, a marker of neurodegeneration. The higher brain-PAG also showed a longitudinal and persistent nature in patients with follow-up examination. The interpretable framework might also provide hope for testing the performance of related drugs or treatments.