Levels of the adrenal hormones dehydroepiandrosterone (DHEA), its sulfate (DHEAS), and testosterone, have all been linked to behavior and mental health during adrenarche, and preclinical studies suggest that these hormones influence brain development. However, little is known about how variation in these hormones is associated with white matter structure during this period of life. The current study aimed to examine associations between DHEA, DHEAS, and testosterone, and white matter microstructure during adrenarche. To avoid the confounding effect of age on hormone levels, we tested these associations in 87 children within a narrow age range (mean age 9.56 years, SD = 0.34) but varying in hormone levels. All children provided saliva samples directly after waking and completed a diffusion-weighted MRI scan. Higher levels of DHEA were associated with higher mean diffusivity (MD) in a widespread cluster of white matter tracts, which was partially explained by higher radial diffusivity (RD) and partially by higher axial diffusivity (AD). In addition, there was an interaction between DHEA and testosterone, with higher levels of testosterone being associated with higher fractional anisotropy (FA) and lower MD and RD when DHEA levels were relatively high, but with lower FA and higher MD and RD when DHEA levels were low. These findings suggest that relatively early exposure to DHEA, as well as an imbalance between the adrenal hormones, may be associated with alterations in white matter microstructure. These findings highlight the potential relevance of adrenarcheal hormones for structural brain development.