Residual stress detection plays an important role in researching, designing and manufacturing of engineering structure and complex equipment. Nowadays, the neutron diffraction methods for measuring residual stress are utilized to predict the fatigue life of components and materials. However, time-consuming and high precision are two main challenges to calculate residual stress by fitting to neutron diffraction data. In this paper, a three-layered peak shape fitting artificial neural network (PSF-NN) model is constructed to get peak shape by fitting to neutron diffraction data. We design an optimized loss function to update weights and bias iteratively by Levenberg Marquardt back-propagation algorithm. Then peak shapes can be obtained by correlate diffraction intensity integration to diffraction angle. We apply the PSF-NN model to neutron residual stress spectrometer at China Advanced Research Reactor. Experimental results show that PSF-NN outperforms Decision Tree, Levenberg Marquardt algorithm, Random Forest and Gradient Descent in RMSE and R2, where RMSE improving at least 31%, and R ² improving at least 8%.