A significant research area in medical imaging analysis is digital mammography breast cancer detection in the early stage. For breast mass classification into the benign or malignant category, an enhanced automated computer-aided diagnosis (CAD) model is suggested in this work, enabling radiologists to identify breast diseases correctly in less time. First, a fast discrete curvelet transform with wrapping (FDCT-WRP) is deployed to extract the curve-like features and create a feature set. Then, a combined feature reduction strategy called principal component analysis (PCA) and linear discriminant analysis (LDA) is used to produce more relevant and reduced feature sets. Finally, a new enhanced learning algorithm called MODPSO-ELM incorporates modified particle swarm optimization (MODPSO) and an extreme learning machine (ELM) proposed for the classification task. In the MODPSO-ELM algorithm, MODPSO is utilized to optimize the hidden node parameters (input weights and hidden biases) of single-hidden-layer feedforward neural networks (SLFN) and analytically determined the output weight. The proposed CAD model has been evaluated on three standard datasets with a 10 × k-fold stratified cross-validation (SCV) test. It is found from the experiment that the suggested CAD model yields the best outcome for the MIAS dataset and obtains an accuracy of 98.94% and 98.76% for DDSM and INbreast datasets, respectively. The experimental results indicate that the proposed model is superior to other state-of-the-art models with a substantially reduced number of features with better classification accuracy.