Credit scoring is a statistical technique that guides financial institutions to make informed decisions regarding the extension of loans to customers based on cautious examination of their historical records with the intent of reducing the organization’s operational costs and eliminate potential risks. Irrelevant attributes often degrade the classification accuracy, thus feature selection can help in dealing efficaciously with large datasets. It has been well established based on numerous studies that heterogeneous ensemble-based models have unparalleled performance among several mathematical and Artificial Intelligence-based techniques devised for the issue. This paper proposes a novel approach namely Multi-Level Classification and Cluster based Ensemble (MLCCE) that incorporates the strengths of both feature selection and ensemble-based classification. MLCCE uses the attribute dependency-based feature selection scheme followed by multi-level classification. Finally the model utilizes Particle Swarm Optimization based clustering followed by a weighted combination that corresponds to the performance of the individual classifier in different spatial regions of data. During performance evaluation, MLCCE has shown remarkable results on both the benchmark credit scoring datasets—Australian and German dataset as compared to other ensemble-based methods.