With the increasing demand for solar energy, accurate, reliable, and efficient parameters extraction of photovoltaic models is becoming more significant and difficult. Accordingly, a more accurate and robust algorithm is continuously needed for this problem. To this end, a classified perturbation mutation based particle swarm optimization algorithm is proposed in this paper. During each generation of the proposed algorithm, the performance of each updated personal best position is evaluated and quantified to be a high-quality or low-quality. Then, for the high-quality personal best position, a mutation strategy with smaller perturbation is developed to enhance the local search ability within the promising search area. For the low-quality personal best position, a bigger perturbation mutation strategy is designed to explore different regions for improving the population diversity. Furthermore, the damping bound handling strategy is employed to mitigate the issue of falling into local optimal. The effectiveness of the proposed algorithm is evaluated by extracting parameters of five different photovoltaic models, and also tested on photovoltaic models under different conditions. Experiment results comprehensively demonstrate the superiority of the proposed algorithm compared with other well-established parameters extraction methods in terms of accuracy, stability, and rapidity.