A hybrid optimization methodology is presented for the probabilistic finite element model updating of structural systems. The model updating process is formulated as an inverse problem, analyzed by Bayesian inference, and solved using a hybrid optimization algorithm. The proposed hybrid approach is a combination of a modified artificial bee colony (MABC) algorithm and the Broyden‐Fletcher‐Goldfarb‐Shanno (BFGS) method. The MABC includes four modifications compared to the standard ABC algorithm, which basically improve the global convergence of ABC in the solution phases of initialization, updating, selection, and rebirth. The BFGS is inserted to improve the finer solution search ability aiming at a higher solution accuracy. In brief, a probabilistic framework based on Bayesian inference is first derived so to get a regularized objective function for optimization. Then the proposed MABC‐BFGS algorithm is applied to determine the unknown system parameters by minimizing the newly defined objective function. System parameters as well as the prediction error covariance are updated iteratively in the optimization process. Posterior distributions of the identified system parameters are determined using a weighted sum of Gaussian distributions. Finally, the effectiveness of the proposed approach is illustrated by the numerical data sets of the Phase I IASC‐ASCE benchmark model and the experimental data sets of a three‐storey frame structure (from the Los Alamos National Laboratory (LANL), New Mexico, United States).