Transonic buffet is undesirable because it causes vibration, and constraining buffet is crucial in transonic wing design. However, there is still a lack of accurate and efficient buffet formulation to impose the constraint. This work proposes a physics-based data-driven buffet analysis model generalizable for airfoil and wing shapes. The model is trained with data obtained from two-dimensional airfoils in a physics-based manner to extend it to buffet analyses of three-dimensional wings. Specifically, the model takes the pressure and friction distributions as inputs to discover the key physics (shock waves and flow separation) of transonic buffeting, rather than using shape and flow parameters as the input. High-quality sample airfoils are used and a mixture model of convolutional neural networks is proposed to improve accuracy. The model exhibits a mean absolute error of 0.05 deg in buffet factor prediction of 14,886 unseen testing data. Buffet boundary predictions using the model compare well with the reference results (a lift-curve break method) for various airfoils and wings. Wing shape optimization using the model appropriately considers buffet constraints, leading to an optimized wing with lower drag (by 1.7 counts) than that obtained by the state-of-the-art method. These results demonstrate the effectiveness of the proposed physics-based data-driven buffet analysis approach. The proposed method is a promising alternative to address other complex off-design constraints in aircraft design.