Implementing an efficient control strategy for heating, ventilation, and air conditioning (HVAC) systems can lead to improvements in both energy efficiency and thermal performance in buildings. As HVAC systems and buildings are complicated dynamic systems, the effectiveness of both data-driven and model-based control methods has been widely investigated by researchers. However, the main challenges that impede the practical application of model-based methods in real buildings are their reliance on the precision of control-oriented models and the dependence of data-based systems on the quantity and quality of input–output data. The objectives of this study are: (1) To present an overview of the prevalent thermal modelling strategies used as control-oriented models or virtual environments in model-based and data-based control methods, addressing the main requirements of thermal models; (2) the state-of-the-art of MPC and RL control techniques; (3) the data requirements for thermal models. The findings emphasise the need for unified guidelines to validate and verify the proposed control methods, ensuring their practical implementation in real buildings. Moreover, the inclusion of occupancy forecasts in models presents challenges due to the intricate nature of accurately predicting human behaviour, occupancy patterns, and their effects on thermal dynamics. Balancing thermal comfort and energy efficiency in HVAC systems with a supervisory controller remains a difficult task, but combining data-driven and physics-based models can help overcome challenges. Further research is needed to compare the effectiveness of MPC and RL approaches, and accurately measuring the impact of human behaviour and occupancy remains a significant obstacle.