Cloud computing has rapidly emerged as a model for delivering Internet-based utility computing services. Infrastructure as a Service (IaaS) is one of the most important and rapidly growing models in cloud computing. Scalability, quality of service, optimum utility, decreased overheads, higher throughput, reduced latency, specialised environment, cost-effectiveness, and a streamlined interface are some of the essential elements of cloud computing for IaaS. Traditionally, resource management has been done through static policies, which impose certain limitations in various dynamic scenarios, prompting cloud service providers to adopt data-driven, machine-learning-based approaches. Machine learning is being used to handle various resource management tasks, including workload estimation, task scheduling, VM consolidation, resource optimisation, and energy optimisation, among others. This paper provides a detailed review of machine learning-based resource management solutions. We begin by introducing background concepts of cloud computing like service models, deployment models, and machine learning use in cloud computing. Then we look at resource management challenges in cloud computing, categorise them based on various aspects of resource management types such as workload prediction, VM consolidation, resource provisioning, VM placement and thermal management, review current techniques for addressing these challenges, and evaluate their key benefits and drawbacks. Finally, we propose prospective future research directions based on observed resource management challenges and shortcomings in current approaches for solving these challenges.