Hospital beds are an essential part of delivering medical services to the patients. Due to the hospital bed demand’s stochastic nature, it is hard to predict future needs and devise an appropriate augmentation scheme. In this work, we consider Bangladesh as a test case where hospital beds are inadequate, and a sudden surge in demand can cause a massive loss of lives and wealth. We propose a deep reinforcement learning (RL) based policy for hospital capacity up-gradation schedule. The deep RL agent monitors population growth and current bed capacity and recommends the optimal number of beds for future inclusion. We utilize the state-of-the-art machine learning (ML) algorithm, Advantage Actor-Critic (A2C), to minimize the cumulative cost. This policy outperforms the straight forward strategies: fixed upgrade and complain based upgrade.