Deep Reinforcement Learning (DRL) algorithms have been recently proposed to solve dynamic Radio Resource Management (RRM) problems in 5G networks. However, the slow convergence experienced by traditional DRL agents puts many doubts on their practical adoption in cellular networks. In this paper, we first discuss the need to have accelerated DRL algorithms. We then analyze the exploration behavior of various state-of-the-art DRL algorithms for slice resource allocation, and compare it with the traditional 5G Radio Access Network (RAN) slicing baselines. Finally, we propose a transfer learning-accelerated DRL-based solution for slice resource allocation. In particular, we tackle the challenge of slow convergence by transferring the policy learned by a DRL agent at an expert base station (BS) to newly deployed agents at target learner BSs. Our approach shows a remarkable reduction in convergence time and a significant performance improvement compared with its non-accelerated counterparts when tested against multiple traffic load variations.