Reconfigurable Intelligent Surface (RIS) has recently emerged as an enabling technology to enhance reliability and overcome blockage in future heterogeneous wireless networks (HetNets). Adjusting amplitudes and phases of the RIS elements to achieve such goals is a major challenge. In this paper, we study the problem of network rate control to achieve users (UEs) fairness and smallcells (SCs) load balancing in multi-cell RIS-assisted multiple-input single-output (MISO) HetNets. We consider dual-connectivity UEs that can simultaneously connect to mmWave-operating SCs and sub-6GHz-operating RIS-assisted macrocell (MC), where RISs are mainly deployed to enhance sub-6GHz signal reception and mitigate interference. Then, we formulate an optimization problem whose objective is to jointly control the active beamforming vectors of SCs and MC on the one hand and the passive beamforming vectors of RISs on the other hand to maximize UEs fairness and network load balancing. Due to the high complexity of the formulated problem, we propose a novel multi-task deep reinforcement learning (MTDRL) model based on the Deep Deterministic Policy Gradient (DDPG) algorithm to solve the problem and learn system dynamics. Through proper definitions of network tasks and their main elements, we show via simulations that our proposed MTDRL-based model ensures fair distribution of rates within UEs and SCs and that it outperforms key benchmarks.