5G ultra-dense network can get more cell splitting gain through high-density base station deployment, which has higher spectral efficiency and capacity than LTE. However, the increase of base station density makes the co-channel interference between adjacent cells more serious, and continuous movement of users will cause a lot of handover operations, which will bring a lot of signaling interaction, greatly increase the network burden, and thus reduce the throughput of the network as well as affecting user experience. Considering that the traditional mobility management methods cannot meet the deployment requirements of future ultra-dense wireless networks, a handover management scheme based on residence time prediction is proposed by using the stochastic geometric mathematical model and the derivation of handover triggering probability and average user throughput. The simulation results show that the scheme proposed in this paper has better comprehensive performance than conventional scheme and handover skipping scheme in terms of the average user throughput under different handover decision threshold and base station density.