With the rapid developments in wireless communication technology, people access Internet applications anytime and anywhere. Recently, academics and industry researchers have been paying increasing attention to the video streaming services over high-speed rail environments. For providing the quality of service of video streaming, we present a novel mobile proxy architecture for improving the system performance over high-speed trains in Long Term Evolution-Advanced networks. The mobile proxy not only saves the wireless bandwidth consumption between the base station and the high-speed train but also reduces the starting delay time of the video. Moreover, we propose a score-based replacement scheme in the memory management of the proxy server based on the feature of the video. Our simulation results demonstrate that the proposed scheme provides a better performance than other existing cache management schemes.