Estates, especially those of public security-related companies and institutes, have to protect their privacy from adversary unmanned aerial vehicles (UAVs). In this paper, we propose a reinforcement learning-based control framework to prevent unauthorized UAVs from entering a target area in a dynamic game without being aware of the UAV attack model. This UAV control scheme enables a target estate to choose the optimal control policy, such as jamming the global positioning system signals, hacking, and laser shooting, to expel nearby UAVs. A deep reinforcement learning technique, called neural episodic control, is used to accelerate the learning speed to achieve the optimal UAV control policy, especially for estates with a large area, against complicated UAV attack policies. We analyze the computational complexity for the proposed UAV control scheme and provide its performance bound, including the risk level of the estate and its utility. Our simulation results show that the proposed scheme can reduce the risk level of the target estate and improve its utility against malicious UAVs compared with the selected benchmark scheme.