This article is concerned with high-precision and fast velocity-tracking control for the harmonic drive-based gimbal system of control moment gyro (CMG). As a major challenge in the controller design, the simultaneous presence of multiple matched and mismatched disturbances should be carefully addressed. Specifically, the rotor imbalance inherent in the CMG will cause vibration of the output torque that significantly degrades the tracking performance. On the other hand, the integration of harmonic drives will lead to nonlinear torsional stiffness and introduce additional frictions into the control system. Furthermore, practical tasks such as fast attitude maneuver have posed higher requirement on the rapidity of the antidisturbance control schemes. To overcome the aforementioned difficulties, we propose a finite-time refined antidisturbance velocity tracking controller, which consists of a finite-time harmonic disturbance observer to compensate for the imbalance vibration, an adaptive finite-time extended state observer to handle the remaining disturbances with unknown rate of change, and a finite-time controller to ensure fast convergence of the tracking error. Both numerical simulation and hardware-in-the-loop experimental results show that the proposed scheme outperforms the existing methods in terms of accuracy and rapidity.