Cognitive Internet of Vehicles (CIoV) can improve spectrum utilization by accessing the spectrum licensed to primary user (PU) under the premise of not disturbing the PU's transmissions. However, the traditional static spectrum access makes the CIoV unable to adapt to the various spectrum environments. In this paper, a reinforcement learning based dynamic spectrum access scheme is proposed to improve the transmission performance of the CIoV in the licensed spectrum, and avoid causing harmful interference to the PU. The frame structure of the CIoV is separated into sensing period and access period, whereby the CIoV can optimize the transmission parameters in the access period according to the spectrum decisions in the sensing period. Considering both detection probability and false alarm probability, a Q-learning based spectrum access algorithm is proposed for the CIoV to intelligently select the optimal channel, bandwidth and transmit power under the dynamic spectrum states and various spectrum sensing performance. The simulations have shown that compared with the traditional non-learning spectrum access algorithm, the proposed Q-learning algorithm can effectively improve the spectral efficiency and throughput of the CIoV as well as decrease the interference power to the PU.