Spectrum opportunities in time, space and power domain are exploited by secondary users in traditional cognitive radio studies. While spectrum opportunities in polarization domain, namely simultaneous utilization of spectrum between primary and secondary system with polarization signal characteristics, is exploited inadequately. With the merits of polarization mismatch, primary system lies in the orthogonal polarized domain of secondary system, and interference to primary system can be mitigated. This work studies the cognitive underlay TV spectrum sharing with the merits of polarization mismatch for TD-LTE system. In this direction, a novel Polarized Underlay Spectrum Sharing (PUSS) scheme is proposed for the coexistence of TD-LTE and DTV system. It aims to characterize the spectrum efficiency of the cognitive radio setup while transmission of Polarization States (PSs) is affected by channel depolarization effect and interference constraint of DTV receiver is taken into account. Transmitting and receiving PSs are optimized by Particle Swarm Optimization with the polarized channel state information and primary PSs. Theoretical analysis and numerical results indicate that the proposed PUSS scheme could enable TD-LTE system share TV spectrum without causing much interference to incumbent DTV communication, while improvement in spectrum efficiency is achieved.