In budding yeast, which possesses simple point centromeres, we discovered that all of its centromeres express long noncoding RNAs (cenRNAs), especially in S phase. Induction of cenRNAs coincides with CENP-A^Cse4 loading time and is dependent on DNA replication. Centromeric transcription is repressed by centromere-binding factor Cbf1 and histone H2A variant H2A.Z^Htz1. Deletion of CBF1 and H2A.Z^HTZ1 results in an up-regulation of cenRNAs; an increased loss of a minichromosome; elevated aneuploidy; a down-regulation of the protein levels of centromeric proteins CENP-A^Cse4, CENP-A chaperone HJURP^Scm3, CENP-C^Mif2, Survivin^Bir1, and INCENP^Sli15; and a reduced chromatin localization of CENP-A^Cse4, CENP-C^Mif2, and Aurora B^Ipl1. When the RNA interference system was introduced to knock down all cenRNAs from the endogenous chromosomes, but not the cenRNA from the circular minichromosome, an increase in minichromosome loss was still observed, suggesting that cenRNA functions in trans to regulate centromere activity. CenRNA knockdown partially alleviates minichromosome loss in cbf1Δ, htz1Δ, and cbf1Δ htz1Δ in a dose-dependent manner, demonstrating that cenRNA level is tightly regulated to epigenetically control point centromere function.