Two thiophene backbone-based hole-transporting polymers, namely, poly[(2,2′-bithiophene-5,5′-diyl)-alt-(5,7-bis(2-butyloctyl)-4H,8H-benzo[1,2-c:4,5-c′]dithiophene-4,8-dione-1,3-diyl)] (PT2-BDD) and poly[(3,3′-difluoro-[2,2′-bithiophene]-5,5′-diyl)-alt-(5,7-bis(2-butyloctyl)-4H,8H-benzo[1,2-c:4,5-c′]dithiophene-4,8-dione-1,3-diyl)] (PF2-BDD), were prepared by using electron-withdrawing pendant groups such as ketone and fluorine moieties. They both exhibited a planar backbone with efficient π conjugation, which is suitable for hole transport in organic thin-film transistors (OTFTs). However, the fluorinated one (i.e., PF2-BDD), despite its perfect backbone planarity and strong intra- and intermolecular interactions, could not enhance the OTFT performance; due to its solvent resistance, electron negativity, and random orientation, PF2-BDD showed 10 times lower hole mobility than the non-fluorinated polymer (i.e., PT2-BDD). Nonetheless, the two-dimensional grazing incidence X-ray diffraction and temperature-dependent absorption spectra of the synthesized polymers provided crucial information to understand the relationship between their hole transport behavior and molecular structure.