In this work, we compare the effect of incorporating selenophene versus thienothiophene spacers into pentacyclic lactam-based conjugated polymers for organic solar cells. The two cyclic lactam-based copolymers were obtained via a new synthetic method for the lactam moiety. Selenophene incorporation results in a broader and red-shifted optical absorption while retaining a deep highest occupied molecular orbital level, whereas thienothienophene incorporation results in a blue-shifted optical absorption. Additionally, grazing-incidence wide angle X-ray scattering data indicates edge- and face-on solid state order for the selenophene-based polymer as compared to the thienothiophene-based polymer, which orders predominantly edge-on with respect to the substrate. In polymer : PC71BM bulk heterojunction solar cells both materials show a similar open-circuit voltage of ∼0.80–0.84 V, however the selenophene-based polymer displays a higher fill factor of ∼0.70 vs. ∼0.65. This is due to the partial face-on backbone orientation of the selenophene-based polymer, leading to a higher hole mobility, as confirmed by single-carrier diode measurements, and a concomitantly higher fill factor. Combined with improved spectral coverage of the selenophene-based polymer, as confirmed by quantum efficiency experiments, it offers a larger short-circuit current density of ∼12 mA cm−2. Despite the relatively low molecular weight of both materials, a very robust power conversion efficiency ∼7% is achieved for the selenophene-based polymer, while the thienothiophene-based polymer demonstrates only a moderate maximum PCE of ∼5.5%. Hence, the favorable effects of selenophene incorporation on the photovoltaic performance of pentacyclic lactam-based conjugated polymers are clearly demonstrated.