There have been few large-scale studies on the relationship between smoking and gut microbiota. We investigated the relationship between smoking status and the composition of gut microbiota. This was a population-based cross-sectional study using Healthcare Screening Center cohort data. A total of 758 men were selected and divided into three groups: never (n = 288), former (n = 267), and current smokers (n = 203). Among the three groups, there was no difference in alpha diversity, however, Jaccard-based beta diversity showed significant difference (p = 0.015). Pairwise permutational multivariate analysis of variance (PERMANOVA) tests between never and former smokers did not show a difference; however, there was significant difference between never and current smokers (p = 0.017) and between former and current smokers (p = 0.011). Weighted UniFrac-based beta diversity also showed significant difference among the three groups (p = 0.038), and pairwise PERMANOVA analysis of never and current smokers showed significant difference (p = 0.01). In the analysis of bacterial composition, current smokers had an increased proportion of the phylum Bacteroidetes with decreased Firmicutes and Proteobacteria compared with never smokers, whereas there were no differences between former and never smokers. In conclusion, gut microbiota composition of current smokers was significantly different from that of never smokers. Additionally, there was no difference in gut microbiota composition between never and former smokers.