Plants harbor a diverse microbiome existing as bacterial populations on the leaf surface (the phyllosphere) and within plant tissues (endophytes). The composition of this microbiome has been largely unexplored in fresh produce vegetables, where studies have tended to focus on pathogen detection and survival. However, the application of next-generation 16S rRNA gene sequencing approaches is beginning to reveal the diversity of this produce-associated bacterial community. In this article we review what is known about the composition of the microbiome of fresh produce vegetables, placing it in the context of general phyllosphere research. We also demonstrate how next-generation sequencing can be used to assess the bacterial assemblages present on fresh produce, using fresh herbs as an example. That data shows how the use of such culture-independent approaches can detect groups of taxa (anaerobes, psychrophiles) that may be missed by traditional culture-based techniques. Other issues discussed include questions as to whether to determine the microbiome during plant growth or at point of purchase or consumption, and the potential role of the natural bacterial community in mitigating pathogen survival.