Unmanned aerial vehicles (UAVs) have attracted much attention for civil and military uses because of their high mobility and adaptable deployment capabilities in open spaces. They facilitate agile communications and ubiquitous connectivity. UAVs benefit from dominant line-of-sight communication links but are more susceptible to adversary eavesdropping attacks. Since upper-layer cryptography methods may be insufficient, physical-layer security (PLS) is an attractive alternative. PLS capitalizes on the inherent randomness of wireless channels to improve information confidentiality, particularly in UAV systems. This article provides a comprehensive overview of PLS in the context of UAV systems, examining various communication channels, including ground-to-air (G2A), ground-to-ground (G2G), air-to-ground (A2G), and air-to-air (A2A). First, we give a general review of UAV communications, emphasizing its functions as user equipment, base station (BS), and mobile relay. Following that, we provide an overview of PLS and its forms. The survey organizes the most advanced PLS techniques against passive wireless eavesdropping regarding the channels they are designed to protect. These schemes are further categorized into optimization and secrecy performance analysis. We examine technical aspects within each sub-category and classify core contributions based on approaches and objectives to improve secrecy performance. Moreover, we added summary tables of the identified schemes considering optimization and secrecy performance analysis domains for each category, providing an in-depth understanding of their technical aspects. In addition, we discuss open research issues and identify future research directions to address evolving threats and requirements in UAV communications to improve information confidentiality.