With the recent establishment of atomically precise nanochemistry, capabilities toward programmable control over the nanoparticle size and structure are being developed. Advances in the synthesis of atomically precise nanoclusters (NCs, 1–3 nm) have been made in recent years, and more importantly, their total structures (core plus ligands) have been mapped out by X‐ray crystallography. These ultrasmall Au nanoparticles exhibit strong quantum‐confinement effect, manifested in their optical absorption properties. With the advantage of atomic precision, gold‐thiolate nanoclusters (Au_n(SR)_m) are revealed to contain an inner kernel, Au–S interface (motifs), and surface ligand (‐R) shell. Programming the atomic packing into various crystallographic structures of the metal kernel can be achieved, which plays a significant role in determining the optical properties and the energy gap (E_g) of NCs. When the size increases, a general trend is observed for NCs with fcc or decahedral kernels, whereas those NCs with icosahedral kernels deviate from the general trend by showing comparably smaller E_g. Comparisons are also made to further demonstrate the more decisive role of the kernel structure over surface motifs based on isomeric Au NCs and NC series with evolving kernel or motif structures. Finally, future perspectives are discussed.