Recording atomic‐resolution transmission electron microscopy (TEM) images is becoming increasingly routine. A new bottleneck is then analyzing this information, which often involves time‐consuming manual structural identification. A deep learning‐based algorithm for recognition of the local structure in TEM images was developed, which is stable to microscope parameters and noise. The neural network is trained entirely from simulation but is capable of making reliable predictions on experimental images. The method is applied to single sheets of defected graphene, and to metallic nanoparticles on an oxide support.