Augmented Reality and Virtual Reality are one of the key advances in technology in the last decade. Their usage is rapidly increasing across various contexts. The COVID-19 pandemic and the pressing need for remote learning tools that imitate as close as possible real training environments motivated more research and investigation of AR/VR tools. We present in this work a review of the applications of Augmented Reality (AR) and Virtual Reality (VR) in electrical engineering education. We apply a methodological review of all available publications in this area and classify them according to the application. We ask key research questions regarding these publications. We present recommendations for researchers and educators who want to apply these promising technologies.