Detection and assessment of cracks in civil engineering structures such as roads, bridges, dams and pipelines are crucial tasks for maintaining the safety and cost-effectiveness of those concrete structures. With the recent advances in machine learning, the development of ANN- and CNN-based algorithms has become a popular approach for the automated detection and identification of concrete cracks. However, most of the proposed models are trained on images taken in ideal conditions and are only capable of achieving high accuracy when applied to the concrete images devoid of irregular illumination conditions, shadows, shading, blemishes, etc. An overview of challenges related to the automatic detection of concrete cracks in the presence of shadows is presented in this paper. In particular, difficulties associated with the application of deep learning-based methods for the classification of concrete images with shadows are demonstrated. Moreover, the limitations of the shadow removal techniques for the improvement of the crack detection accuracy are discussed.