Structural health monitoring (SHM) is usually focused on damage detection (eg, Yes/No) or approximate estimation of damage size. Any additional details of the damage such as configuration, shape, networking, geometrical statistics, etc., are often either ignored or significantly simplified during SHM characterization. These details, however, can be extremely important for understanding of damage severity and estimations of follow-up damage growth risk. To avoid expensive human participation and/or over-conservative SHM decisions, solutions of computational recognition for damage characterization are needed. Autonomous SHM from visual data is one of the significant challenges in the field of structural prognostics and health monitoring (PHM). The main shortcomings of the image-based PHM algorithms arise from the lack of robustness and fidelity to handle the variability of environment and nature of damage types. In recent times, deep learning has drawn huge amount of traction in the field of machine learning and visual pattern recognition due to its superior performance compared to the state-of-the-art techniques. This paper proposes to formulate and apply a deep learning technique to characterize the damage in the form of cracks on a composite material. The deep learning architecture is constructed by multi-layer neural network that is based on the fundamentals of unsupervised representational learning theory. The robustness and the accuracy of the approach is validated on an extensive set of real image data collected via applying variable load conditions on the structure. The paper has shown a high characterization accuracy over a wide range of loading conditions with limited number of labeled training image data.