Detection and classification of surface defects of the rolled metal are one of the main tasks for correctly assessing product quality. Historically, these tasks were performed by a human. However, due to a multitude of production factors, such as high rolling rate and temperature of the metal, the results of such human work are rather low. Replacing a human controller with an artificial intelligence system has been relevant for a long time; it is merely necessary within the concept of Industry 4.0. This paper is devoted to the development of the prototype system automatic detection and classification of defects for one of the Iron-and-Steel Works of the Chelyabinsk region in the Russian Federation. The prototype consists of the Preprocessor, Classifier, Server, Database, and User interface. The main focus is on achieving high classification accuracy, which is planned to be obtained through the use of convolutional neural networks.