In the present study, ultrasonic guided wave testing (UGWT) is used as a nondestructive inspection technique to identify flaws in carbon steel pipe. The selection of a wave mode in UGWT is critical. Therefore, the flexural wave mode F (1, 1) has been found to be a suitable choice. Not only that, but the pipe’s length is also subject to variation, and the defect may manifest at disparate locations along its length. To address this, several studies have been explored such as convolutional neural networks and recurrent neural networks. Nonetheless, a novel technique is proposed in this research, the transformer neural network architecture due to its capability to capture long-range dependencies and temporal relationships. The accuracy of the chose deep learning architecture was evaluated, and suitable hyperparameters were selected to optimize the model. As a result, the transformer architecture accurately predicts the flaw pattern for the vast majority of the test data, attaining an accuracy and F1-score of 99 %.