Keywords

1 Introduction

With the development of industry 4.0, a growing number of steel products have been manufactured to satisfy the demand for various areas, particularly in civil engineering such as building or infrastructure. The manufacturing process may cause some problems or faults in the steel surface [2, 8]. The low quality of steel products results in their inefficient capacity for use. Therefore, inspecting surface defects plays a critical role in steel manufacturing. The conventional techniques show adequate accuracy for recognizing surface errors. However, these methods do not need the increasing requirement of manufacturing standards [10]. With the aid of computer vision, an automatic approach can be used to detect steel sheet defects in manufacturing processes. Currently, deep learning technique has been widely utilized in different fields for defect recognition by using images [4, 9]. From previous studies, some deep learning models such as GoogLeNet or region-based convolutional neural network (R-CNN) have been developed successfully in terms of detecting or classifying defects of steel sheets [2, 10]. Among these networks, a proposed DenseNet121 architecture with fine-tuning transfer learning was developed in this study.

2 Methodology

2.1 DenseNet121 Architecture

A pre-trained deep learning model, DenseNet121 was applied in this study. The detailed architecture can be found in [5]. Briefly, the model was enhanced by DenseNet architecture with a shorter network between layers. The proposed model was trained, evaluated, and tested by using transfer learning (TL) and k-fold due to various advantages [1, 3]. TL has been widely applied to deep learning algorithms due to various advantages [3]. The diagrams of feature extraction and fine-tuning in TL were as shown in Fig. 1. k equals 5 folds used in this paper to estimate the predicted accuracy of the model as presented in Fig. 2.

Fig. 1.
figure 1

Basic flow of transfer learning

Full size image
Fig. 2.
figure 2

5-fold cross-validation

Full size image

2.2 Evaluation Metric

The training and testing procedures of the proposed Densnet121 model were evaluated by the accuracy metric. This evaluation metric has been widely applied in deep learning [3] by using true/false positive (TP/FP) and true/false negative (TN/FN). It can be calculated by values of (TP + TN) divided by the values of (TP + FP + TN + FN). The testing set was then evaluated by using confusion matrices to compare the actual and predicted defects in the suggested model. Furthermore, this paper applied Grad-CAM to locate defects in color.

3 Results and Discussion

3.1 Dataset

This study used 1800 images of steel surface defects that were obtained from [7]. For the purpose of training, all images were converted to 224 × 224 pixels and divided into training and testing sets as given in Table 1. It should be noted that the training set was separated into the 80% training and 20% validation subsets. The defects were classified into six groups including rolled-in scale, patches, crazing, pitted surface, inclusion, and scratches. Typical images for each defect were shown in Fig. 3.

Table 1. Categorized summary of images in the dataset
Full size table
Fig. 3.
figure 3

Samples of six kinds of typical surface defects on the NEU surface defect database [7]

Full size image

3.2 Performance of DenseNet121

The network was trained with 20 epochs using stochastic gradient descent (SGD) and adaptive moment estimation (Adam) [6, 8]. Both optimizers were applied with an initial learning rate of 0.0001. Moreover, data augmentation such as shearing, zooming, or flipping images was employed to increase the generality of the database and decline the influence of overfitting during the training process. The changes in cross-entropy loss function and accuracy values of training and validation subsets were depicted in Fig. 4. It is clear to see that the proposed model rapidly converged around the fifth epoch and the third epoch for SGD and Adam, respectively.

Fig. 4.
figure 4

Loss and accuracy histories: a SGD, b Adam

Full size image

Figure 5 depicted the confusion matrices with and without normalization on the testing set. It can be observed that while three true samples in the categories of inclusion, pitted surface, and scratches each were incorrectly predicted in SGD, Adam exhibited better performance with the prediction of two incorrect samples of inclusion and pitted surface each.

Fig. 5.
figure 5

Confusion matrices on the testing set: a SGD, b Adam

Full size image

The quantitative analysis of the accuracy of 5 folds on the testing set was revealed in Table 2. While the testing results revealed a high performance of over 98% for all folds, Adam showed better accuracy than SGD.

Table 2. Accuracy results of 5 folds on the testing set, by percentage
Full size table

3.3 Defect Visualization Using Grad-CAM

To investigate the feasibility of Grad-CAM visualization, this study evaluated whether Grad-CAM can be used for locating steel defects. Figure 6 depicts the correct location of failures for each type of defect with bright colors using Grad-CAM, which corresponds to the actual images shown in Fig. 3. It is worth noting that the severe failures indicated the brighter colors.

Fig. 6.
figure 6

Grad-CAM localization of defects

Full size image

4 Conclusion

This study evaluated the performance of the proposed DenseNet121 model for predicting surface defects in steel manufacturing industries. SGD and Adam optimization algorithms with fine-tuning transfer learning and k-fold techniques were conducted on the improvement and estimation of the trained network. The different validation metrics such as accuracy, confusion matrix, or Grad-CAM were implemented. The results present that the high performance of over 98% accuracy was obtained from both optimizers in all 5 folds. Moreover, the outperformance of Adam in comparison with SGD was gained by using confusion matrices. The visualization of Grad-CAM can be found as an efficient tool for locating steel surface defects. Last but not least, the proposed model was sufficient for the prediction of defects in steel industry production.