Keywords

1 Introduction

In the textile industry, reports on fabric defects are used as a measure of quality of a fabric roll. Such reports also serve the purpose of eliminating cause of defects, hence improve production quality. The conventional approach is human inspection to determine the location, type and size of the defects. However, this approach is prone to human errors, optical illusions and has the risk of missing small defects. It also suffers from low accuracy rate, poor consistency among inspectors, and poor efficiency. Therefore, computer enabled automated defect detection based on image processing emerged as a promising approach to improve efficiency and accuracy, and has been an active research area over the last decade. Camera systems mounted on textile production machines together with advanced data processing and machine learning technology can potentially improve accuracy of fabric defect detection and classification. However, deployment of such systems requires significant work on development and performance improvement as compared to the current practice.

A flaw on the manufactured fabric is known as a fabric defect. Defects may be caused by textile machine/process malfunction or oils from the machinery or faulty yarns or improper mixing of dye stuff in the dye solution. Fabric defects can be classified as minor, major and critical defects. Major defects may affect the sale of the product, and critical defects may result in the entire roll to be rated as second or worse in the quality check process. There are more than 60 defect are given in the document on Standard Test Methods for Visually Inspecting and Grading Fabrics [1]. The fabric defect can be classified into seven main classes, named as yarn, weaving, isolated, pattern, wet processing, raising and milling defects. Some of these are due to surface colour change, and others are due to local texture irregularity.

It is very important to detect these fabric defects because of the financial loss concerns that may arise from faulty production [2]. Considering the fact that 85% of the faults in the clothing industry are caused by fabric faults; if the companies producing woven fabrics use an effective quality control system to produce the desired quality fabric and sell it in this way, they will have adopted the most accurate production system [3]. Traditionally, fabric defect detection is carried out by a skilled worker who tries to detect defects in 2–2.5 m length fabric rolls that are passed at a speed of 8–10 m per minute through an illuminated inspection platform. The employee who detects the error stops the machine and marks the area containing the defect and records it. At the end of this process, acceptance/rejection decision is performed according to the total defect rate in the roll. Defect detection is a long and tiring process. Defect inspectors who constantly look at the same point can audit for a maximum of 1 h due to eyestrain. In traditional systems, the error rate is still around 60%. In addition, the risk of Alzheimer's disease has increased in workers who do this type of work. For these reasons, automated/computerized fabric defect detection systems have been researched.

Automated real-time inspection system is usually composed of camera, light, im-age capturing and processing unit. Defect detection techniques vary from conventional image processing techniques to machine learning techniques. In [4], algorithms developed for detecting fabric defects were divided into three groups, namely statistical [5,6,7], spectral [8,9,10] and model-based approaches [11,12,13].

Recently, machine learning approaches have attracted a lot of attention and been applied in detection of fabric defects, as well as in many different fields. For example, neural network and deep learning network approaches were used in [14, 15], while the convolutional deep learning approach was used in [16,17,18]. In traditional convolutional deep learning models, a large dataset is necessary for successful performance. However, for some classification problems, the amount of data that can be collected is limited in practice, leading to a small dataset. It has been shown that, with small datasets, U-Net gives better results than classical models in pixel-based image segmentation problems.

In this paper, we are proposing a hybrid classification model which employs U-Net to determine whether image contains any defects. If a defect is detected, then defect classification is carried out by using a hybrid model made up of VGG16 and Random Forest. We used the AITEX dataset [19] for training and validation purposed. The results show that the proposed model resulted in 99.8% accuracy for detecting defects which is better than those presented in the literature.

2 Dataset and Experimental Work

We used the AITEX fabric defects dataset [19] for training and testing our model. The original AITEX dataset consists of 245 images of 4096 × 256 pixels captured from seven different fabric structures. The fabrics in the dataset are mainly plain. There are 140 defect-free images, i.e. 20 images for each of fabric structure type. The remaining 105 images contain 12 different types of fabric defects which commonly appear in the textile industry. After data enhancement and decomposition of the original dataset into smaller 256 × 256 pixels images, we obtained a data set of 165 defected samples of 256 × 256 pixels. Then we carried out augmentation on the 165 samples (90° clockwise rotation, 180° clockwise rotation and 270° clockwise rotation), we obtained a set of 660 defected samples. Table 1 shows the numbers of occurrences of the defect types before and after the augmentation. Table 2 shows samples of images for different defect types and corresponding numbers of occurrences after augmentation. We used 592 images corresponding to the seven most common defect types as our data set. This dataset was randomly divided into training set (85%) and validation set (15%).

Table 1. Defective sample counts before and after data augmentation.
Full size table
Table 2. Enhanced AITEX fabric defects dataset.
Full size table

Our tests were conducted on a computer with a processor of ‘Intel Core i7-11800H’, a graphics card of ‘RTX 3070 8 GB’ and operating system of Windows10. Our source code was developed in Python programming environment.

3 Defect Detection and Classification

A two-stage classification approach is used in the proposed model. Images of fabrics are first applied to a customized U-Net to determine whether the fabric is defected or not. Images of defected fabrics are then put through a classification process where VGG16 is used for extracting features and a random forest is used to determine defect type. Following subsections describe U-Net, VGG16 and random forest methods used.

3.1 Defect Detection Using a Customized U-Net

U-Net, a type of Convolutional Neural Networks (CNN) approach, was first developed in 2015 by Olaf Ronneberger, Phillip Fischer, and Thomas Brox to perform better segmentation on biomedical images [20]. Different from traditional convolutional deep learning models which require a large dataset for successful performance, U-Net performs well with small datasets. Since data sets available for fabric defects are small, U-Net was used for defect detection in this work.

The architectural structure of the proposed U-Net is shown in Fig. 1. The number of filters in the original U-Net has been reduced from 64,128, 256, 512, 1024 to 16, 32, 64, 128, 256, respectively. The reduction in the number of filters speeds up the training process and reduces the computational load. As seen in Fig. 1, in the encoder side or in the feature extraction side of the U-Net, different numbers and sizes of features are extracted along the convolutional neural network model, and the pooling process of their height and width dimensions is carried out using the pooling layer.

Fig. 1.
figure 1

Architectural structure of the proposed U-Net.

Full size image

The second half of the model serves to increase the size, hence the resolution of the image at the output of the model in accordance with the input image. In order to reduce complexity and computation time, each convolution output in the Encoder part is combined with the corresponding upsampling layers.

Loss Function.

The number of fabric defects is usually small and the total area affected by the defects is quite small compared to the defect-free part of the fabric. Using an ordinary loss function such as standard cross entropy may cause the model to fall into its local optimum points. To overcome this imbalance problem, cross entropy and Dice binary loss function are used together. Dice function is a frequently used performance criterion to evaluate success in biomedical images, and will be used in this work. The Cross-entropy (C) and Dice (D) loss functions are expressed as follows;

$$C(\mathrm{p},\mathrm{ q})=-\sum\nolimits_{x\in X}p(x)\mathrm{ logq}(\mathrm{x})$$
(1)
$$D(\mathrm{p},\mathrm{ q})=\frac{2 {\sum }_{i}^{N}{p}_{i} {q}_{i}}{{\sum }_{i}^{N}{p}_{i}^{2}+ {\sum }_{i}^{N}{q}_{i}^{2}}$$
(2)

where p(x) and q(x) in (1) represents distribution functions of predicted and actual sets, the values pi and qi values in (2) are the pixel values of prediction and ground truth, respectively.

Training Evaluation Metrics.

In traditional binary segmentation problems, accuracy metric is typically used. The Accuracy metric is defined as follows

$$Accuracy=\frac{TP+TN}{TP+TN+FP+FN}$$
(3)

where (TP) represents true positive, while (TN), (FP) and (FN) stands for true negative, false positive and false negative respectively.

Since defects affect only a small area of the fabric, the number of pixels in defective regions is very small compared to that in the defect-free region. Therefore, Recall (also known as sensitivity), precision and F1 Score are taken into account as well as the accuracy in the evaluation of the model. These evaluation criteria are expressed as,

$$Recall=\frac{TP}{TP+FN}$$
(4)
$$Precision=\frac{TP}{TP+FP}$$
(5)
$$F1 score=2* \left(\frac{Precision * Recall }{Precision + Recall}\right)$$
(6)

Training.

We used the Adam optimizer with a mini-batch size of 16. The training settings of the model were chosen as follows: initial learning rate was set to 0.001, and exponential decay rates for the estimates of mean and variance were set to β1 = 0.9 and β2 = 0.999, respectively. We trained the model for 100 epochs. Table 3 lists the results from our model and compares them with other models. Accuracy for our model is 99.2% and recall is 94.3%, both of which are superior as compared to other models.

Table 3. Performance comparison of the proposed model
Full size table

Then, we trained the model for 500 epochs. Figures 2, 3, 4 and 5 present the results of accuracy, recall, F1-Score and precision for training and testing phases. As can be seen, the proposed model showed superior performance after being trained for 500 rounds. The accuracy was 99.8% for the training phase and 99.3% for the validation phase. Corresponding values were 98.5% and 90.2% for recall, 96.2% and 91.3% for F1, and 96.5% and 92.2% for precision.

Fig. 2.
figure 2

Accuracy performance of the proposed model.

Full size image
Fig. 3.
figure 3

Recall performance of the proposed model.

Full size image
Fig. 4.
figure 4

Precision performance of the proposed model.

Full size image
Fig. 5.
figure 5

F1-score performance of the proposed model.

Full size image

The loss function performance figures of the proposed model for the training and validation phases are shown in Fig. 6. As can be seen in the figure, values of the loss functions were reduced to very small values, i.e. being 0.0095 for Training-Loss and 0.0529 for Validation-Loss.

Fig. 6.
figure 6

Loss performance of the proposed model.

Full size image

We tested the trained model on numerous images from the validation data set. For this we randomly selected an image from the validation data set and applied it to the input of the model, and obtained the model output. We have seen that our model successfully detected the defects in all the images. Figure 7 shows two sample images of defective fabrics on the left column, and model output on the right, and shows that the defects were detected successfully by the model.

Fig. 7.
figure 7

Samples of defect detection test results of the proposed model.

Full size image

3.2 Hybrid Model for Defect Classification

We used a hybrid model for fabric defect classification in this work. Images of fabrics are first applied to the customized U-Net to determine whether the fabric is defected. Images of defected fabrics are then put through a classification process where VGG16 is used for extracting features and a random forest to determine defect type.

VGG16.

VGG16 is a convolutional neural network model proposed in an article by K. Simonyan and A. Zisserman from Oxford University [22]. In VGG architecture, the image is passed through a series of convolutional layers which followed by three Fully-Connected (FC) layers. Filters with a very small receptive field (3 × 3) are used in the convolutional layers. The convolution stride is fixed to 1 pixel; the spatial padding of convolutional layer input is used to preserve spatial resolution after convolution, i.e. the padding is 1-pixel for 3 × 3 convolutional layers. Max-pooling follows every stack of convolutional layers, and performs spatial pooling over a 2 × 2 pixel window, with stride 2. Three Fully-Connected (FC) layers follow a stack of convolutional layers with 4096, 4096 and 1000 channels. Each fully connected layer is followed by a nonlinear activation function, such as ReLU, The final layer is the soft-max layer which normalizes output real values from the last fully connected layer to target class probabilities. In our work, we used the VGG16 only for feature extraction, therefore the fully connected layer which used in original model for classification is removed (see Fig. 8), and the feature set of 8 × 8 × 512 which obtained from the last convolution layer of VGG16 were fed to random forest for fabric defect classification.

Fig. 8.
figure 8

Architectural structure of VGG16 used.

Full size image

Random Forest.

Random Forest is a supervised learning algorithm. As the name suggests, it creates a forest in a random manner. The “forest” is a collection of decision trees that are mostly trained by the “bagging” method. With this method, hundreds of decision trees are created and each decision tree makes an individual prediction. The structure of a simple random forest consists of multiple decision trees as shown in Fig. 9 and combines them to get a more accurate and stable prediction. For example, in regression problems, average of the predictions of the decision trees is used. In classification problems, most voted one is chosen among the predictions. The biggest advantage of the random forest is that it does not cause an over-learning/overfitting problem that is often experienced in traditional machine learning methods.

Fig. 9.
figure 9

Simple random forest.

Full size image

3.3 Testing the Model

We tested our model by using images of fabrics with seven most common defect types (namely broken pick, fuzzy ball, nep, sloughed fill, cut selvage, broken end and broken yarn). We had 592 samples images for the seven most common defect types. The proposed model performed well on the limited data set of fabric defects; U-Net detected fabric defects with high accuracy (99.3%). The confusion matrix for classification performance of our model is given in Fig. 10. It can be seen that the proposed hybrid model classifies sloughed filling type with 100% accuracy, broken pick type with 97% accuracy, broken yarn type with 80% accuracy and fuzzy ball type with 74% accuracy. However, the model could classify nep type only with 12.5% accuracy, and confused this type with fuzzy ball and broken end types. The model failed to classify cut selvage type, and confused it with broken pick. High accuracy for the broken pick defect is attributed to the fact that this defect type is the most occurring type in the data set (220 out of 592 samples) and the area affected by defect is not small. The sloughed fill, although not so common (56 out of 592 samples), due to the large areas affected by this type resulted in a very high classification accuracy. Performance was not good for the nep and cut selvage. The reason for this is thought to be that only a small area of the fabric is affected by these two types.

Fig. 10.
figure 10

Confusion matrix for classification results.

Full size image

4 Conclusions

We proposed a hybrid detection and classification model for fabric defects. The model first detects defects by using a customized U-Net, and then carries out classification on defected fabrics by using VGG16 for feature extraction and random forest for classification. AITEX data set was used for training and testing purposes. We applied data enhancement, decomposition, augmentation on the data set, and obtained 660 images of 256 × 256 pixels for defected fabrics and used 592 of those corresponding to the seven most common defect types. The customised U-Net performed well on binary classification of defected and defect-free images, giving accuracy level of 99.3%. On the defect classification side of our model, we achieved high accuracy for defect types of broken pick, sloughed fill, broken yarn and fuzzy ball. On the other hand, exhibited poor classification performance for types of cut selvage and nep. Our results show that defects that affect large area on the fabric surface can be detected with high accuracy. Future research efforts can be directed towards improving the classification performance for defects with small size. The developed model will be evaluated by collecting data in real field trials at a company producing textile machinery in Turkey.