A neighbourhood feature-based local binary pattern for texture classification

Abstract

The CNN framework has gained widespread attention in texture feature analysis; however, handcrafted features still remain advantageous if computational cost needs to take precedence and in cases where textures are easily extracted with few intra-class variation. Among the handcrafted features, the local binary pattern (LBP) is extensively applied for analysing texture due to its robustness and low computational complexity. However, in local difference vector, it only utilizes the sign component, resulting in unsatisfactory classification capability. To improve classification performance, most LBP variants employ multi-feature fusion. Nevertheless, this can lead to redundant and low-discriminative sub-features and high computational complexity. To address these issues, we propose the neighbourhood feature-based local binary pattern (NF-LBP). Inspired by gradient’s definition, we extract the neighbourhood feature in a local region by simply using the first-order difference and 2-norm. Next, we introduce the neighbourhood feature (NF) pattern to describe intensity changes in the neighbourhood. Finally, we combine the NF pattern with the local sign component and the centre pixel component to create the NF-LBP descriptor. This approach provides better complementary texture information to traditional local sign pattern and is less sensitive to noise. Additionally, we use an adaptive local threshold in the encoding scheme. Our experimental results of classification accuracy and F1 score on five texture databases demonstrate that our proposed NF-LBP method attains outstanding texture classification performance, outperforming existing state-of-the-art approaches. Furthermore, extensive experimental results reveal that NF-LBP is strongly robust to Gaussian noise and salt-and-pepper noise.

1 Introduction

Texture classification has become a major research topic in various fields, including biomedical image analysis [1], remote sensing [2], image retrieval [3], object recognition [4], and face recognition [5], among others. Texture classification poses a primary challenge in dealing with variations that occur within a given class, which are usually caused by rotation, illumination, viewpoint, and scale. Feature extraction methods based on local descriptors garnered considerable interests owing to the stronger robustness to noise compared to global descriptors, which are easily affected by external conditions. However, most local descriptors face challenges when dealing with high-dimensional and low-discriminatory sub-features, which lead to redundant and inefficient feature information. To address these challenges, some local descriptors like scale-invariant feature transform (SIFT) [6] and speeded-up robust features (SURF) [7] are presented for describing patches surrounding carefully selected keypoints within a texture image. Other descriptors like Histogram of Oriented Gradients (HOG) [8] are utilized to extract global image features. Among all local descriptors, local binary pattern (LBP) [9] has become one of the primary methods for texture feature extraction, owing to its theoretical simplicity and computational efficiency. LBP is a nonparametric texture feature descriptor that extracts features from different texture regions and combines them together as an overall texture feature. In the LBP method, the centre pixel $g_{c}$ is compared with its neighbourhood pixels in the sampling template to obtain a binary code (0 or 1), which is then converted into the corresponding decimal number, which represents the local feature. The conventional LBP method has a limitation in that it only takes into account the sign component of the local difference vector and fails to incorporate other important local structural texture information, leading to incomplete representation of the texture features. As a result, its practical applications are significantly hindered.

Since Ojala’s original work [9], many LBP variants have been proposed to overcome the limitations of the original LBP. The local ternary pattern (LTP) [10] applies three quantization intervals to threshold the centre pixel, making it effective in dealing with minor noise. The ResExLBP [11] utilizes image resizing and greyscale transformation to improve its classification accuracy in detecting COVID-19. In recent years, it has been suggested that capturing and combining more discriminative and complementary local features is an effective approach to achieve stronger robustness and attain a comprehensive texture representation. [12] proposed a completed local binary pattern (CLBP), which achieves a comprehensive representation of local features by combining the three highly complementary local features: sign, magnitude, and centre pixel. Inspired by the framework of CLBP, lots of LBP variants have been proposed, including CLBC [13] and multi-scale CLBP [14]. To enhance the noise robustness, several approaches have been proposed. One such approach is the SFB-OCPS presented by [15]. This technique replaces centre pixels on edges with an optimal pixel based on statistical features, which effectively improves the classification performance. Another approach is the ACPS strategy [16] that chooses an adaptable centre pixel to transform non-uniform patterns into uniform patterns while preserving crucial microstructural texture features. Additionally, the BRINT descriptor presented by [17] replaces the sampled neighbourhood pixels with the average greyscales of the neighbourhood pixels, while the RCLBP framework [18] enhances the robustness of feature extraction in the presence of noise by integrating the non-local means filter, wavelet thresholding, and completed local binary pattern framework. SALBP [19] selects an adaptive sampling scale for every neighbourhood pixel. In order to enhance the robustness of the centre pixel, a novel approach called image segmentation-based central multi-scale local binary pattern (ISCM-LBP) was introduced [20], which connects the centre pixel with its surrounding neighbourhood pixels.

To fully exploit texture features and enhance the classification capabilities of the LBP algorithm, several LBP variants have been presented. LTrPs [21] extract the relationship between the centre pixel and its neighbourhood pixels in the vertical and horizontal directions. LETRIST [22] constructs features from extremum responses of Gaussian derivative filters and quantizes them into texture codes. The Single Direction Gradient (LBP-SDG) [23] extracts discriminative movement features. In order to depict local feature information, FbLBP algorithm [24] leverages both the sign information and the mean and variance of magnitude of the difference vector $d_{P}$. Local Binary Circumferential and Radial Derivative Pattern (CRDP) [25] fuses circumferential and radial derivative features based on different orders. By leveraging a pixel to patch-based sampling structure to emulate the sampling pattern, the innovative Local Neighbouring Intensity Relationship Pattern (LNIRP) [26] captures local features by investigating neighbourhood greyscale properties. Sorted Local Gradient Pattern [27] utilizes two local gradient patterns to encode gradient information in the local neighbourhood and extracts local features by classifying pixels into two classes based on their intensities.. The Local Neighbourhood Difference Pattern (LNDP) [28] captures the interdependencies among neighbourhood pixels $g_{p}$ at a particular scale R and demonstrates exceptional classification performance on natural scenes. The CMPE [29] combines the maxima and minima of the neighbourhood pixels with the conventional sign information, improving the classification capability. The CJLBP [30] joins neighbourhoods across different scales to acquire large-scale texture features. The OD-LBP [31] suggests the utilization of orthogonal values from the surrounding texture region to portray local texture characteristics. A neighbourhood and centre difference-based-LBP (NCDB-LBP) [32] considers the differences between different neighbourhood as the local features. [33] proposes the LBP-RGB feature to specially extract image features in RGB mode. To enhance the quality of extracted features, a novel approach based on genetic programming (GP) [34] was introduced, which utilizes a three-layer tree-based binary program, learned through the GP optimization process, to integrate patch detection, feature fusion, and classification. In addition, local grouped order pattern and non-local binary pattern (LGONBP) [35] were proposed to combine two innovative texture descriptors, including LGOP that groups nearby points based on a dominant direction, encoding their intra-group intensity order, and NLBP that computes anchors using global image statistics and progressively encodes non-local intensity differences between neighbouring points and anchors. A descriptor CLGC [36] was proposed to combine local and global information to extract colour-texture features, incorporating wavelet transform and a modified version of local ternary pattern for global feature fusion, and utilizing speeded-up robust feature descriptor and bag of words model for local features. A hybrid texture feature extraction method called Hess-ACS-LBP [37] was proposed, which combines the Hessian matrix and ACS-LBP to effectively reveal the macro- and microstructural changes in textures.

Besides the handcrafted LBP and its variants, the CNN architecture [38] showcased exceptional classification capabilities in 2012, and it has gained considerable interest ever since. It has found numerous applications in texture analysis, including medical image analysis [39], remote sensing [40], and face recognition [41]. In recent years, many algorithms for texture feature description and extraction have made significant progress, such as the $\mathrm LPMF^{2}Net$ model [42] that explores complementary features at diverse hierarchical levels. Additionally, the dynamic reparameterization network (DRPN) [43] is effective in dealing with scale variations in the classification of infrared images. Researchers have proposed other CNN-based models, such as the attention mechanism-based CNN proposed by [44], which combines LBP features and attention mechanisms to improve classification results, and the local LBP feature and deep feature blending approach proposed by [45] to enhance the recognition capability of handwritten digits. Despite some inherent limitations such as high computational complexity and challenging parameter tuning, the use of deep features for texture representation remains a topic of debate. In [46], the authors conducted a comparative analysis of handcrafted features versus CNN-based features. Notably, the study revealed that handcrafted features outperformed CNN-based features in cases where textures were easily extracted with few intra-class variation. As a result, this paper focuses on exploring the improvement of the traditional handcrafted descriptors.

For instance, most currently existing LBP variants completely discard the local features in the neighbourhood, resulting in misclassification of different patterns. Furthermore, LBP-based algorithms extract different sub-features to represent local texture features. The sub-features employed in these LBP variants often lack rotational invariance and tend to possess low discriminative capability while being high-dimensional, thereby contributing to complex and redundant classification process. Additionally, there is still untapped potential in leveraging the complementarity between the extracted features of LBP variants. The extraction of local features requires a more comprehensive combination of sub-features to adequately describe the local region, as low complementarity leads to suboptimal classification capabilities.

To tackle these challenges, we introduce a neighbourhood feature-based local binary pattern (NF-LBP) descriptor for texture classification, which highlights the local neighbourhood features by using the proposed neighbourhood feature (NF) pattern and combines it with local sign component and centre pixel component. The main contributions of this paper are as follows:

• 1. We propose a novel method to represent the neighbourhood features by simply using the first-order difference and 2-norm.

• 2. We apply a new and effective coding method to encode the neighbourhood feature into a local neighbourhood feature pattern with a local adaptive thresholding quantization method. The resulting local neighbourhood feature pattern is denoted as NF pattern. Our experiments have proved that it has a strong feature classification capability and is strongly complementary to sign information.

• 3. We propose a texture descriptor called neighbourhood feature-based local binary pattern (NF-LBP), which integrates the local sign component and the centre pixel component with neighbourhood feature pattern. This integration results in a highly comprehensive local texture representation.

• 4. We perform comprehensive experiments on 5 different texture databases, which includes texture databases with complex conditions (Outex, CUReT, XU$_{-}$HR, and ALOT) and real-world database (UIUC). Our comparison demonstrates that NF-LBP achieves the outstanding classification performance. Furthermore, extensive experiments with noise demonstrate the excellent noise robustness of the proposed NF-LBP.

The remainder of this paper is as follows: Section 2 provides a brief review of LBP and its related works. Section 3 presents the detailed description of the proposed NF-LBP. Section 4 presents experimental results and analyses. Section 5 concludes the paper.

2 A concise overview of relevant works

Ojala et al. [47] proposed local binary pattern (LBP) and improved it in the consequent works [9]. The conventional LBP focuses on extracting relations of centre pixel $g_{c}$ and the corresponding neighbourhood pixels in a circular local neighbourhood template, which is defined by the chosen radius R and P neighbourhood pixels $g_{p}$ $(p = 0, 1,\ldots , P-1)$, as shown in Fig. 1. The original coding method of LBP is described as follows:

$$\begin{aligned} \text {LBP}_{P,\ R}(g_{c})= & {} \sum _{p=0}^{P-1}2^{p} \times s(g_{p}-g_{c}),\quad \nonumber \\ s(x)= & {} \left\{ \begin{matrix} 1,&{}x\ge 0\\ 0,&{}otherwise \end{matrix} \right. \end{aligned}$$

(1)

Bilinear interpolation is utilized to estimate neighbourhood pixels $g_{p}$ located at non-integer positions within texture image. The LBP method is utilized to every centre pixel $g_{c}$ and thus extracts texture features of a whole image.

Fig. 1

Different circular neighbourhood templates of LBP

Additionally, [48] discovered that the frequency of occurrence of some LBP patterns is obviously higher than other LBP patterns. To classify LBP patterns, they proposed the uniform measure U to represent the spatial transition, which counts the number of bitwise “0/1” and “1/0” in a LBP pattern. The uniform measure U of a LBP pattern $\text {LBP}_{P,\ R}(g_{c})$ is defined as follows:

$$\begin{aligned} U(\textrm{LBP}_{P,\ R}(g_{c}))= & {} \sum _{p=1}^{P-1}|s(g_{p}-g_{c})-s(g_{p-1}-g_{c})|\nonumber \\{} & {} +|s(g_{P-1}-g_{c})-s(g_{0}-g_{c})| \end{aligned}$$

(2)

The value U quantifies the frequency of local pattern changes, where lower values represent low-frequency image signal and higher values indicate high-frequency image signal. Given that natural images are predominantly composed of low-frequency signal, LBP patterns which satisfy $U\le 2$ are considered uniform patterns. Meanwhile, non-uniform patterns refer to the remaining LBP patterns.

To improve classification performance and achieve rotation invariance, based on the definition of the uniform measure U, rotation-invariant uniform LBP operator, which is usually abbreviated as $\text {LBP}_{P,R}^{\text {riu}2}(g_{c})$, is defined as follows:

$$\begin{aligned} \textrm{LBP}_{P,R}^{\text {riu}2}(g_{c})=\left\{ \begin{array}{ll} \sum _{p=0}^{P-1}s(g_{p}-g_{c}),&{}\quad U(\textrm{LBP}_{P, R}(g_{c}))\le 2\\ P+1,&{}\quad \textrm{otherwise} \end{array} \right. \end{aligned}$$

(3)

The LBP referred to in the following context is the $\text {LBP}_{P,R}^{\text {riu}2}(g_{c})$, which is particularly useful for rotation-invariant texture classification.

Figure 2 shows an example of $\text {LBP}_{P,R}^{\text {riu}2}(g_{c})$.

Fig. 2

An example of $\textrm{LBP}_{P,R}^{\text {riu}2}(g_{c})$ descriptor

3 The proposed method

3.1 Neighbourhood feature in local region

The conventional LBP method describes the texture feature of a local region by extracting the difference vector $d_{P}=\left[ g_{0}-g_{c}, g_{1}-g_{c}, \ldots , g_{P-1}-g_{c}\right]$ $(p=0, 1,\ldots , P-1)$. In order to depict the local features, the corresponding LBP pattern in the sampled image region uses the sign component $s_{P}$ of difference vector $d_{P}$. However, the texture feature may not be comprehensively represented by only the sign component $s_{P}$. To complement the sign component $s_{P}$ and enhance the classification capability, CLBP [12] introduced magnitude component CLBP$_{-}$M and centre pixel component CLBP$_{-}$C. Magnitude component is generally denoted as $m_{P}$. As shown in Fig. 3, $s_{P}$ and $m_{P}$ can be directly obtained from $d_{P}$. Based on the framework of CLBP, BRINT [17] achieved excellent classification capability by using the arc-based averaging method. The BRINT descriptor can be decomposed into three components: the sign BRINT$_{-}$S, the magnitude BRINT$_{-}$M, and the centre pixel BRINT$_{-}$C. BRINT uses the magnitude information of $d_{P}$ in the same way as CLBP. Although $m_{P}$ can supply complementary texture feature information, there are certain problems with its use. One of these is that in certain cases, if $m_{P}$ cannot be used properly, it is low-informative and low-discriminative, thus leading to inefficient classification performance. The other problem is that the $m_{P}$ is high-dimensional, but it may provide redundant texture feature information. Therefore, how to enhance the classification performance of the LBP algorithms and how to sufficiently extract distinctive texture features in the local region have become the main research direction in the field of texture classification.

Fig. 3

An example to show the decomposition of the difference vector $d_{P}$

Most current existing LBP variants are dedicated to extracting the relations of neighbourhood pixels $g_{p}$ and the corresponding centre pixel $g_{c}$. Under these circumstances, the texture feature information of the neighbourhood is completely discarded, and this can lead to inefficient classification. As shown in Fig. 4, in conventional LBP, the same LBP code can be obtained even if the centre pixels $g_{c}$ are the same, and the sampled neighbourhood pixels $g_{p}$ are different in the local texture structure.

Fig. 4

An example to represent that different local structures with the same $g_{c}$ may be encoded into the same value

Actually, the texture feature information of the neighbourhood has strong texture classification capability. To avail the local features, texture features of sampled neighbourhood pixels can be exploited to contribute auxiliary and useful neighbourhood feature information. Based on this consideration, neighbourhood feature-based local binary pattern (NF-LBP) is proposed. The important feature contained in sampled neighbourhood pixels can reflect the local intensity and provide crucial complementary texture feature, which is informative and discriminative.

Gradient information has been considered as a significant texture feature of an image since it can indicate the contrast of the local image texture. For the high-contrast areas or edges of an image, the gradient is large; conversely, the gradient is small in the smooth texture regions of an image. For the purpose of discriminative and additional feature extraction, unlike most variants of LBP, we forgo the use of the magnitude part $m_{P}$ extracted from $d_{P}$. We propose to extract the texture feature contained in sampled neighbourhood pixels in the local image region. Gradient is selected as the feature of the sampled neighbourhood in the local pattern.

Strictly speaking, gradient calculation requires a derivative, but to simplify the calculation process and reduce computational complexity, we obtain an approximate derivative of the gradient by directly calculating the first-order difference of the neighbourhood pixels along the arc of the circular neighbourhood. As shown in Fig. 5, we define the two gradients of the neighbourhood pixel in terms of the first-order difference in clockwise and counterclockwise directions, denoted, respectively, as $G_{+,\ g_{p}}$ and $G_{-,\ g_{p}}$ $(p=0, 1,\ldots , P-1)$. The formulas are as follows:

$$\begin{aligned} G_{+,\ g_{p}}= & {} \left\{ \begin{matrix} g_{P-1}-g_{0},&{}\quad p = 0\\ g_{p-1}-g_{p},&{}\quad \textrm{otherwise} \end{matrix} \right. \end{aligned}$$

(4)

$$\begin{aligned} G_{-,\ g_{p}}= & {} \left\{ \begin{matrix} g_{0}-g_{P-1},&{}\quad p = P-1\\ g_{p+1}-g_{p},&{}\quad \textrm{otherwise} \end{matrix} \right. \end{aligned}$$

(5)

where P is the number of neighbourhood pixels in the sampling template.

The defined gradient of neighbourhood pixel $G_{g_{p}}$ $(p=0, 1,\ldots , P-1)$ merges its respective gradients $G_{+,\ g_{p}}$ and $G_{-,\ g_{p}}$ in the form of 2-norm, which is defined as follows:

$$\begin{aligned} G_{g_{p}}=\sqrt{G_{+,\ g_{p}}^{2}+G_{-,\ g_{p}}^{2}} \end{aligned}$$

(6)

Fig. 5

An illusion of the two gradients $G_{+,\ g_{p}}$ and $G_{-,\ g_{p}}$ of the neighbourhood pixels in a local region

After the calculations above, we can obtain a new feature vector in the local region of a texture image. Taking Fig. 5 as an example, in the end, we get a feature vector of neighbourhood pixel gradients $G_{g_{p}}$ in the local sampling pattern, abbreviated as $G_{P}$. Our proposed gradient vector $G_{P}$ can succinctly describe neighbourhood features and has a number of outstanding advantages. First, the gradient information can straightforwardly reflect the local contrast of a texture image. Second, the difference vector between neighbourhood pixels remains the same regardless of how the image is rotated. Thus, the gradient vector $G_{P}$ is rotation invariant. Third, the centre pixel $g_{c}$ may be easily polluted by noise. As shown in Fig. 6, in this case, the difference vector $d_{P}=\left[ g_{0}-g_{c}, g_{1}-g_{c}, \ldots , g_{P-1}-g_{c}\right]$ will be completely changed, thus losing its discriminative capability. Compared with the difference vector $d_{P}$, the gradient vector $G_{P}$ will not be altered by the noise and still contain discriminative feature information of the local texture. Hence, the gradient vector $G_{P}$ is more robust to noise than $d_{P}$.

Fig. 6

An example to show that the gradient vector $G_{P}$ will not be changed when the centre pixel $g_{c}$ is polluted by noise

3.2 Neighbourhood feature (NF) pattern

The gradient vector $G_{P}$ consists of $G_{g_{p}}$, which are continuous values. Same as the difference vector $d_{P}$, $G_{P}$ cannot be straightforwardly utilized for texture classification but needs to be converted to binary strings. Therefore, we propose a neighbourhood feature (NF) pattern by applying a novel coding method for the gradient vector $G_{P}$.

Firstly, as shown in Fig. 7, we perform an image segmentation to obtain $N \times N$ discrete sub-images in a whole texture image. Then, local adaptive threshold of each sub-image is used for binary quantization of gradient vector $G_{P}$ because of the strong correlation of the same feature in a local region of a texture image. We define each local adaptive threshold of $G_{P}$ as the mean value of $G_{g_{p}}$ of each sub-image. It is clear that the local adaptive threshold is more reflective of texture changes in local regions than the global threshold used in LBP and most LBP variants. To achieve a trade-off between computational complexity and discrimination capability, we set N = 4 in our experiments, thus acquiring 4 $\times$ 4 = 16 sub-images. The gradient vector $G_{P}$ in the local region can be therefore encoded into a local NF binary pattern to participate in the subsequent texture classification task.

Fig. 7

The generation flow gram of local adaptive threshold

Secondly, as proposed in Ojala’s work [9], there are high-frequency and low-frequency patterns in the local NF pattern, and their contribution to the texture classification is not of equal importance. Accordingly, we need to encode them differently. The uniform metric parameter, denoted as $\tilde{U}$ in the following, which represents the bitwise of “0/1” and “1/0” in NF binary string, is taken as the threshold to distinguish between uniform NF patterns that occur with high-frequency and non-uniform NF patterns that occur with low frequency. When $U(\text {NF}) \le \tilde{U}$, then this NF binary pattern is defined as an NF uniform pattern; conversely, it is a non-uniform NF pattern.

Subsequently, to fix a reasonable threshold $\tilde{U}$, we select different $\tilde{U}$ and conduct a series of experiments on the UIUC texture database, which contains 25 different texture classes, and each class consists of 40 different texture images. The texture image size on the UIUC database is 640 $\times$ 480. Table 1 shows that for different choices of U values, the average percentage of non-uniform NF patterns on the UIUC database. Observations reveal that if $\tilde{U}$ is chosen to be less than or equal to 4, the non-uniform NF patterns occur with higher frequency than uniform NF patterns, which is inconsistent with the previous definition. If $\tilde{U} = 6$, at sampling radius R = 1 or R = 2, the non-uniform NF patterns hardly appear, which is not a reasonable situation. When $\tilde{U}$ is fixed at 4, the occurrence frequency of non-uniform NF patterns is consistent with Ojala’s research. We, therefore, set the $\tilde{U}$ at 4 in the following encoding process of the NF pattern.

Table 1 Average percentage of non-uniform patterns (%) on UIUC database using different $\tilde{U}$

Finally, we propose the definition of neighbourhood feature (NF) pattern. Inspired by the local binary pattern (LBP) method [47], the NF pattern is calculated by comparing a pixel with its corresponding local adaptive threshold in a sub-image. We introduce the coding strategy of locally rotation-invariant pattern $\textrm{LBP}_{P,\ R}^{\text {riu}2}$ [9]. Thus, the definition of the NF pattern is as follows:

$$\begin{aligned} \textrm{NF}_{P,\ R}=\left\{ \begin{array}{ll} \sum _{p=0}^{P-1}s(G_{g_{p}}-t_{G}),&{}U(\textrm{NF})\le 4\\ P+1,&{}\textrm{otherwise} \end{array} \right. \end{aligned}$$

(7)

where the function s(x) is defined in Eq. (1), $G_{g_{p}}$ is the proposed gradient of neighbourhood pixel $g_{p}$, $U(\textrm{NF})$ represents the number of bitwise changes between “0” and “1” in the NF binary pattern, and the local adaptive threshold $t_{G}$ is the average of $G_{g_{p}}$ in corresponding located sub-image.

3.3 Neighbourhood feature-based local binary pattern (NF-LBP)

In the subsection, we introduce a neighbourhood feature-based local binary pattern (NF-LBP), which consists of three parts: the sign part NF-LBP$_{-}$S, the neighbourhood feature part NF-LBP$_{-}$NF, and the centre pixel part NF-LBP$_{-}$C. The sign part NF-LBP$_{-}$S is the same as the locally rotation-invariant pattern $\textrm{LBP}_{P,\ R}^{\mathrm{\text {riu}}2}$. The neighbourhood feature part NF-LBP$_{-}$NF, defined in Eq. (7), reflects the local contrast and therefore preserves the neighbourhood feature of the local texture region. The centre pixel part NF-LBP$_{-}$C expresses important local intensity information. We convert it to binary code by employing the local adaptive threshold as follows:

$$\begin{aligned} {\text {NF}}{\text {-}}{\text {LBP}}_{-}\text {C}=s\left( g_{c}-t_{c}\right) \end{aligned}$$

(8)

where the function s(x) is the sign function defined in Eq. (1), $g_{c}$ is the centre pixel, $t_{c}$ is the mean of $g_{c}$ in corresponding located sub-image.

For clarity, the complete flow diagram of NF-LBP descriptor is shown in Fig. 8.

In the end, we combine NF-LBP$_{-}$S, NF-LBP$_{-}$NF, and NF-LBP$_{-}$C jointly to build a 3D feature histogram of the texture image for texture classification.

3.4 Complementarity to LBP and classification performance comparisons of NF-LBP$_{-}$NF and CLBP$_{-}$M

To verify the complementarity of our proposed NF-LBP$_{-}$NF to LBP and to demonstrate its excellent texture classification capability, in this subsection, we conduct a series of experiments on the UIUC database [49]. The UIUC database consists of 25 different texture image classes, each containing 40 different texture images. In our experiments, k images of every texture class are randomly chosen for training. The rest texture images of each texture class are used for testing. k is usually chosen as 5, 10, 15, or 20 to evaluate the algorithms’ discrimination performance of texture classification.

To illustrate the complementarity of the proposed NF-LBP$_{-}$NF to LBP, we test its texture classification capability on the UIUC database and compare it with LBP and CLBP${-}$M. In this validation experiment, the sampling radius R is chosen as 1, the number of sampled neighbourhood pixels is chosen as 8, and k is set to 10. That is, we randomly select 10 images from each texture class as training images, and the remaining 30 images as test images.

Firstly, as shown in Fig. 9, compared with LBP and CLBP$_{-}$M, NF-LBP$_{-}$NF can correctly classify the largest number of texture images, which implies that the classification capability of NF-LBP$_{-}$NF is better than LBP and CLBP$_{-}$M. Secondly, we assess the complementarity of NF-LBP$_{-}$NF to LBP and make a comparison with CLBP$_{-}$M. How well NF-LBP$_{-}$NF and CLBP$_{-}$M complement LBP is demonstrated by how many texture images they can correctly classify that LBP misclassifies. Of the 374 texture images correctly classified by CLBP$_{-}$M, 227 images can be correctly classified by LBP, i.e. it correctly classifies 147 $=$ 374 − 227 texture images, which are misclassified by LBP. Meanwhile, for NF-LBP$_{-}$NF, of the 465 texture images it correctly classifies, 264 can be correctly classified by LBP, which means that NF-LBP$_{-}$NF can additionally correctly classify 201 $=$ 465 − 264 texture images which are incorrectly classified by LBP. Therefore, NF-LBP$_{-}$NF shows stronger complementarity to LBP than CLBP$_{-}$M, which proves the efficacy of the proposed NF-LBP descriptor.

Fig. 8

The complete flow diagram of NF-LBP descriptor

Fig. 9

The classification results of LBP, CLBP$_{-}$M and NF-LBP$_{-}$NF on UIUC

Fig. 10

The classification accuracy of CLBP$_{-}$M, CLBP$_{-}$S/M, NF-LBP$_{-}$NF, and NF-LBP$_{-}$S/NF on UIUC database

Fig. 11

The classification accuracy of CLBP$_{-}$S/M and NF-LBP$_{-}$S/NF on UIUC database with different sampling parameters

Table 2 Ablation study about texture classification accuracy (%) and histogram dimensionality of NF-LBP on UIUC database

To further illustrate the outstanding classification capability of the neighbourhood feature NF-LBP$_{-}$NF, we construct 2D joint histograms, namely “CLBP$_{-}$S/M” and “NF-LBP$_{-}$S/NF”, and conduct supplementary experiments on the UIUC database. To ensure the reliability and authenticity of our experimental results, we randomly select training images and repeat this process independently 100 times. The deciding result is the mean value of 100 experiments.

Firstly, we fixed R to 1 and P to 8. Then, we choose the number of training images k to be 5, 10, 15, and 20, respectively. As shown in Fig. 10, regardless of the choice of k, NF-LBP$_{-}$NF achieves a higher classification accuracy than CLBP$_{-}$M. The classification accuracy of NF-LBP$_{-}$S/NF is also higher than that of CLBP$_{-}$S/M when combined with LBP. This experiment further reveals that the proposed neighbourhood feature NF-LBP$_{-}$NF can extract discriminative information from the local region.

Secondly, we set the number of training images k to 10 and choose (R, P) as (1, 8), (2, 16) and (3, 24), separately. As shown in Fig. 11, accuracy results of NF-LBP$_{-}$S/NF are always higher than CLBP$_{-}$S/M, thus proving that NF-LBP$_{-}$NF has superior classification performance.

Table 2 illustrates the accuracy results of ablation study by using three components of our proposed NF-LBP descriptor, namely NF-LBP$_{-}$S, NF-LBP$_{-}$NF, and NF-LBP$_{-}$C, and D represents the corresponding the dimensionality of feature histogram. It can be observed that the highest texture classification accuracy is achieved when all parts are used (i.e. NF-LBP), which can further prove the overall complementarity among these components. Additionally, by comparing the classification accuracy of different combinations, we can also observe the mutually complementary relationships between two corresponding parts. Combining the sign part and neighbourhood feature part (i.e. NF-LBP$_{-}$S/NF) consistently outperforms than other combining methods. Similarly, using the sign component and centre pixel component (i.e. NF-LBP$_{-}$S/C), as well as the neighbourhood feature part and centre pixel part (i.e. NF-LBP$_{-}$NF/C), also demonstrate improved classification accuracy. Therefore, we can observe from the ablation results that the three parts of the proposed NF-LBP exhibit complementary relationships, and the best texture classification accuracy is always achieved when all parts are utilized.

3.5 Dissimilarity measure

In order to calculate the similarity between different texture images, we adopt Chi-square statistics $\chi ^2$ as the distance between different extracted feature histograms in this paper. If $M = {m_{i}}$ and $N = {n_{i}}$ represent two feature histograms of two texture images, respectively, the $\chi ^2$ statistics can be calculated as follows:

$$\begin{aligned} {\text {Dis}}_{\chi ^{2}}(M,\ N))=\sum _{i=1}^{B}\frac{(m_{i}-n_{i})^{2}}{m_{i}+n_{i}} \end{aligned}$$

(9)

where B is the dimension of these two histograms M and N.

The effectiveness of content-based feature descriptors can be evaluated by the accuracy of texture classification. In the experiments, we used the nearest-neighbour classifier (NNC), which is determined by the ratio of accurately classified images to the total number of images, and this reflects the precision of the nearest-neighbour classifier in classifying texture images.

Moreover, we employed the F1 score as another performance assessment metric to demonstrate the superior classification capability of the proposed NF-LBP method. The F1 score is a widely recognized metric for assessing the performance of content-based image classification. It takes into account both precision P(L) and recall R(L). In our experiments, we firstly employ LBP-based algorithms to obtain a collection of the L highest-ranked images corresponding to every query image. We then determine the number of texture images that are relevant in both the top L list and corresponding texture database. At last, we can compute P(L) and R(L) using the following formulas:

$$\begin{aligned} \begin{aligned} P(L)=N_{r} / M\\ R(L)=N_{r} / N_{t} \end{aligned} \end{aligned}$$

(10)

where the count of relevant images obtained from the top L ranked images is denoted as $N_{r}$, while $N_{t}$ represents the number of relevant images of entire texture database corresponding to query images.

As the average evaluation metric depending on P(L) and R(L), F1 score f(L) is calculated as follows:

$$\begin{aligned} f(L) = \frac{P(L) \times R(L)}{P(L) + R(L)} \times 2 \end{aligned}$$

(11)

4 Experimental results and analyses

In order to showcase effectiveness and superior performance of our proposed NF-LBP algorithm in texture classification, we conducted a range of experiments using six representative texture databases, which include UIUC [49], CUReT [50], Outex [48], XU$_{-}$HR [51], and ALOT [52]. Table 3 summarizes key characteristics of the chosen five databases.

In order to demonstrate the performance and efficacy of our proposed NF-LBP method, we compare its classification accuracy with that of the LBP algorithm and several widely used and notable LBP variations LBP variants, including LBP [9], CLBP [12], LTP [10], CLBC [13], CRDP [25], BRINT [17], LNDP [28], CMPE [29], and LNIRP/LBP/DCI (denoted as LNIRP) [26]. In following subsections, R indicates selected sampling radius, and P indicates neighbourhood pixels’ number. In the following result tables of classification accuracy, the bold numbers represent the highest classification accuracy of all LBP-based algorithms.

4.1 Detailed analyses of the computational complexity and feature histogram dimension of the NF-LBP

In this subsection, we discuss the computational complexity and feature histogram dimension of the proposed NF-LBP descriptor and analyse them experimentally, as the algorithm’s complexity can significantly impact its evaluation. However, accurately calculating the computational complexity of texture classification algorithms can be challenging and is not always represented by a simple formula. To address this, we consider the runtime as the computational cost of the LBP-based algorithm, which allows for a direct comparison of computational complexity among LBP-based algorithms.

A series of experiments are conducted on the CUReT texture database using a PC equipped with an AMD Ryzen 5 4600U, 2.1 GHz CPU, 16GB RAM, and MATLAB R2019b. There are 61 texture classes in CUReT database, each containing 92 different texture images. For our runtime tests, we randomly select 46 images from each class for training and use the remaining 46 images for testing.

Generally, a texture classification algorithm typically consists of two primary stages: feature extraction and texture classification. Thus, the computational complexity includes the time for both stages. Table 4 presents the average time taken for feature extraction of one image on the CUReT database for the original LBP, LBP variants, and the proposed NF-LBP. Table 5 shows the average time for texture classification of one image on the CUReT database, as well as the feature histogram dimension of the original LBP, LBP variants, and the proposed NF-LBP.

Table 3 A prospectus of each representative texture database’s key characteristic

Table 4 Average runtime (s) of feature extraction for one image on CUReT database

Table 5 Feature histogram dimension of LBP-based algorithm and their average runtime (s) of texture classification for one image on CUReT database

Table 4 shows the results of the runtime analysis of the proposed NF-LBP and other LBP-based methods for feature extraction on the CUReT texture database. The computational time of NF-LBP is linearly related to the radius R and the number of neighbourhood pixels P. For instance, with $R = 1$ and $P=8$, NF-LBP can extract texture features from an image in the CUReT database in 0.0397 s, and the time increases to 0.1356s for $R=3$ and $P=24$. Although the computational cost of NF-LBP for feature extraction exceeds that of the original LBP and its variants, excluding LNIRP, due to its extraction of neighbourhood features and local adaptive thresholding method for neighbourhood pattern encoding, it does not grow exponentially, and it is an acceptable cost considering its strong complementarity and excellent classification capability.

Following feature extraction, a feature histogram is generated to represent each texture image. For feature classification stage, LBP-based algorithms utilize the $\chi ^2$ statistics to measure the dissimilarity between two histograms, as discussed in Sect. 3.5. As shown in Eq. (9), the computational cost of texture classification is closely tied to the dimensionality of the feature histogram. Therefore, the dimension of the feature histogram should be taken into account when evaluating the computational complexity of texture classification algorithms.

The results shown in Table 5 confirm that the runtime of the classification process is influenced by the dimensionality of the feature histogram. Significantly, when the feature dimension is large, the runtime of texture classification surpasses that of feature extraction. With R = 3 and P = 24, the runtime of NF-LBP for texture classification is 1.1865s, and that of CLBP for texture classification is also 1.1865s. Despite the increased computational cost for feature extraction, the runtime of NF-LBP is still lower than that of texture classification. Furthermore, feature histogram dimension of MC-LBP is as large as that of CRDP and CLBP, resulting in the same runtime for texture classification. Although the feature histogram dimension of NF-LBP is larger than that of other LBP variants, the additional feature dimension brings about improvements in classification capability. Thus, the increased feature histogram dimension is justified.

Using our proposed NF-LBP descriptor incurs additional computational cost. However, it enables a comprehensive extraction of local features by effectively utilizing neighbourhood features, thereby improving the discrimination capability of texture classification. Nonetheless, the program code used in our experiments is not fully optimized, nor is it parallelized. In future work, we will focus on optimizing the program code for large texture images and implementing LBP-based methods on GPU to improve their speed.

4.2 Experimental results and analyses on UIUC database

Figure 12 shows a collection of 25 high-definition texture image categories from the UIUC database. This database consists of 40 texture images in each class, all captured in real-world environments with notable variations in viewing angles. Therefore, texture classification on the UIUC database poses a significant challenge. We select N images randomly from each class for training, where N is set to 5, 10, 15, or 20. The remaining 40 - N images are then used for testing. To evaluate the algorithm’s texture classification capability, we conduct experiments by choosing N as 5, 10, 15, and 20. To ensure reproducibility, we repeat each experiment 100 times. The accuracy result is determined by calculating average classification accuracy.

Fig. 12

The 25 different texture classes on UIUC

As presented in Table 6, NF-LBP, CLBP, and CLBC all exhibit significantly higher accuracy compared with other LBP-based algorithms. Compared to CLBC, the accuracy of the NF-LBP is 4.25%, 3.94%, 5.08%, and 4.65% higher when R = 3 and P = 24. Moreover, NF-LBP shows more than a 9% improvement in classification accuracy over BRINT across all (P, R). Additionally, in the case of (R, P) = (1, 8), LNIRP’s classification accuracy is lower than NF-LBP by 9.18%, 9.89%, 11.62%, and 14.43%, respectively. The comparison of classification accuracy demonstrates the superior performance of the NF-LBP.

Table 6 Texture classification accuracy (%) on UIUC

Table 7 shows the F1 scores of LBP and its variants on UIUC. We can observe that NF-LBP outperforms all other LBP variants. The NF-LBP achieves F1 scores of 62.03%, 70.31%, 75.78%, 76.15%, 63.32%, 72.20%, 78.31%, 79.25%, 63.68%, 72.48%, 78.31%, and 78.73% for different combinations of R and P. Comparing NF-LBP with other LBP variants, it is clear that NF-LBP significantly outperforms all other methods in terms of F1 scores. For example, the F1 scores of NF-LBP are about 3${-}$16% higher than the second-best performing method, CLBP, across all combinations of R and P. The F1 scores of NF-LBP are also significantly higher than other LBP variants. In summary, the proposed NF-LBP method achieves the best F1 scores on the UIUC texture database compared to all other LBP variants, demonstrating its superior performance in texture classification tasks.

Table 7 The F1 scores (%) on UIUC

4.3 Experimental results and analyses on CUReT database

Figure 13 illustrates that CUReT database comprises 61 distinct image classes, which were captured under varying external conditions including illumination, viewing angles, and rotations. Each class includes 92 different texture images. N images of each class are chosen randomly for training, where N is set to 6, 12, 23, or 46. The remaining images (92 $-\, N$) are then used for testing. We conduct per experiment 100 times. The mean accuracy will be computed as the outcome.

Fig. 13

The different texture classes on CUReT

Table 8 presents the classification accuracy, where 46, 23, 12, and 6 indicate the values of N.

As the classification accuracy presented in Table 8, among the LBP-based methods, compared to LBP and LTP, NF-LBP, CLBP, CLBC, CRDP, BRINT, LNDP, CMDP, and LNIRP exhibit significantly superior classification accuracy. Notably, BRINT shows better performance when R = 1. The proposed NF-LBP achieves the highest classification accuracy for (R, P) = (1, 8) or (2, 16), surpassing CLBP, CLBC, CRDP, BRINT, LNDP, CMPE, and LNIRP. The best classification accuracy for NF-LBP is 96.85%, 93.41%, 87.28%, and 77.04%, respectively, for N of 46, 23, 12, and 6. On CUReT database, NF-LBP attains its best classification accuracy at radius R = 2 with the same dimensionality of CLBP and CRDP. The comparison results on CUReT database demonstrate the effectiveness and outstanding performance of the proposed NF-LBP in texture classification.

Table 8 Texture classification accuracy (%) on CUReT

Table 9 shows the F1 scores on the CUReT database for different LBP variants. Among all the LBP-based algorithms, NF-LBP has the highest F1 score. Specifically, with $(R, P) = (1, 8)$, NF-LBP outperforms other algorithms with the F1 scores of 64.29%, 79.56%, 80.49%, and 74.04%. When $R=2$ and $P=16$, NF-LBP also performs the best, achieving F1 scores of 64.52%, 80.01%, 81.13%, and 74.51%. At $R=3$ and $P=24$, NF-LBP achieves F1 scores of 63.15%, 77.61%, 77.71%, and 70.71%, which is lower than the best-performing algorithm, CLBP. Overall, based on the F1 scores on the CUReT database, NF-LBP demonstrates superior classification performance compared to the majority of other LBP variants.

Table 9 The F1 scores (%) on CUReT

4.4 Experimental results and analyses on Outex database

The popular Outex database consists of two subsets of images, namely Outex$_{-}$TC10 and Outex$_{-}$TC12. The sample images of Outex database are shown in Fig. 14. Each subset includes a total of 24 texture image classes, which were captured with three distinct illuminations (“inca”, “tl84”, and “horizon”) and nine unique rotation angles ($0^{\circ }$, $5^{\circ }$, $10^{\circ }$, $15^{\circ }$, $30^{\circ }$, $45^{\circ }$, $60^{\circ }$, $75^{\circ }$, and $90^{\circ }$), with each rotation angle under a given illumination condition having twenty corresponding texture images. For the Outex$_{-}$TC10 subset, images captured with “inca” illumination and a rotation angle of $0^{\circ }$ are used for training, while the images with illumination “inca” are employed for testing. On the Outex$_{-}$TC12 subset, we select texture images obtained with illumination “tl84” and “horizon” for testing. In this case, the images for training are chosen exactly the same as for the Outex$_{-}$TC10 subset.

Fig. 14

The 24 different texture classes on Outex

The accuracy result of LBP-based algorithms is presented in Table 10, where ‘h’ represents the “horizon” illumination condition, and ‘t’ represents the “tl84” illumination condition.

Table 10 showcases the superior texture classification accuracy achieved by our proposed NF-LBP algorithm in comparison with other LBP-based methods. On Outex$_{-}$TC10 image subset, NF-LBP has the highest accuracy results of 98.44%, 99.45%, and 98.96%, which is 1.93%, 0.8%, and 0.03% higher than CLBP, and 1.64%, 1.25%, and 0.97% higher than CRDP, respectively. Compared with LNDP, NF-LBP achieves 3.75%, 10.37%, and 10.44% higher classification accuracy when (R, P)=(3, 24) on three different subsets, respectively. Hence, the NF-LBP method demonstrates outstanding texture discrimination performance.

Table 10 Texture classification accuracy (%) on Outex

Table 11 shows the F1 scores of compared LBP-based methods on Outex. When considering (R, P) = (1, 8), the NF-LBP algorithm achieved an F1 score of 92.65% on the TC10 dataset, which is the highest score achieved by any algorithm in this category. In contrast, the LBP algorithm achieved an F1 score of 79.88%, which is significantly lower than NF-LBP. Similarly, on the TC12 dataset, the NF-LBP algorithm achieved an F1 score of 85.59%, which is higher than all other LBP variants, except for CLBP, CMPE and LNIRP. When (R, P) = (2, 16), the NF-LBP algorithm achieved the highest F1 score of 93.60% on the TC10 dataset, which is significantly higher than the LBP algorithm’s F1 score of 84.14%. Similarly, on the TC12 dataset, the NF-LBP algorithm’s F1 score of 88.05% is higher than most of other LBP variants, except for CRDP, CMPE, and LNIRP. Finally, when (R, P) = (3, 24), the NF-LBP algorithm achieved an F1 score of 93.01% on the TC10 dataset, which is higher than all other LBP variants, except for CRDP and CMPE. Similarly, on the TC12 dataset, the NF-LBP algorithm achieved an F1 score of 89.19%, which is higher than all other LBP variants, except for CLBP and LNIRP.

These comparison results suggest that the proposed NF-LBP algorithm is able to capture more discriminative texture features than these existing algorithms. Compared to CLBP and CRDP, which achieve the highest F1 scores in some cases, NF-LBP has comparable performance. However, it is worth noting that the computation complexity of NF-LBP is much lower than that of CRDP, as shown in Table 4. Therefore, NF-LBP is a more efficient alternative for practical applications where computational resources are limited. In conclusion, NF-LBP demonstrates superior performance compared to several LBP variants on the Outex texture database, and it is an efficient and effective approach for texture analysis.

Table 11 The F1 scores (%) on Outex

4.5 Experimental results and analyses on XU$_{-}$HR database

Fig. 15

The 25 different texture classes on XU$_{-}$HR

XU$_{-}$HR database consists of similar images as those on UIUC, but with a larger size of 1280 $\times$ 960, which is shown in Fig. 15. The XU$_{-}$HR database has the same size and factors contributing to intra-class variations as UIUC, containing 25 diverse classes. Every class comprises 40 texture images, with notable changes in viewing angles, illumination, and scales. N images are selected randomly from each class for training, where N is set to 5, 10, 15, or 20. The remaining images (40 $- N$) are then used for testing. We conduct per experiment 100 times and compute the deciding result by calculating the average accuracy of these 100 experiments.

The accuracy on XU$_{-}$HR is presented in Table 12. As the average classification results shown in Table 12, NF-LBP achieves the best performance compared to other LBP-based algorithms for all parameter settings of (R, P). Additionally, NF-LBP achieves superior performance compared to CLBP and CRDP, despite having the same histogram size (the histogram dimensionality for NF-LBP, CLBP, and CRDP at R = 1, 2, and 3 are 200, 648, and 1352, respectively). Compared with CMPE, classification accuracy of NF-LBP is 1.64%, 2.16%, 2.93%, and 4.41% higher when R = 1. Hence, the comparison results on XU$_{-}$HR serve as further evidence of the outstanding classification performance of the NF-LBP, showcasing its state-of-the-art capabilities.

Table 12 Texture classification accuracy (%) on XU$_{-}$HR

Table 13 The F1 scores (%) on XU$_{-}$HR

Table 13 shows the F1 scores of LBP and its variants on XU$_{-}$HR. The table presents the results for three sets of LBP parameters, namely (R, P) = (1, 8), (2, 16), and (3, 24). Comparing the results, we observe that NF-LBP achieves the highest F1 scores among all LBP variants for all parameter settings. For example, for (R, P) = (3, 24), NF-LBP achieves the highest F1 score of 86.01%, which is higher than the best-performing alternative, CRDP with 85.09%, by a margin of 0.92%. Furthermore, when R = 1, F1 scores of NF-LBP are, respectively, 0.31%, 0.84%, 1.53%, and 2.46% higher than those of LNIRP. The results show that NF-LBP outperforms all other LBP variants across different parameter settings. Among the other LBP variants, CRDP achieves the second-best performance, followed by CMPE, BRINT, CLBP, CLBC, LTP, and LNDP, in decreasing order of F1 scores. In summary, NF-LBP outperforms all other LBP variants across different parameter settings, indicating its effectiveness for texture classification tasks.

Fig. 16

The different texture classes on ALOT

Table 14 Texture classification accuracy (%) on ALOT

Table 15 The F1 scores (%) on ALOT

4.6 Experimental results and analyses on ALOT database

Figure 16 presents a small sampling of the ALOT database, which comprises 250 classes of texture images. Each class contains 100 texture images with variations in viewing angles and illumination. In our experiments, we convert all texture images on ALOT database to greyscale. We select N images of each class at random for training. The rest images (100 $- N$) are then used for testing. We conduct experiments 100 times to eliminate randomness in classification accuracy and get the deciding result by calculating the mean accuracy of these 100 experiments.

Table 14 shows the classification accuracy of all compared methods on ALOT texture database. Based on the results in Table 14, it can be observed that the conventional LBP and most of its variants are outperformed by our proposed NF-LBP method. For instance, when $(R,\ P)$ = (1, 8), NF-LBP attains higher classification accuracy of 4.65%, 6.78%, 7.66%, and 8.17% than the BRINT algorithm and of 1.57%, 1.65%, 1.49%, and 1.72% than the CRDP algorithm. Compared with LNIRP, NF-LBP can achieve an average 2.08% higher classification accuracy. The results on ALOT further confirm the extraordinary performance of the NF-LBP method.

Table 15 shows the F1 scores of compared LBP-based algorithms on ALOT, evaluated at different values of R and P. Among the LBP variants evaluated, the NF-LBP algorithm consistently achieves the highest F1 score across all parameter settings. Specifically, NF-LBP outperforms all other algorithms with respect to all values of R and P, achieving an F1 score of 63.69% at $R=1$ and $P=8$, and an F1 score of 68.23% at $R=3$ and $P=6$. It is worth noting, however, that some of these algorithms, such as CLBP, CRDP, and LNIRP, still achieve relatively high F1 scores compared to others, indicating their usefulness in certain texture recognition applications.

4.7 Experimental results and analyses of noise robustness to Gaussian noise

To test the noise robustness of the proposed NF-LBP to Gaussian noise, we conduct experiments on two texture databases: Outex$_{-}$TC10 and CUReT. Specifically, in our experiments, 46 images in each texture class are chosen at random for training with the rest 46 images for testing on CUReT database. Additionally, the greyscale values of each image are normalized in the range of [0, 1]. Next, we add Gaussian noise of mean zero and variance ranging from 0.005 to 0.025 with the interval of 0.005. As demonstrated in Fig. 17, the increasing variance $\sigma$ of Gaussian noise results in increasingly illegible texture images, thereby making classification more difficult.

Fig. 17

A Gaussian-noised sample image on CUReT database with different Gaussian noise variance $\sigma$

To evaluate the noise robustness of the proposed NF-LBP, we compare its classification accuracy with the representative LBP-based algorithms, namely the conventional LBP, CLBP, and BRINT against Gaussian noise. The accuracy with different sampling radius R and under different variance $\sigma$ of Gaussian noise is, respectively, presented in Figs. 18, 19, 20, 21, 22, and 23.

Fig. 18

Classification accuracy when R = 1 with the different Gaussian noise variance $\sigma$ on CUReT

Fig. 19

Classification accuracy when R = 1 with the different Gaussian noise variance $\sigma$ on Outex$_{-}$TC10

Fig. 20

Classification accuracy when R = 2 with the different Gaussian noise variance $\sigma$ on CUReT

Fig. 21

Classification accuracy algorithms when R = 2 with the different Gaussian noise variance $\sigma$ on Outex$_{-}$TC10

Fig. 22

Classification accuracy algorithms when R = 3 with the different Gaussian noise variance $\sigma$ on CUReT

Fig. 23

Classification accuracy algorithms when R = 3 with the different Gaussian noise variance $\sigma$ on Outex$_{-}$TC10

Figures 18, 19, 20, 21, 22, and 23 illustrate that compared with LBP, CLBP, and BRINT, NF-LBP is less sensitive to Gaussian noise for all parameter pairs of (R, P) on CUReT and Outex$_{-}$TC10. For instance, in Fig. 20, when R is 2, the accuracy of the proposed NF-LBP decreases at a slower rate than that of LBP, CLBP, and BRINT as the variance $\sigma$ increases. In Fig. 19, our proposed NF-LBP shows stronger noise robustness than BRINT, especially when the variance $\sigma$ is larger. In Fig. 23, accuracy results reveal that the classification performance of NF-LBP and CLBP is very similar without Gaussian noise on Outex${-}$TC10 database when R = 3. However, with increasing Gaussian noise variance $\sigma$, the advantages of the proposed NF-LBP become more apparent, resulting in better classification performance than LBP and CLBP.

4.8 Experimental results and analyses of noise robustness to salt-and-pepper noise

We conduct noise-robustness experiments on CUReT texture database to demonstrate the effectiveness of the NF-LBP against salt-and-pepper noise. Specifically, we choose 46 images of every texture class at random, which are considered as training images. Meanwhile, the rest texture images are chosen as test images on CUReT. As shown in Fig. 24, we add the salt-and-pepper noise into the texture images with noise density $\nu$ ranging from 5% to 25% with the interval of 5%. As $\nu$ increases, the identification of texture images becomes more challenging, making the classification task more difficult.

Fig. 24

A sample texture image on CUReT with salt-and-pepper noise at varying noise density $\nu$

We compare the classification accuracy obtained from the proposed NF-LBP with that obtained from the conventional LBP, CLBP, and BRINT. The accuracy on CUReT under increasing noise density $\nu$ is presented in Figs. 25, 26, and 27.

Fig. 25

Classification accuracy when R = 1 with the different salt-and-pepper noise density $\nu$ on CUReT

Fig. 26

Classification accuracy when R = 2 with the different salt-and-pepper noise density $\nu$ on CUReT

Fig. 27

Classification accuracy when R = 3 with the different salt-and-pepper noise density $\nu$ on CUReT

The classification results of three different local binary pattern (LBP) methods are evaluated on CUReT under the presence of salt-and-pepper noise, namely LBP, CLBP, BRINT, and the proposed NF-LBP, with different sampling radius R. The results are, respectively, presented in Figs. 25, 26, and 27. In Figs. 25, 26, and 26, we can observe that NF-LBP presents better classification performance than other LBP-based algorithms, especially with $\nu$ increasing. As depicted in Fig. 27, the classification accuracy of NF-LBP is not as high as that of CLBP at low salt-and-pepper noise density $\nu$ when R = 3. However, as $\nu$ increases, NF-LBP outperforms CLBP in terms of classification performance, owing to its superior noise robustness. Furthermore, Figs. 25, 26, and 27 reveal that NF-LBP exhibits a slower decrease in classification accuracy as $\nu$ increases, highlighting its robustness to salt-and-pepper noise.

5 Conclusion

We analyse several existing LBP-based texture descriptors, including the conventional LBP, the CLBP, and the BRINT in this paper. We highlight neighbourhood texture features and introduce the neighbourhood feature (NF) pattern for describing greyscale changes in neighbourhood intensity. Consequently, we combine this pattern with the traditional local sign component to create a remarkably discriminative and robust descriptor for texture classification, called the neighbourhood feature-based local binary pattern (NF-LBP). The addition of the NF pattern provides stronger complementarity to LBP and can enhance the noise robustness of the algorithm. Furthermore, we propose an adaptive local thresholding to encode the NF pattern and the centre pixel into the corresponding binary strings. Three binary codes can be used to construct the resulting NF-LBP histogram in a joint 3D manner, allowing for straightforward construction: the sign part (NF-LBP$_{-}$S), the neighbourhood feature part (NF-LBP$_{-}$NF), and the centre pixel (NF-LBP$_{-}$C). The NF-LBP method exhibits remarkable effectiveness and exceptional classification capabilities when compared with the conventional LBP and its representative LBP variants, including LBP, CLBP, LTP, CLBC, CRDP, BRINT, LNDP, and CMPE, supported by a comprehensive set of experiments conducted on five well-known texture databases, namely UIUC, CUReT, Outex, XU$_{-}$HR, and ALOT. Extensive experimental results also demonstrate that NF-LBP maintains high discriminative capability even under the complex external conditions with noise.

While the NF-LBP method exhibits remarkable effectiveness and exceptional classification capabilities compared to other LBP variants, we acknowledge certain limitations and limited scope of our current work: (1) The proposed NF-LBP method, although effective, may have high computational complexity and relatively high dimensionality of the resulting feature histograms due to its extraction of three separate local features and the joint construction of three binary codes (i.e. NF-LBP$_{-}$S, NF-LBP$_{-}$NF, and NF-LBP$_{-}$C). (2) While our research primarily focuses on texture extraction, analysis and classification, the potential applications of the NF-LBP method extend beyond this domain.

Considering the existing drawbacks of the work, our future research will focus on the following aspects:

• 1. Reductions in both computational complexity and the dimensionality of feature histograms will be implemented to make the proposed method yield more benefits.

• 2. We aim to extend the application of NF-LBP to other fields, such as object detection and face recognition. Additionally, with the increasing use of visual media methods in 3D data, our future plans involve integrating NF-LBP into 3D engineering data to enhance its practicality and broaden its application scope.

• 3. Considering the CNN model’s robust texture extraction and classification capabilities and the computational efficiency of hand-crafted descriptors, our future research will also focus on exploring the integration of the CNN architecture with traditional LBP-based algorithms. This integration will allow for the exploration of the benefits of both techniques while minimizing the computational cost and required training time.