Keywords

1 Introduction

In recent years, with the continuous improvement of environmental problems, the coverage of many plants that quickly release a large amount of highly allergic pollen has increased year by year, such as poplar, resulting in the incidence of pollen allergic diseases. At present, the environmental departments of many countries in the world are gradually developing air pollen. The monitoring and forecasting services of pollen species are also attempting related work in some cities in China. However, the current pollen classification and identification mainly rely on manual operation under a microscope. This method is inefficient compared with the use of computers to classify and identify pollen. It is time-consuming and labor-intensive and is affected by the operator’s subjective experience, and the recognition rate is generally not high. The pollen images under the microscope have different morphologies. Different pollen images have a high degree of distinction in the structure and texture, so the computer is used Pollen recognition can effectively improve the accuracy and efficiency of pollen classification.

Many scholars have conducted in-depth research on pollen image recognition in recent years and have achieved specific research results. The Treloar team designed an automatic classification and recognition algorithm for pollen images based on the geometric features of pollen and achieved a 95% recognition rate on the best subset of variables [1]. The Rodriguez-Damian team combined the texture features and shape features of the pollen. In the classification of Urticaceae pollen, a recognition rate of 89% was obtained. Da Silva’s team combined machine learning and image classification and recognition algorithms, using wavelet transform (WT) for texture feature extraction, and achieved good recognition results. The Kong team designed an automated sub-recognition method (Combined Global Shape and Local Texture, GSLT) that matches the global shape feature on the image block with the local texture feature and has a good effect on recognizing the fossil pollen data set. The Daood team proposed a new idea, the classification and recognition method (MultitLayer Feature Decomposition, MLFD), which first divides the image into layers and then extracts each layer’s texture and geometric features, achieved some results in recognition of ancient pollen samples [2].

At present, most texture feature extraction methods have a specific recognition ability for certain unique pollen textures. Still, they are not universal enough to be applied to the recognition of all pollen images. The method proposed by the Rodriguez-Damian team has high time complexity and feature dimensionality. The method proposed by the Treloar team has too many strict requirements on the shape and contour of the pollen. The Da Silva team only uses wavelet transform to extract the texture of the pollen image. The way of Kong’s team is not effective in traditional pollen recognition. The technique used by the Daood team is not robust to the rotation of pollen.

This paper proposes a pollen image recognition method using local binary patterns to respond to the above problems. The process first performs preprocessing, such as sharpening and normalization of the pollen image. Then calculates the regional binary pattern for the preprocessed image. Finally, extract the results. The statistical histogram descriptor of the calculation results of the local binary mode is used as the identification feature, and finally, SVM is used as the classifier for the classification and recognition of the three-dimensional pollen image. Through experimental verification, the method proposed in this paper can achieve high recognition efficiency while retaining the complete information of pollen images [3].

2 Methods

2.1 Traditional Local Binary Pattern

This local binary pattern is defined in the neighborhood of a 3 × 3 matrix. The target image is convolved with this matrix as a calculation template. The pixel value of each district of the center pixel is binarized with the center pixel as the threshold: The position of the center pixel is more significant than code 1. Otherwise, it is coded to form a local binary numerical matrix. Obtain the values in the matrix clockwise from a given point to obtain a binary string, and convert the string to decimal. The number is used to mark the central pixel uniquely, then the LBP value defined at this point is expressed as:

$$ LB{\text{P}}_{R} (x_{c} ) = \sum\limits_{n = 0}^{R} {2^{n} s(x_{n} - x_{c} )} $$

Where R is the neighborhood range, xn is all pixels contained in the neighborhood under this range, xc is the sampling center point, and s(x) is:

$$ s(x) = \left\{ \begin{gathered} 1, \;\; x \ge 0 \hfill \\ 0, \;\; x < 0 \hfill \\ \end{gathered} \right. $$

The simplest form of the LBP feature vector is established as follows:

  1. (1)

    Divide the detection window into small blocks (such as 16 * 16 small blocks);

  2. (2)

    For each pixel in the small block, compare its neighboring eight neighbors. Follow the pixels clockwise or counterclockwise along the circle;

  3. (3)

    If the value of the center pixel is greater than the neighbor’s value, write 0, otherwise write 1. This constructs an 8-digit number (usually, for convenience, it will be converted to a decimal);

  4. (4)

    Calculate the frequency histogram of each “number” in a small block;

  5. (5)

    The histogram can be standardized selectively;

  6. (6)

    Connect all the small (normalized) histograms. This constitutes the feature vector of the entire window (Fig. 1).

Fig. 1.
figure 1

Local Binary Pattern of pollen image

Full size image

2.2 Extended Local Binary Pattern

To adapt to the texture features of different scales and meet the grayscale and rotation invariance requirements, Ojala made improvements, extending the 3 × 3 neighborhood to any neighborhood and replacing the square with a circular area.

Fig. 2.
figure 2

Extended Local Binary Pattern of pollen image

Full size image

Where, \(LBP_{p}^{r}\) Represents the LBP local coding with p sampling points in a circular area with r as the radius, for a given center point (xc, yc), its neighborhood pixel position is (xp, yp), p ∈ P, and its sampling point (xp, yp) is calculated as follows:

$$ \begin{aligned} x_{p} & = x_{c} + R\cos (\frac{2p\pi }{P}) \\ y_{p} & = y_{c} - R\sin (\frac{2p\pi }{P}) \\ \end{aligned} $$

However, we will zoom in on the calculated coordinate position and assume that the image coordinates around the calculated position are from 0 to 1 or 1 to 0. The computed coordinates are not necessarily integers. How to determine its pixel value? In this case, we generally use interpolation calculation to calculate its reasonable pixel value. There are many interpolation methods at present, and the more widely used one is a bilinear interpolation. The formula for bilinear interpolation is as follows:

$$ f(x,y) \approx \left[ {\begin{array}{*{20}c} {1 - x} & \;\;\; x \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} {f(0,0)} & \;\;\; {f(0,1)} \\ {f(1,0)} & \;\;\; {f(1,1)} \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} {1 - y} \\ y \\ \end{array} } \right] $$

2.3 Uniform Local Binary Pattern

The LBP features already have gray and illumination invariance, but they do not yet have rotation invariance. To achieve rotation invariance, researchers have proposed a rotation invariance LBP feature. The main idea is rough: firstly, continuously rotate the LBP features in the circular neighborhood, obtain a series of LBP feature values according to the selection, and select the LBP feature with the smallest LBP feature value as the central pixel from these LBP feature values. The specific approach is shown in the figure below (Fig. 3):

Fig. 3.
figure 3

Uniform Local Binary Pattern of pollen image

Full size image

As shown in the figure, by rotating the obtained LBP features, a series of LBP feature values are obtained, and finally, the feature pattern with the smallest feature value is used as the LBP feature of the center pixel.

The statistical histogram of the LBP feature combines the LBP feature with the spatial information of the image. The LBP feature image is divided into m local blocks, and the histogram of each local block is extracted. Then these histograms are sequentially connected to form a statistical histogram of LBP features, that is, LBPH. The specific process of calculating LBPH for an image:

  1. (1)

    Calculate the LBP feature image of the image;

  2. (2)

    Divide the LBP feature image into blocks. Opencv divides the LBP feature image into eight rows, eight columns, and 64 areas by default;

  3. (3)

    Calculate the histogram cell LBPH of the feature image of each area, normalize the histogram, and the histogram size is 1 * n;

  4. (4)

    Arrange the histogram of each area feature image calculated above into a row according to the spatial order of the blocks to form the LBP feature vector, the size is one * (n * 64);

  5. (5)

    Use this vector as the classifier’s input to realize automatic classification of pollen images.

3 Experiential Results

The experimental environment of this paper is a PIV computer, 2.8 GHz CPU, and 8 GB memory. To verify the method’s effectiveness, the pollen image of the experiment in this paper uses the Confocal data set. It currently includes most of the image data of allergic pollen detected in the air. The data set uses a laser scanning confocal microscope to ensure data concentration. In this paper, SVM is used to calculate the similarity between features. 20% of each type of pollen is selected as the training sample, and the rest are used as the experimental sample.

Before calculating the LBP feature value, the original image needs to be interpolated or down-sampled according to the resolution of the original image. And then, image filtering and other preprocessing are performed to improve the image quality. In the experiment, the accuracy rate Correct Recognition Rate (CRR), Recall Rate (RR), and Recognition Time (RT) verify the performance of the experiment.

3.1 Experiential Results on Confocal Pollen Dataset

It can be seen in Fig. 2 that in the Confocal pollen image with minor deformation and pollution, the pollen image particles have apparent edge contours. As well as transparent textures, most of them can be correctly identified and classified (Fig. 4).

Fig. 4.
figure 4

Experiential results of proposed method

Full size image

Table 1 shows the classification and recognition performance of six representative pollen images. From the experimental results, it can be seen that the recognition rate on the Confocal image database is generally higher. The correct recognition rate can reach more than 90%, and the recall rate can reach 95%. For pollen images with relatively poor quality, such as Juglans, it can also get about 80%. From the recognition results, it can be concluded that the recognition performance of pollen will be affected by the quality of pollen to a certain extent. The wrong classification is mainly caused by the noise or the unclear sampling. The experimental results show that the LBP is not the spatial geometric transformation of pollen images has good invariance.

Table 1. Experiential results on confocal pollen dataset
Full size table

3.2 Results and Discussion

This paper compares the experimental results with the WT feature descriptor, GSLT feature descriptor, and MLFD feature descriptor. Because different types of pollen have various internal texture features such as thorns, nodules, rods, holes, nets, depending on the type of pollen, these features will become more evident in the three-dimensional pollen image. Hence, this article’s LBP texture feature extraction has more vital discrimination ability, better recognition effect, and higher recognition efficiency. Table 2 shows the comparison results of this method’s average recognition rate and recognition time and the other four methods.

Table 2. Comparison between PROPOSED METHOD and other methods
Full size table

It can be seen from the experimental results that compared with the other three methods, the average recognition rate can reach 88.46%, 9.9% higher than other similar methods on average. It is also superior to most methods in terms of computational efficiency. For example, compared with WT, the correct recognition rate of the method provided in this article is about 14.96% higher. Compared with GSLT, the method in this article has significantly reduced time and space complexity and improved recognition rate in the process of extracting 3D features. Compared with MLFD, the correct recognition rate of the method provided in this article is 2.42% higher. Experiments show that this method has a higher recognition rate and good geometric invariance than other texture feature extraction methods and is suitable for actual pollen classification and recognition.

4 Conclusion

This paper proposes an automatic classification method for pollen images based on local binary patterns. This method is based on the low-complexity LBP operator. It directly extracts the features of the image in the spatial domain, avoiding the processing of transforming the image to the frequency domain. The calculation step has high operating efficiency. This paper combines the gray change direction of the pollen image with the texture feature to extract the image feature with relatively low dimensionality and reasonable recognition rate and obtained in the experiment High pollen recognition rate. However, for partially occluded pollen images, especially pollen images in internal slice images of pollen images with a fuzzy texture. Experimental analysis shows that the classification method in this paper has strong pollen characteristics, representation ability, and distinguishability and has high robustness to illumination, rotation, and position changes of pollen.

In addition, due to the dimensionality reduction characteristics of the local binary mode in the calculation process of this paper, the local features of the source image are retained. At the same time, the computational efficiency is ensured so that the descriptor meets the requirements of real-time pollen classification. Further work will focus on how to solve the recognition problem of partially occluded pollen slice images and improve the recognition efficiency of pollen images.