Deep Recognition of Chinese Herbal Medicines Based on a Caputo Fractional Order Convolutional Neural Network

Abstract

Chinese herbal medicines are the treasure of Chinese traditional medicine, which contains active ingredients that have significant therapeutic effects for many diseases. To solve the fluctuation and inefficiency problems faced by the existing image recognition of Chinese herbal medicines relying on manual subjectivity, a deep classification and recognition method based on a fractional order convolutional neural network is proposed for the images of Chinese herbal medicines. This method introduces Caputo fractional order gradient descent to update the model parameters, which improves the accuracy of gradient descent in the process of training the model, and solves the problem that integer order gradient descent is prone to fall into the local optimal solution and thus leads to low accuracy. By establishing a multi-scale standard Chinese herbal medicines dataset, the feature recognition ability based on Caputo Fractional Order Convolutional Neural Network (CFO-CNN) is trained and tested. And at the same time, the fractional order backpropagation function is used, so that the model can effectively find the global optimal solution, and achieve the output of the feature deep recognition. Experiments show that the method can effectively achieve the deep classification recognition of Chinese herbal medicines varieties, species and grades.

1 Introduction

As an important part of traditional Chinese medicine, the active ingredients of Chinese herbal medicines can be used in the prevention and treatment of diseases, and it is also very important to accurately identify their traits. In traditional trait identification, Chinese herbal medicines are often classified and identified by human subjective behaviors such as tasting, sniffing, and eye observation. However, many herbs are morphologically similar but have different effects, e.g., Psyllium and Cuscuta, which makes it difficult to classify and identify herbs on a large scale using only the human appearance of the herbs.

The application of automation and intelligent technology to medical and health services is of great significance to enhance the development of medical care [1, 2]. To improve the accuracy and consistency of the trait identification technology of Chinese herbal medicines, the study of how to break through the traditional trait identification methods of Chinese herbal medicines using machine learning-based image recognition and classification techniques has become a hot research issue. Article [3] proposes the use of Attention Pyramid Networks for the recognition of Chinese herbal medicines, introducing competitive attention and spatial collaborative attention applied to the recognition of Chinese herbal medicines. The authors in the article [4] utilize a mutual triple attention learning approach that allows two student networks to collaborate on parameter updates. Meng Han et al. [5] classified Chinese herbal medicines through a new mutual learning model that could extract stronger and richer features without increasing the parameter size. Although the above deep learning methods can identify the image types of some Chinese herbal medicines, they do not consider that there are different varieties of the same kind of Chinese herbal medicines, and the same variety has different specifications, quality, and price. The traditional deep learning model-based image recognition and classification methods for Chinese herbal medicines are facing significant challenges.

Convolutional neural network (CNN) is one of the main deep learning models [6, 7], which has attracted much attention because of its outstanding advantages in image recognition and classification. CNN has been widely used to solve image recognition problems in many fields of production and life, such as portrait recognition [8], plant disease identification [9,10,11], QR code recognition [12], Human Activity Recognition [13], and so on. Meanwhile, optimizing CNN using fractional order is an important method in the field of deep learning, and some studies [14, 15] have shown that the fractional order gradient method can avoid the problem of the CNN model falling into the local optimal solution. Therefore, this paper investigates a Caputo fractional order Convolutional Neural Network (CFO-CNN) based method to achieve deep recognition of Chinese herbal medicines images. The main contributions of this paper include:

• 1. A model structure based on CFO-CNN is proposed. This model adopts Caputo fractional order gradient descent instead of integer order gradient descent to update the model parameters. By doing so, it addresses the issue of the integer order gradient descent algorithm’s susceptibility to local optima due to fine-grained defects. The optimization of the backpropagation algorithm improves the recognition accuracy of the model.

• 2. A convolutional neural network model incorporating the fractional order gradient descent method applied to Chinese herbal medicines is proposed. This model can achieve deep classification recognition of multiple Chinese herbal medicines varieties, species, and grades. It solves the problem of fluctuation and inefficiency faced by manual subjective identification of Chinese herbal medicines, which is of great significance in improving the clinical application of Chinese herbal medicines.

2 Approach

In this paper, a Caputo Fractional Order Gradient Descent Convolutional Neural Network (CFO-CNN) for deep classification and recognition of Chinese herbal medicines is proposed to extract features from the data of Chinese herbal medici-nes using a convolutional neural network. The fractional order gradient descent method is used to update the extracted parameter information in the model and continuously optimize the parameters in the feature extraction process to improve the classification and recognition of Chinese herbal medicines.

There is no publicly available dataset in the field of Chinese herbal medicines identification and classification. In this paper, we extracted images of various Chinese herbal medicines from web pages in different scenes and different backgrounds to form a dataset, and manually created the label information for each variety of herbs. The images of different varieties, species, and grades of different herbal medicines are used as datasets and input data x to the neural network model, and the feature expressions of herbal medicines are obtained through convolutional feature extraction

$$\begin{aligned} x_j^{(l)}=f\big (\sum _{(i=1)}^lw_{ij}^{(l)}*x_i^{(l-1)}+b_j^{(l)}\big ) \end{aligned}$$

(1)

where $x_j^{(l-1)}$ denotes the Chinese herbal medicines data information of the layer i channel of layer $l-1$, $f(\cdot )$ denotes the activation function, $w_{ij}^{(l)}$ denotes the weight of the convolutional layer $\textit{l}$, $b_j^{(l)}$ denotes the bias term of layer $\textit{l}$, and $x_i^{(l)}$ is the output of Chinese herbal medicines features of the j channel of convolutional layer $\textit{l}$.

In the backpropagation, the Loss function Loss updates the parameters learned by the network, and the gradient is updated in the traditional integer order of

$$\begin{aligned} w_{k+1}=w_k-\mu \frac{\partial Loss}{\partial w_k} \end{aligned}$$

(2)

$$\begin{aligned} b_{k+1}=b_k-\mu \frac{\partial Loss}{\partial b_k} \end{aligned}$$

(3)

where $w_k$ and $b_k$ denote the current weights and bias information, $w_{k+1}$ and $b_{k+1}$ denote the updated weights and bias information. $\mu$ denotes the learning rate of the gradient descent algorithm. In non-convex function problems, the objective function may have multiple local optimal solutions and only one global optimal solution. The fine-grained defects of the integer order gradient descent algorithm lead to falling into local optimal solutions in the gradient update. To improve the model identification accuracy of gradient descent and prevent the problem of falling into local optimal solutions during updating, the Caputo fractional order derivative is introduced, and the general form of Caputo fractional order derivative is obtained after the sum of partial integrals [16] as

$$\begin{aligned} _{a}^{C}D_{x}^{\alpha }f(t)=\sum _{n}^{\infty }\frac{f^{(n)}(t_{0})}{\varGamma (n+1-\alpha )}(t-t_{0})^{n-\alpha } \end{aligned}$$

(4)

where $_{a}^{C}D_{x}^{\alpha }$ is an operator of Caputo fractional order, $\alpha$ denotes the order, and when $\alpha$ is a positive integer, it denotes the integer order derivative in, the usual sense. When denoting the fractional order, it is necessary to satisfy $n-1<\alpha <n$, where $n\in N+$ and $\varGamma \left( \alpha \right) =\int _{0}^{\infty }\ x^{\alpha -1}e^{-x}dx$ is the Gamma function. Each time the feature parameters are updated by Caputo fractional order gradient descent. The updated characteristic parameter information of Chinese herbal medicines that can be obtained by combining Eq. (2), Eq. (3) and Eq. (4) can be expressed as

$$\begin{aligned} w_{k+1}^{*}=w_{k}-\mu _{a}^{C}D_{x}^{\alpha }Loss(w) \end{aligned}$$

(5)

$$\begin{aligned} b_{k+1}^{*}=w_{k}-\mu _{b}^{C}D_{x}^{\alpha }Loss(b) \end{aligned}$$

(6)

where $w_{k+1}^*$ and $b_{k+1}^*$ denote the updated weights and biases of the fractional order gradient descent method, $w_k$ and $b_k$ denote the current weights and biases, $\mu _{a}^{C}D_{x}^{\alpha }Loss(w)$ denotes the derivatives of the Caputo fractional order concerning the current weights w, and $\mu _{b}^{C}D_{x}^{\alpha }Loss(b)$ denotes the derivative of the Caputo fractional order concerning the current bias b. After Taylor expands the Caputo fractional order and retains the results of the first term, Eq. (5) and Eq. (6) can be rewritten as follows

$$\begin{aligned} w_{k+1}^{*}=w_{k}-\mu \frac{f^{(1)}(w_{k}-1)}{\varGamma (2-\alpha )}|w_{k}-w_{k-1}|^{(1-\alpha )} \end{aligned}$$

(7)

$$\begin{aligned} b_{k+1}^{*}=b_{k}-\mu \frac{f^{(1)}(b_{k}-1)}{\varGamma (2-\alpha )}|b_{k}-b_{k-1}|^{(1-\alpha )} \end{aligned}$$

(8)

where $w_{k-1}$ and $b_{k-1}$ denote the weights and biases of the last time. After many iterations, the feature parameters of the final output image of Chinese herbal medicines are classified by Softmax function, and the probability of different varieties, species and grades under different species of Chinese herbal medicines obtained can be expressed as follows

$$\begin{aligned} P=\left\{ \begin{array}{llllllllll}{{Softmax{\bigl (}\sum _{v=1}^lw_v^{*(l)}x^{(l-1)}+b_v^{*(l)}),x\in x_{variaty}}}\\ {{Softmax{\bigl (}\sum _{s=1}^lw_s^{*(l)}x^{(l-1)}+b_s^{*(l)}),x\in x_{specias}}}\\ {{Softmax{(\sum _{g=1}^lw_g^{*(l)}x^{(l-1)}+b_g^{*(l)}),x\in x_{grade}}}}\end{array}\right. \end{aligned}$$

(9)

$\textit{P}$ denotes the probability of identifying different varieties, species and different grades of herbs under the same species. $x_{variety}$ denotes the variable data of herbal medicines, $x_{species}$ denotes the species data of Chinese herbal medicines, $x_{grade}$ denotes the data of different grades of Chinese herbal medicines under the same type of herb. The deep classification and recognition method of Chinese herbal medicines based on Caputo fractional order convolutional neural network is shown in Fig. 1

Fig. 1.

Deep classification recognition system for Chinese herbal medicines based on Caputo fractional order convolutional neural network

3 Experiments

Through tests of Chinese herbal medicine picture classification and identification, the study is methodically assessed. Image recognition shows this method’s benefits over previous deep learning-based categorization approaches for Chinese herbal medicines.

Chinese herbal medicine image identification is not restricted to a particular setting, one event, or even one database [17]. Here, the experiment’s data consists of randomly selected photos of Chinese herbal medicines against various backdrops and scenarios from the Chinese herbal medicines dataset. The data contains nine representative Chinese herbs, each with about 700 images, and a total of about 6300 images of Chinese herbal medicines as training data and about 100 images as test data, which are annotated by manually labeling the data with labels that include the variety, species and grade of the Chinese herbal medicines. After loading the Chinese herbal medicines dataset, the data resolution is set to 224 $\times$ 224, and the step size of each experiment is 16. Convolutional pooling is used to down-sample the images of Chinese herbal medicines; the fully connected layer provides parameter information in dimensionality reduction; fractional order gradient optimization is used to optimize the parameters for backpropagation; and lastly, visualization is applied to the experimental results. The parameter settings of the CFO-CNN model used in this study are shown in Table 1.

The model uses five convolutional layers and pooling layers to downsample the input data, the first fully connected layer spreads the down-sampled feature maps into one-dimensional vectors, and the second fully connected layer outputs the dimensions of the number of varieties of herbs to be recognized for the final classification prediction of Chinese herbal medicines.

Table 1. Parameters of the CFO-CNN model

From the test dataset, a random selection of Chinese herbal medicine images was used to create test data, which the CFO-CNN model then categorized. Fig. A identified this set of Chinese herbal data as Bark, Leaf, Hawthorn, Tuckahoe, Medlar, Ganoderma lucidum, Rattan, Cordyceps, and Panax notoginseng. From the figure, it can be seen that the CFO-CNN proposed in this study can classify accurately all the Chinese herbal medicines given by the test.

Three plants were identified as belonging to the Ginseng variety-Panax ginseng, Codonopsis, and Radix glehniae-after deep recognition of the Chinese herb varieties was carried out, as shown in Fig. B. Three herbs were recognized under the variety of Ginger as Dried ginger, Curcuma and Baked. Three herbs were identified under the variety of Chrysanthemum, namely Wormwood, Taraxacum, and Daisy. The CFO-CNN is very accurate for deep recognition of different species in the variety of Chinese herbal medicines as shown in Fig. B.

Fig. C demonstrated the analysis of different classes of herbs according to their respective classes, in which it was identified that in Ginseng, Panax ginseng belongs to the first class, Radix glehniae is second class, Codonopsis belongs to the third class. Ginger Curcuma is first class, Baked ginger is second class, and Dried ginger belongs to the third class. In Chrysanthemum Daisy is first class, Wormwood is second class, and Taraxacum belongs to third class. From Fig. A, Fig. B, and Fig. C, it can be judged that the convolutional neural network based on fractional order gradient descent can deeply classify and identify the classes of Chinese herbs, different kinds of herbs under different herb classes as well as different grades of the same kind of herbs, and the results are accurate.

Compare the effect of fractional order gradient descent CNN algorithm and integer order gradient descent algorithm in the task of Chinese herbal medicines classification and recognition. Fig. D compares ResNet18, GoogLeNet, MobileNet with CFO-CNN. Because of the fractional order compared to the integer order’s complex computational form, under the same Chinese herbal medicines dataset, the convolutional neural network with fractional order gradient descent method is about 7.5% more accurate than the traditional ResNet18 model, about 4% more accurate than the traditional GoogLeNet model, and about 23% more accurate compared to the traditional MobileNet model, which is a significant improvement in the accuracy.

The varying convergence speeds exhibited by fractional orders greater than one and less than one. To further compare different orders of gradient descent algorithms, Fig. E selected and compared the accuracy of model recognition for traditional Chinese medicine at different orders. The resulting accuracy change curve graph reveals insights. Analyzing the range of fractional orders, it’s evident that for fractional orders of 0.1, 0.4, and 0.7, when the fractional order is less than one, the performance of fractional order gradient descent algorithms is inferior to both integer order gradient descent algorithms and fractional order gradient descent algorithms greater than one. When the order is greater than one, as seen from the curves at fractional orders of 1.3 and 1.7, fractional order gradient descent algorithms show significant improvement compared to first-order integer order algorithms. Analyzing the magnitude of fractional orders, it’s evident from the curves at orders 0.1, 1.9, and 1.3 that the further the order is from the corresponding integer order, the worse the gradient descent performance becomes. Conversely, when the order is greater and closer to the corresponding integer order, the advantages of fractional order gradient descent algorithms become prominent. To illustrate, a point-line graph was created based on the maximum accuracy values for different orders after 150 iterations. In the case of integer order gradient descent algorithms, the obtained accuracy was 87%. When the fractional order is 1.3, the accuracy of the fractional order gradient descent algorithm reaches 95.5%, reflecting an enhancement of 8.5% compared to integer orders. Conversely, when the fractional order is 0.1, the fractional order contributes almost negligibly to parameter updates.

Fig. F compares the gradient descent method with fractional orders of 0.7, 1.0 and 1.3. It is obvious from the figure that when the order is 1.3, the function has converged to the extremum point after about 120 iterations, and the loss error is about 0.12. When the order is 1.0, the function has completed convergence in 60 iterations, but the error is larger, about 0.88, which is not satisfactory. The convergence is not satisfactory. When the order is 0.7, the model stops converging after 90 iterations with an error of about 1.04, and the model performs poorly in recognizing Chinese herbal medicines during training. It can be seen that when the order of the fractional order is larger than the integer order and the closer to the integer order, the better the convergence of the gradient descent method of fractional order is, on the contrary, when the order is far away from the integer order or smaller than the integer order, the model is easy to fall into the local optimal solution. From this, it can be determined that the value of the order of the fractional order gradient descent method should be elected as 1.3 in the deep classification and recognition task of Chinese herbal medicines.

In Fig. G, the CFO-CNN learning recognition accuracy by fractional order gradient descent method is compared with various network models’ deep learning techniques. For the comparative experiment, the deep learning models Resnet18, GoogLeNet, and MobileNet are chosen. The box plot indicates that when compared to the conventional deep learning Chinese herbal medicine image recognition method, the accuracy of the convolutional neural network with the Caputo fractional order gradient descent method is improved by at least 4%. Additionally, the results of multiple experiments are stable and reliable. The reason for this is that the traditional artificial neural network for the image recognition of Chinese herbal medicines is based on the computational rules of integer order gradient descent to update the weight information, while this paper adopts the fractional order gradient descent method to introduce Caputo fractional order to update the weights, which is superior to the non-local nature of the fractional order that allows it to better capture the global information, and its update method is more accurate compared to that of the integer order updating method.

Fig. H compares the loss values of the models derived from the loss function based on the learning process of Chinese herbal medicine photos in order to further compare the gradient descent techniques at various orders. We find that the ultimate loss value of the model decreases minimally as the fractional order moves farther away from the integer order, suggesting that gradient updates are not clearly affected by this change. The computed loss values from the loss function computation are reduced when the fractional order approaches an integer order and exceeds the first-order integer. This suggests that picture learning recognition gets more accurate, which is better for correctly identifying and categorizing Chinese herbal medicines.

The confusion matrix used in this study to identify nine Chinese herbal types is displayed in Fig. 2. The information reveals that the estimated probability of identifying Panax ginseng is approximately 100%, that of identifying Codonopsis is 100%, that of identifying Radix glehniae is 95.6%, that of identifying Dried ginger is 99.8%, that of Baking ginger is 87.9%, that of Curcuma is 87.3%, that of Wormwood is 100%, that of Daisy with 84.4% predicted probability, and that of Taraxacum with 88% predicted probability. It can be seen that the convolutional neural network based on the fractional order gradient descent method has high accuracy in predicting multi-class Chinese herbal medicines varieties, and the recognition accuracy of similar different kinds of Chinese herbal medicines can reach more than 80% even under the same class.

Fig. 2.

A represents the class of herbs recognized by the model; B is the corresponding variety of the recognized herbs; C is the classification of the recognized herb classes; D is the experimental comparison plot of integer order gradient descent and fractional order gradient descent convolutional neural networks with different models; E represents the recognition ability of fractional order gradient descent with different orders; F is a plot of the error analysis for different orders of the fractional order; G is the boxplot of the recognition effect of different models; H is the magnitude of the loss value of Chinese herbs at different orders; I is the confusion matrix of a certain nine varieties of herbs.

4 Conclusion

In this paper, a fractional order convolutional neural network based on gradient descent is investigated for deep recognition and classification of Chinese herbal medicine images. Among them, the model adopts Caputo fractional order gradient descent instead of integer order gradient descent method to update the CNN model parameters, solve the problem that the simple computation rule of integer order leads to easily fall into the local optimal solution, optimizing the network backpropagation algorithm. Improved model convergence performance and improved the accuracy of the deep classification and recognition of Chinese herbal medicines images by about 8.5%.

This CFO-CNN model can realize the variety recognition of different Chinese herbal medicines, further identify the species of Chinese herbal medicines under different varieties of Chinese herbal medicines, and distinguish different grades under the same type of Chinese herbal medicines according to the advantages and disadvantages, which solves the fluctuation in the classification of Chinese herbal medicines due to manual subjectivity and the inefficiency of manual recognition.

The main limitation of this paper is that its data sources are only randomly grabbed from several Chinese herbal medicine image databases, and thus may not cover all types of Chinese herbs. Therefore, datasets covering more types of Chinese herbs are needed to improve the breadth and persuasiveness, and further research and experiments are needed to fully evaluate the performance and feasibility of the present method for the application of Chinese herbal medicines recognition.