Keywords

1 Introduction

Peru is one of the countries with the highest export of Persea Americana (avocado) in the world, as well as in its production [1]. The most common avocado varieties in Peru are Hass and “Fuerte” [2]. However, 95% of avocados exported by Peru are of the Hass variety.

The growing demand for both the production and export of this fruit is reflected in the consolidation of trade with the USA and the European Union and with the opening of new markets such as Chinese since 2015 [1].

However, the yield of avocado tree crops is not as expected. Peru produced an average of 11.8 t  \({\text{ha}}^{ - 1}\) in 2017, which places it out of the top ten countries in avocado yield [1]. One of the causes of low agricultural yield was the poor control and care of the vegetation. For this reason, hard work is necessary to achieve the reduction of pests and deficiencies in avocado crops.

The vegetation type detection or classification in multispectral images using artificial neural networks has been studied in recent years. For instance, the images have been segmented to restrict the area where a leaf or tree is located and, from this result, the color characteristics are extracted in order to train a classifier [3,4,5,6,7]. In that sense, the addition of thermal images into training has improved the accuracy of those classifiers [7].

Thermal images in crops have been studied [8, 9], demonstrating that, in general, any leaf increases the temperature in the diseased areas of its beam surface [8] and, in addition, the average of the temperature over a tree canopy varies depending on the hour of acquisition [7].

For this work, the acquisition and processing of thermal and RGB images is performed to detect anomalies in Hass avocado leaves, using a proposed method that considers the temperature distribution in the beam of the leaves, as a descriptor of the anomalous characteristics that may occur on a leaf, and color characteristics, through the Normalized Green-Red Difference Index (NGRDI).

Section 2 details the procedure for acquiring images using an infrared camera. In Sect. 3, the method used in extracting information from images, using the k-means algorithm, and the classification stage, with a Support Vector Machine (SVM), is explained.

Section 4 has the results and discussions of experiments related to the average surface temperature variation of the Hass avocado leaf beam at different times of the day and the resultant performance of the proposed method for the classification of healthy leaves and leaves with anomalies.

2 Image Acquisition

2.1 Instruments and Environmental Considerations

Image acquisition was carried out in the avocado cultivation area of the National Institute of Agrarian Innovation (INIA), located in Lima (Peru), during this time, 499 thermal images and 499 RGB images of Hass avocado leaves were acquired, of which 100 of each type were selected, according to the visible and thermal characteristics useful for anomalies detection. These images were acquired with the FLUKE TiS 45 9 Hz infrared camera [10], of which 20 correspond to healthy leaves and 80 to diseased leaves or with visible anomalies. Thermal images have a size of 320 × 240 pixels and RGB images have a size of 640 × 480 pixels.

Regarding the configuration of the infrared camera, this was adjusted to an emissivity of 0.95, as an average of the accepted values for the surface of a leaf [9]. In addition, the specific background temperature is considered for each of the acquisitions. This temperature was measured with a FLUKE 80bk-a thermocouple.

2.2 Image Acquisition Procedure

The proposed procedure for image acquisition is the result of various field tests and consists of the following steps:

  1. 1.

    Locate a leaf, which must not be directly exposed to the Solar light but has adequate lighting.

  2. 2.

    Gently wipe the surface of the leave with a cloth until dust and dirt are removed, then wait a few minutes until the leaf surface again acquires thermal equilibrium with the environment.

  3. 3.

    Insert the background temperature in the infrared camera. This will be the temperature measurement that thermocouple marks at leaf position. For this, the measurement is made by placing the thermocouple as close to the leaf beam surface as possible.

  4. 4.

    Position the camera so that the leaf extension is located vertically inside the camera’s viewfinder. Then, focus, capture and finally, store.

  5. 5.

    Repeat the acquisition procedure for all the chosen leaves.

In conditions of high air temperatures, it will be necessary not to keep the camera on for more than an hour to avoid electronic system overheating. In these cases, it is recommended, after one hour of activity, turn it off for 15 min and then continue with the acquisition.

The thermographic camera configuration only allows manual focus adjustment. Then, a leaf will be correctly focused as soon as the RGB and thermal images fit as well as possible. Likewise, it is recommended that the camera position does not obstruct direct radiation that is toward the leaf surface.

The hands and, in general, the human body emits thermal radiation that can alter the surface measurement of the leaf in the infrared camera. Therefore, it is recommended to be placed at one meter away from the leaf at the acquisition time.

3 Proposed Image Processing and Acquisition Method

The thermal images acquired with the thermographic camera are exported as a text file, which contains a matrix of representative temperature values for each acquisition.

Within the mentioned matrix, the leaf is segmented, selecting four points (on a displayed image) that determine a rectangle of minimum dimensions but that encloses the leaf. Then, the k-means technique [11] is used in order to divide the leaf into k = 15 clusters. From each of these clusters, the statistical descriptors, mean and standard deviation, are extracted to obtain a total of 30 descriptors of the thermal image of the avocado leaf.

On the other hand, RGB images acquired by the infrared camera are also exported, but in jpg format. As in thermal imaging, these are segmented in each of their bands, following the same procedure.

Once a leaf is cut, it is resized to the size of the corresponding thermal image, then, Normalized Difference Green-Red Index (NGRDI) [12], is calculated in each pixel and It is defined by the following expression:

$${\text{NGRDI}} = \frac{{{\text{ND}}_{\text{G}} - {\text{ND}}_{\text{R}} }}{{{\text{ND}}_{\text{G}} + {\text{ND}}_{\text{R}} }}$$
(1)

where \({\text{ND}}_{\text{G}}\) is the digital value of each pixel in the green band and \({\text{ND}}_{\text{R}}\) is the digital value of each pixel in the red band. While the resulting image of a single color band whose pixel values correspond to the respective NGRDI value will be called the NGRDI image.

Similar to the thermal image processing, the K-means algorithm is used to divide the NGRDI image into 15 clusters and for each one, the mean and standard deviation is calculated. These values add a total of 30 descriptors.

Finally, the number of pixels of each cluster is calculated, both in the thermal image and in the NGRDI image, adding 15 descriptors. In total, there is a set of 90 descriptors for each leaf, they are inserted into an SVM network [13], which uses a linear Kernel transformation function. Likewise, the images were distributed in training, validation and test groups (70%, 15%, and 15% respectively).

In Fig. 1, the diagram of the stages of the proposed method is shown.

Fig. 1
figure 1

Diagram with the processing steps of the proposed method

Full size image

4 Results and Discussions

The experiments carried out and shown below are proposed to analyze the characteristics of descriptors and the effects of environmental conditions on these and, consequently, on the detection of the anomalous and non-anomalous condition of a tree leaf.

These experiments were carried out in the experimental station of the National Institute of Agrarian Innovation (INIA), in Lima. Thus, in the proposed first experiment to verify the effects due to direct solar radiation, images were acquired of two healthy Hass avocado leaves, one under the shade and another exposed to direct solar radiation, both belonging to different trees. The results are shown in Fig. 2.

Fig. 2
figure 2

a Variations in the average surface temperature of b a healthy leaf exposed to direct solar radiation and c a healthy leaf under the shade and in different Hass avocado trees

Full size image

During this experiment, the sky remained cloudy during the first five acquisitions, in which, there was a minimum temperature difference over time between both leaves. However, from the following acquisitions, the solar radiation modified the temperature of the exposed leaf significantly. In Fig. 2, the hourly temperature is shown where there was an incidence of direct solar radiation on the leaf, as well as the temperature of a leaf under the shade, called an unexposed leaf. To quantify the relationship between these temperatures, the Spearman correlation coefficient [14] was used, whose value is 0.4857; this represents a moderate correlation. In addition, temperature differences between both measurements are of 4–10 °C.

The results are shown in Fig. 2, require caution to avoid direct solar radiation in the beam of Hass avocado leaves during acquisition procedure in the following experiments. The second experiment consisted of acquiring images from healthy leaves and leaves with anomalies. Thus, the acquisition of 22 thermal images and 22 RGB images of the Hass avocado leaf was carried out, in the range from 11 h to 15 h on May 31, 2019, at INIA. Each acquisition was simultaneously carried out for a healthy leaf and an anomalous leaf, both were under shade. The resulting temperature curves, shown in Fig. 3a, represent the variation of the average surface temperature of the healthy and the anomalous leaf beam. The temperature of the curve that represents the healthy leaf is below the temperature of the curve that represents the leaf with an anomaly, for each acquisition. Both leaves, of the same tree and very close to each other, always remained in the shade. In addition, the curves show a similar trend over time. Spearman’s correlation for this case has a value of 0.9018, with significance value p = 0.0001; this result indicates a high correlation.

Fig. 3
figure 3

a Average surface temperature variation of c a healthy leaf and d an anomalous leaf of the same avocado tree and, b the average surface temperature variation of e a healthy leaf of different avocado tree and the same anomalous leaf in 11 different hours of the day

Full size image

On the other hand, Fig. 3b shows the comparison of the same leaf with an anomaly, this time with a healthy leaf belonging to another tree also under shade. In this case, Spearman’s correlation of 0.8064, with significance value p = 0.0027, is still high. In spite of a reduction in the correlation, a significant difference was maintained between the healthy leaf temperatures and the leaf anomalous temperatures, which allows those to be differentiated.

Consequently, a third experiment was carried out to analyze the variation of the temperature distribution in three different acquisition positions on the Hass avocado leaf beam. In this experiment, three thermal images were acquired by rotating on the azimuth: front and two lateral sides, all of them correspond to a healthy leaf under a shade, but with sufficient illumination. The acquired thermal images were preprocessed, cropping the region bounded by the leaf, then, they were divided into k = 15 significant clusters, following the method mentioned in Sect. 3. Next, an analysis of the regions of the leaf that belong to clusters with the highest number of pixels, as well as a surface distribution of clearly identifiable temperatures.

Figure 4 shows the temperature distribution of three clusters on the two healthy leaves for the three views (one front and two sides), where one is under a shade and the other exposed to direct solar radiation.

Fig. 4
figure 4

Images of two healthy leaves in three views. From left to right, the RGB image and three views (front, side 1 and side 2, respectively) are shown. In (s) the top row, (s, a) a leaf under a shade and their respective clusters for (s, b) the front view, (s, c) side 1 and (s, d) side 2. In (r) the bottom row, (s, a) a leaf exposed to solar radiation and their respective cluster for (r, b) the front view, (r, c) side 1 and (r, d) side 2. Clusters 1, 2 and 3 can be observed in green, red and blue respectively

Full size image

Table 1 shows that there is a variation of pixels between the different views, which varies from 26 to 28% for cluster 1 and 33–35% for cluster 2; Despite this, the average temperature does not vary. While the number of pixels for cluster 3, for all three views, varies from 35 to 62%, in addition to the temperature variation up to 0.3 °C. This means that different views have an effect on temperature for clusters with the least number of pixels; consequently, this variation may have an effect on the performance of the classification algorithm.

Table 1 Effects of the infrared camera rotation on three clusters that belong to a healthy leaf under the shade and obtained with the K-means algorithm. In the upper region of the avocado leaf cluster 1 and in the lower region cluster 2. In addition, cluster 3 with a variation of the average temperature
Full size table

In the case of Table 2, in cluster 1 it is observed that the average temperature varies from 0.1 to 1.4 °C, in cluster 2 from 0.5 to 1.7 °C and in cluster 3 from 0.5 to 1.4 °C. In this case, when acquisition in any different view is carried out, the variations in average temperature are significant, so that it is necessary to perform the acquisition under shade.

Table 2 Effects of the infrared camera rotation on three clusters that belong to a healthy leaf exposed to direct solar radiation and obtained with the K-means algorithm
Full size table

In both Tables 1 and 2, uniform temperature distributions are assumed on the leaf and, consequently, standard deviations of uniform distribution. For the analysis of the three experiments described above, only the region bounded by the leaf is considered, where temperatures have a distribution similar to a uniform distribution. However, for the method proposed in Chap. 3, the rectangular re-cutting of the leaf justifies the use of a standard deviation for a normal distribution, due to the variability of the environment surrounding the leaf.

It should be noted that the standard deviation of cluster 3 increase in a leaf under a shade, for a smaller number of pixels. While its behavior is erratic in a leaf exposed to solar radiation.

The SVM classifier to healthy leaves and anomalous leaves was trained and tested with 100 thermal and NGRDI images, which are images that correspond to leaves not exposed to solar radiation, each of which was processed following the method proposed in Sect. 3. This classifier uses a linear Kernel transformation function, as well as a distribution of seventy-five training images, fifteen for validation and fifteen for the test. The result of an average of 10 experiments for the same set of 100 images shows an accuracy of 82.67%.

5 Conclusions

In this work, we propose a new method to process, cluster and classify Hass avocado leaves condition using thermal and NGRDI images and a procedure to acquire thermal and RGB images of leaves in the field, using a handheld infrared camera.

The four experimental results about the effects of solar radiation on avocado leaves show alteration in the temperature measurements and, consequently, to clustering of healthy or dead zones of a leaf.

In future works, the accuracy of the classification stage can be improved with a greater variety and quantity of images obtained by the image acquisition process and processed by the method proposed in this work.