Keywords

7.1 Introduction

Crops are threatened by various stresses (e.g., pests, diseases, drought, nutrient deficiencies) in their life-cycle. Wheat, the most widely grown cereal crop in the world, is able to provide 20% of protein and food calories for 4.5 billion people worldwide, but is also facing severe challenges from abiotic stresses, pathogens and pests due to climate changes and inappropriate management strategies, posing serious threats on food security. Among various crop diseases, wheat yellow (or stripe) rust is a devastating global wheat disease, particularly in the regions with temperate climates [1]. This disease is able to develop and spread very quickly under favourable conditions such as a suitable temperature of 5–24 \(^{\circ }\)C, a moderate precipitation in spring (for winter wheat) and suitable wind conditions (for spore spread) [2].

Generally speaking, there are two available approaches to control yellow rust disease including chemical control and genetic resistance (please refer to [3] and the references therein). Conventional chemical control methods for yellow rust disease rely on calendar-based (timing) and uniform (blanket) application of pesticides regardless of the current disease status (e.g. stress level and its spatial distribution), which not only results in a higher cost (more pesticide) but also inevitably increases the likelihood of ground water contamination and pesticide residues in agricultural products, generating significantly economical, environmental and social burdens. It is highly desirable to develop an automated and non-destructive wheat disease mapping system so that a decision-based precision (site-specific) management strategy can be implemented [4] at the right time/location while with the right amount of pesticides.

Wheat yellow rust disease generally results in certain physical and chemical changes on wheat leaves such as ChlorophyII content reduction, water loss, and even visible rust symptoms at later stages (i.e. yellow-orange to reddish-brown spores) [5]. It is noted that when visible rust symptoms appear on wheat canopy, the chemical treatment is to stop its spread to surrounding areas. These changes can be observed by human eyes and can also be captured by optical sensors such as RGB [3, 6], multispectral [4] and hyperspectral cameras [1, 7]. Human inspection is time-consuming, labour-intensive, prone to error, and so not suitable for real-life applications at large scales. Different cameras also have pros and cons in terms of price, weight, flexibility and performance. In this study, multispectral camera is adopted since compared against RGB camera, it possesses spectral bands beyond the three visible bands (e.g. NIR band), and in comparison to hyperspectral camera [8], it is much easier to operate (e.g. imaging, calibration and orthomosaic) and is usually with a much lower price.

In addition, different sensing platforms are also available for crop stress monitoring, which range from ground proximity sensing [3] to aircraft (or even satellite) remote sensing [4]. It is shown in the existing studies that UAV based remote sensing is able to provide a user-defined spatial-temporal image resolution (e.g., imaging period, flight height), a low cost and a high flexibility, and therefore is drawing ever-increasing popularity for field-scale applications since 2010 in different application domains such as disease monitoring [1], weed mapping [9], stress detection [10], crop parameter extraction [11] among many others. Therefore, this work aims to adopt UAV based multispectral remote sensing for the automated wheat yellow rust disease mapping for the purpose of target treatment.

There are also studies on using UAV based multispectral remote sensing for wheat yellow rust disease mapping [2, 4, 12]. For example, yellow rust disease mapping is formulated as a supervised classification problem, and both conventional machine learning (i.e. random forest) and state-of-the-art deep learning algorithm (e.g. U-net) [13] are adopted to address the semantic segmentation problem in [2, 4], respectively. It is also shown that deep Convolutional Neural Network (CNN) outperforms the random forest approach due to its capability in learning both spectral and spatial information concurrently in an end-to-end manner. Although this method is able to obtain a good result in small local areas, its generalization performance (location-wise, season-wise, variety-wise) is unclear as the model is trained by using the dataset from one variety and a small area in a supervised manner.

This work aims to initially develop a simple unsupervised yellow rust disease mapping framework by integrating UAV based remote sensing, multispectral imaging, Spectral Vegetation Indices (SVIs) generation, spectral analysis and Otsu’s thresholding. The proposed framework is initially validated by field experiments, where multispectral images and ground data are recently collected on yellow rust infected wheat field using a small UAV at about 20 m above ground with a spatial resolution of 1.3 cm in 2019. To the best of the authors’ knowledge, this work is the first attempt to develop a unsupervised learning framework to address the problem of wheat yellow rust disease mapping. To be more precise, the main contributions are summarized as below.

  1. (1)

    A unsupervised learning framework is proposed for wheat yellow rust disease mapping based on SVI analysis and thresholding, and is initially validated by field experiments;

  2. (2)

    The best SVIs for wheat segmentation from background pixels, and yellow rust disease segmentation from healthy wheat pixels are obtained based on SVI generation and mutual information analysis;

  3. (3)

    The challenges of wheat yellow rust disease mapping in real-life applications are also discussed in terms of early detection, image resolution, disease identification, model generalization, time window for disease management.

7.2 UAV Remote Sensing

7.2.1 Wheat Yellow Rust Experiment

The field experiments are carried out at Caoxinzhuang experimental station of Northwest Agriculture and Forestry (A&F) University, Yangling, China in 2019 (please refer to Fig. 7.1 for the geographic location). Background information about the region is referred to the previous studies [4]. In this study, wheat variety Xiaoyan 22 is chosen since it is easily susceptible to yellow rust disease. The yellow rust inoculation is done at wheat seedlings stage in March/2019, where more detailed information such as inoculation method is available in [2]. In term of experiment layout, as displayed in Fig. 7.2, letters A, B, and C represent the three replicates to reduce random errors; numbers 0–5 denote the different levels of yellow rust inoculum with 0 being empty (no inoculum) and 5 being the highest level of inoculum; each plot with a size of \(2\,{\text {m}} \times 2\,{\text {m}}\) in each replicate is randomly inoculated with one of the six levels of yellow rust inoculum: 0 g (health wheat plots for blank comparison), 0.15 g, 0.30 g, 0.45 g, 0.6 g and 0.75 g corresponding to the level of 0–5, respectively. Disease cross-infection was maximally avoided by separating all 18 wheat plots from each other via healthy wheat gaps.

Fig. 7.1
figure 1

Geographic location of the experimental wheat field

Full size image
Fig. 7.2
figure 2

Wheat yellow rust inoculum experiment: three replicates (column-wise) with various levels of yellow rust inoculum in the range of 0–5, where the false-color RGB image from [2] is at diseased stage on 02/May/2019

Full size image

7.2.2 UAV Remote Sensing

7.2.2.1 UAV Imaging Platform

Different commercial UAVs (e.g. fixed-wing UAV [14] for large-scale operation, multi-rotor UAV for small areas with flexible/low flight heights) and multispectral cameras (e.g. Parrot Sequoia, RedEdge and Altum from MicaSense, P4 Multispectral from DJI) can be integrated to be the UAV imaging platform. In this study, a commercial off-the-shelf aircraft named DJI Matrice 100 (M100) Quad-copter (DJI Company, Shenzhen, China) and a 5-band multispectral camera (RedEdge, MicaSense Inc., Seattle, USA) are integrated to be the low-altitude UAV remote sensing platform (please refer to [2] for the graphic illustration) for yellow rust disease mapping.

7.2.2.2 Data Collection

In this study, data collection is done on 02/May/2019, when yellow rust is in diseased stage with visible symptoms (e.g., yellow color by spores). The UAV flight height is set to be about 20 m above ground, where the ground spatial resolution of the image is about 1.3 cm/pixel after image stitching. Commercial free-of-charge software, Pix4DCapture, is used to plan, monitor and control the UAV for data collection. In particular, the mode of Polygon for 2D maps in Pix4DCapture is chosen with an image overlap and sidelap up to 75%, the UAV forward speed is set at 1 m/s and the camera triggering period is also 1 sec. In addition, before the flight, reflectance calibration panel is also imaged at 1 meter height so that an accurate reflectance data can be obtained for the follow-up image calibration even under environmental (including lighting) variations. The UAV flight track along with camera triggering points, the overlapping images under the defined flight track and the calibrated NIR orthomosaic are shown in Fig. 7.3, respectively.

Fig. 7.3
figure 3

UAV flight track and camera triggering points (left); the number of overlapping images at each points (middle); and the raw calibrated NIR orthomosaic (right)

Full size image

RedEdge camera is equipped with GPS module and is able to capture five raw narrow bands simultaneously including Blue, Green, Red, RedEdge and NIR bands (please refer to Fig. 7.4 for one sample image of the RedEdge camera). All necessary information for the follow-up image stitching is embedded in each image such as GPS information and camera information (e.g. camera model, exposure time, focal length).

Fig. 7.4
figure 4

Sample RedEdge image including five raw spectral bands

Full size image

7.2.3 Image Pre-processing and Data Labelling

After all images are collected, a number of image preprocessing steps are then conducted offline to generate the calibrated (using CRP panel) and georeferenced reflectance data for each spectral band and different SVIs. This can be achieved in commercial (e.g. Pix4DMapper, Agisoft Metashape, DroneDeploy) or open-source software (e.g. WebODM). In this study, Pix4DMapper is adopted, where the detailed steps are referred to [2, 4] such as initial processing, orthomosaic generation and reflectance calibration for each band. The outputs include five GeoTIFF images and the defined SVIs of the covered area, where the NIR orthomosaic is displayed in Fig. 7.3 as an example. All orthomosaic images of the five spectral bands and SVIs can be then input into MATLAB environment, where image rotation (by using the function “imrotate”) and region of interest (ROI) can be performed and defined for the follow-up analysis. The ROI for five spectral bands, the false-color RGB image and data labelling process are shown in Fig. 7.5.

Fig. 7.5
figure 5

ROI for different spectral bands, false-color RGB image and the final Label image from [2]

Full size image

To achieve yellow rust disease mapping (e.g., spectral analysis and SVI selection in this study) for wheat field, image labelling is required. In this study, three classes are defined for the field of interest, which include plants with visible yellow rust lesions pixels (Rust class), healthy wheat pixels (Healthy class) and background pixels (Backg class, i.e. non-vegetation soil background). The details for image labelling is referred to [2], which mainly relies on low-altitude high-resolution RGB image and manual visual labelling. The main steps for this labelling method are summarized for the purpose of completeness : (1) false-color RGB image (see Fig. 7.5) is first generated by using the calibrated Red, Green and Blue bands in Matlab with function “imadjust”  for brightness adjustment, on this basis, the yellow rust regions can be manually and visually defined; (2) the classical spectral index Optimized Soil Adjust Vegetation Index (OSAVI) [15] generated from the five spectral bands is first used to segment wheat pixels from the background pixels; surely other vegetation segmentation methods can also be adopted including machine learning based methods [11]; (3) the yellow rust infected wheat pixels, healthy wheat pixels and the background pixels can be then defined (see the final Label image in Fig. 7.5).

7.3 Yellow Rust Disease Mapping

This section considers the problem of yellow rust disease mapping, where the overall framework is displayed in Fig. 7.6 such as feature definition, spectral and mutual information (MI) analysis, wheat segmentation and rust segmentation. Different elements in the proposed framework are introduced in the following subsections, respectively.

Fig. 7.6
figure 6

Overall framework for the proposed wheat yellow rust disease mapping system including feature definition, spectral and MI analysis, wheat and rust segmentation

Full size image

7.3.1 Spectral Analysis

Spectral analysis of the three selected classes is first performed to see the differences for three classes. It has been shown in many previous studies that some SVIs calculated from the five raw spectral bands may be able to provide an even better ability in mapping yellow rust disease [4, 12] against the healthy wheat and background pixels. Therefore, in addition to the five raw spectral bands, 18 widely used SVIs are also calculated to see whether they are able to provide better yellow rust disease mapping performance (please refer to [4] for the definitions of the 18 widely used SVIs). The mean reflectance values of the original five bands and 18 SVIs for the three classes are calculated respectively in MATLAB2020b and displayed in Figs. 7.7 and 7.8.

Fig. 7.7
figure 7

Spectral reflectance of the five bands for the three classes including rust infected wheat (Rust), healthy wheat (Healthy) and background (Backg) pixels

Full size image
Fig. 7.8
figure 8

Spectral reflectance of the 18 SVIs for three classes including Rust, Healthy and Backg classes

Full size image

It can be visually seen from Figs. 7.7 and 7.8 that different classes have different reflectance values at the five bands and SVIs. For example, Healthy wheat pixels have the highest NIR values but the lowest red values, which is mainly due to the high ChlorophyII content in healthy plants over diseased wheat or background. It is, however, noted that the mean value is NOT sufficient for quantitatively evaluating the capabilities of different bands/SVIs in wheat yellow rust disease mapping as the uncertain information (e.g. variance) is missing. And so feature ranking method will be introduced in the next subsection.

7.3.2 Mutual Information Ranking

In order to test the different discriminating capabilities of the five spectral bands and 18 SVIs in yellow rust disease mapping, MI based feature ranking is adopted (please refer to [16] and the references therein for different feature selection methods). The class label is defined into three categories, where Category 1 contains three classes including Rust, Healthy and Backg, Category 2 contains two classes including Rust and Healthy with Backg being removed, Category 3 contain two classes including Wheat (the Rust and Healthy classes are integrated into Wheat) and Backg. The MI values between each band/SVI and the class label (Category 1 with three classes) are calculated and displayed in Fig. 7.9, while the MI values between each band/SVI and the class label (Category 2 and Category 3) are displayed in Fig. 7.10

Fig. 7.9
figure 9

MI values between bands/SVIs and Category 1 class label

Full size image
Fig. 7.10
figure 10

MI values between bands/SVIs and Category 2/3 class labels

Full size image

It follows from Figs. 7.9 and 7.10 that

  1. (1)

    Different spectral bands/SVIs have different MI values with different class labels and therefore with different discrimination capabilities;

  2. (2)

    OSAVI has the best discrimination capability for the three classes (Fig. 7.9), this is mainly due to its strongest capability in discriminating wheat pixels from background pixels (red line of Fig. 7.10).

  3. (3)

    Red band is the best one among the five spectral bands for both wheat pixel segmentation from the background pixels, and yellow rust disease pixel segmentation from the healthy wheat pixels.

  4. (4)

    SVIs may have a better discrimination capability (i.e. with a higher MI value) than the raw spectral bands, although the capabilities of different SVIs differ a lot. And NDREI has the strongest discrimination capability for separating Rust class from Healthy class.

7.3.3 Unsupervised Classification

Based on the ranking results via mutual information in Sect. 7.3.2, this section develops a simple unsupervised classification approach based on SVI thresholding. Considering the fact that the best discrimination SVIs for Rust vs Healthy, and Wheat vs Backg are different, wheat segmentation and rust segmentation are done one by one based on their best SVIs.

7.3.3.1 Wheat Segmentation

The first step is wheat segmentation. It follows from Fig. 7.10 that OSAVI has the strongest discrimination capability in separating Wheat class from Backg class, where its histogram for Wheat and Backg class is shown in Fig. 7.11. Then the well-known Otsu’s method can be adopted to automatically choose the optimal thresholding for wheat segmentation where the wheat pixels have higher OSAVI values.

Fig. 7.11
figure 11

Histogram of OSAVI for Wheat and Backg class

Full size image

7.3.3.2 Yellow Rust Segmentation

After the Wheat class is separated from the Backg class, the yellow rust separation from wheat class is then considered. It follows from Fig. 7.10 that NIR and NDREI have the weakest and strongest discrimination capability in separating Rust class from Healthy class, where their histograms for Rust and Healthy class are shown in Fig. 7.12. Therefore, the NDREI index is chosen for Rust segmentation, where the well-known Otsu’s method is adopted again to automatically choose the optimal thresholding, where the healthy wheat pixels have higher NDREI values.

Fig. 7.12
figure 12

Histogram of NIR and NDREI for Rust and Healthy class

Full size image

Following these two supervised segmentation steps, the final classification result is shown in Fig. 7.13. The quantitative classification results are not provided in this initial study.

Fig. 7.13
figure 13

NDREI (left), NDREI with Backg (middle) and the classification results based on the simple unsupervised approach (right)

Full size image

7.4 Discussion

In this section, the pros and cons of the proposed method are discussed against the state-of-the-art methods. On this basis, the challenges of UAV remote sensing for yellow rust disease mapping are discussed in the context of real-life field applications.

7.4.1 Pros and Cons

7.4.1.1 Supervised Versus Unsupervised Learning

In this study, data labelling is to assess the reflectance differences of various classes and evaluate the different discriminating capabilities of various spectral bands and SVIs. Then unsupervised method is proposed in Sect. 7.3.3 for wheat yellow rust disease mapping based on automated thresholding of different SVIs. It is noted that most existing methods for wheat yellow rust disease mapping are based on supervised learning by using conventional machine learning ([4]) or state-of-the-art deep learning ([2]).

  1. (i)

    Unsupervised approach is simple to implement, and does not require labelled dataset for model training. This is attractive for crop disease mapping, since obtaining a large amount of high-quality (and representative) labelled dataset is costly, labour-intensive and time-consuming and so not practically feasible for yellow rust disease mapping. However, the disease mapping performance of the unsupervised approach (particularly the one in this study) is relatively limited (mainly due to the limited information used in the approach). In this regards, more advanced supervised approaches should be further developed to embed more useful information.

  2. (ii)

    Supervised approach, particularly the state-of-the-art deep Convolutional Neural Network, is able to provide better performance than unsupervised ones. However, this approach generally requires a large volume of high-quality labelled dataset for its model training. Obtaining raw UAV remote sensing images is not hard, however, obtaining a representative and high-quality labelled dataset under different crop varieties, crop growth and disease development stages, and under various environmental condition is a challenge. In this regards, developing semi-automatic labelling approach to generate labelled dataset may be a feasible and promising solution.

7.4.1.2 Spectral Information Versus Spatial Information

For (yellow rust) disease mapping in wheat field, different types of information can be adopted which may include spectral information (different spectral bands), spatial information (disease usually appears from small regions and spreads to the surrounding areas), or even temporal information (combining different measurements together).

  1. (i)

    Disease mapping based on purely spectral information may result in a high proportion of false positives (pepper and salt noises as shown in Fig. 7.13) due to the absent of spatial information in the algorithm.

  2. (ii)

    Different methods for embedding spatial information are available in the literature including at feature stage or post-processing stage. Fusing spectral and spatial information by using deep CNNs (e.g. FCN [17], SegNet, U-Net [2, 18] would be the trend. However, deep learning approach generally requires more labelled data (a representative pixel-wise labelled dataset is even harder than image level labelled dataset) than conventional machine learning approaches, which is a challenge as highlighted in Sect. 7.4.1.1 for supervised methods.

7.4.2 Challenges for Real-Life Applications

The challenges of wheat yellow rust disease mapping system in real-life applications are summarized as below.

  1. (1)

    Early disease detection of wheat (yellow rust) is paramount so that early intervention can be conducted to stop its spread to the surrounding areas. However, early disease detection is generally not easy. UAV remote sensing, as a top viewing method, can only see the crop canopy information. Diseases start from the bottom leaf layers can not be detected until the changes appear on the top leaf layers, which may be too late for its treatment although still useful for stopping its spread. In this regards, hyperspectral imaging which does not rely on visual symptoms may be a proposing solution, however, this method is still costly, not easy to operate, and poses challenges for follow up image processing.

  2. (2)

    Image resolution is important for (early) wheat disease mapping. However, image resolution highly relies on UAV flight altitude and therefore there is a trade-off between image resolution and working efficiency. As flying higher is more efficient but with a lower image resolution, while flying lower obtains a higher image resolution but is generally inefficient. The required image resolution for yellow rust disease mapping (at different disease development stages) should be investigated.

  3. (2)

    Disease identification is also challenging. This is because different crop stresses (e.g., diseases/pests) may result in very similar (top viewing) symptoms on the crop (wheat) such as the ChlorophyII content reduction, water loss, and visual symptoms at later stages. Therefore, disease/pest habitat (environmental) information (e.g. temperature, humidity, season) may be useful to reduce the scope of crop stresses for the purpose of disease identification.

  4. (3)

    Model generalization is important but at the same time challenging for wheat (yellow rust) disease mapping. This is because the developed algorithms (particularly supervised learning algorithms) are actually facing very complex operating environments. How to guarantee the algorithm performance under different wheat varieties, wheat growth stages, disease development stages, lighting/environmental conditions is generally challenging. As a result, it is yet to be tested whether an algorithm trained from one wheat variety can work in other varieties (variety-wise), an algorithm trained from history dataset can work in future seasons (season-wise), or an algorithm trained in one location can work in other locations (location-wise). This validation method should be considered in future publications in addition to conventional random splitting of the lumped dataset.

  5. (4)

    Time window for wheat (yellow rust) disease control is generally short, as a result, the issue that covering a large area of interest within a short time period should also be considered in real-life applications. Multiple UAVs and parallel processing may provide the solution in terms of data collection and data processing.

7.5 Conclusions

This work considers the problem of wheat yellow rust disease mapping by using UAV based multispectral remote sensing. A simple unsupervised yellow rust disease mapping framework is developed based on spectral analysis, mutual information analysis and Otsu’s thresholding. Field study is conducted to generate a dataset to initially validate the developed framework. It is shown that (i) OSAVI is the best SVI for segmenting wheat from background pixels; (ii) NDREI is the best SVI for segmenting yellow rust infected wheat from healthy wheat pixels. The pros and cons of the simple unsupervised algorithm is discussed by comparing against supervised algorithms and the ones with both spectral and spatial information. Moreover, the challenges of yellow rust disease mapping in real-life applications are also discussed.