Introduction

Soybean rust is one of the most destructive diseases for soybean production in Asia since the 1960s [1] and often causes significant yield losses. It has been reported that this disease causes 10–40% yield loss in Thailand, 10–90% loss in India, 10–50% loss in Southern China, 23–90% loss in Taiwan, and 40% loss in Japan [2]. Soybean rust has now occurred in at least 39 countries in both Eastern and Western hemispheres [2–4]. In the United States, soybean rust was first detected at Louisiana State University AgCenter Research Farm in 2004 [5]. Soybean rust has now been found in most soybean-producing states in the continental United States [6]. Because of the rapid spread of soybean rust pathogen (Phakopsora pachyrhizi), it is vital to find an effective and quantitative early inspection means to detect the infection and determine the severity in field for supporting timely fungicide application to minimize the losses caused by rust.

As a foliar disease, the typical symptom on soybean caused by soybean rust are lesions that range from 2 to 5 mm2 and can be initially just chlorotic and can develop into tan to dark brown polygonal lesions depending on the specific reaction with the host. Within each of the lesions is one to many erumpent, globose uredinia. Soybean rust is usually found on the bottom leaves to start with and gradually spreads to the top leaves and finally infects all the leaves on the plant (Fig. 1). Such features of soybean rust make it possible to detect the disease using multispectral imaging technology.

Fig. 1
figure 1

Illustration of a soybean plant infected by rust pathogen [7]

Full size image

Traditionally, crop disease diagnosis is performed by experienced plant pathologists or plant disease diagnosticians, that with a trained eye can diagnose most diseases in the field, and make disease control recommendations specifically to apply fungicides if needed. In recent years, various sensing technologies have been developed for automatically detecting crop diseases. Satellite-based multispectral sensing is one of such technologies that have been used; however, mainly due to the limitation in spatial resolution the space-based sensing technology was not satisfactory on detecting early disease symptoms, even when the reflectance wavelengths of the infected crop canopy were within the collectable bandwidth of the satellite-based sensor [8]. On the other hand, high resolution of multispectral images could provide the capability of monitoring the severity of large scale crop diseases [9]. Heald et al. [10] reported using Color Infrared (CIR) photography to estimate infestation areas caused by cotton root rot (Phymatotrichum omnivorum). Lorenzen and Jensen [11] tried to detect powdery mildew on barley leaves using the reflectance spectrum. They found that the spectral reflectance of inoculated resistant leaves was significantly higher between 400 and 1100 nm than control plants as soon as 10 days after inoculation. This method also was used to distinguish diseased cucumber leaves from healthy leaves by inspecting leaf reflectance of cucumber [12].

Under laboratory conditions, Peterson and Aylor [13] found that high intensity emission spots corresponded to the regions where bean rust infection occurred on bean leaves. They also observed that significant chlorophyll fluorescence changes preceding the first visual symptoms by 3–5 days. However, those fluorescence signals could easily be swamped by background ambient illumination, which restricted its applications in the field. In order to overcome this problem, Ludeker et al. [14] have tried to use a laser beam as excitation light to induce leaf fluorescence. Johansson et al. [15] applied a short pulsed laser synchronized with a gated intensified sensor to develop a multispectral fluorescence imaging system (MFIS). The results showed that the MFIS was able to exclude background illumination and could be used in outdoor conditions. Pinter et al. [16] found that some soil-borne fungus infected crops, such as cotton plants infected with P. omnivorum and sugar beets roots infected with Pythium apanidermatum, would display a noticeable sunlit leaf temperature that could be 3–5 °C warmer than adjacent healthy plants. Chaerle et al. [17] also reported that thermal lesions were normally 0.3–0.4 °C warmer than that of surrounding tissues and such hot spots were visible before characteristic disease symptoms appeared. To improve the sensitivity and the accuracy of the sensing devices, a common approach has been to use more than one sensor and to extract the otherwise hidden information from outputs of those sensors for detecting crop diseases. Bravo et al. [18] fused hyperspectral reflection and fluorescence imaging to improve the capability of detecting yellow rust (Puccinia striiformis) on winter wheat. The authors reported that the data fusion approach allowed them to reduce disease diagnosing error by 1–2%.

The goal of this research was to investigate the possibility of quantitatively detecting soybean rust infection with different levels of severity using multispectral sensors mountable on field equipment. This goal was accomplished through (a) searching for sensitive reflectance spectral bands of soybean leaves; (b) investigating the possibility of detecting soybean rust infection on leaves using a multispectral imaging sensor; and (c) formulating appropriate methods for quantifying its severity. Based on the defined goal, this research was limited to proof of concept on baseline data collected from a controlled environment. The application study would be conducted at a follow-up study after the concept has been proven.

Methodology and procedures

Sensing devices

A portable FieldSpecTM spectroradiometer (Analytical Spectral Device Inc., Boulder, CO), with a sensitivity range from 325 to 1075 nm at a 1.6 nm sampling interval and a 25° field of view, was used to measure the reflectance spectrum of soybean leaves. A white panel with approximately 100% reflectance across the entire spectrum was used as a reference for each test. It was made of Spectralon which was a proprietary material made of cintered polytetrafloraethylene.

To collect multispectral images of rust infected leaves, a multispectral CDD camera (MS3100-RGB, Duncan Tech, Auburn, CA) was used. This camera consisted of three CCD channels of blue (B), green (G) and red (R) with an image resolution of 1392 H × 1040 V at 8-bit pixel−1. The B, G and R channels were centered at 475, 540 and 625 nm, with a bandwidth of 50, 50 and 60 nm, respectively. The focal length of the lens was 14 mm. A digital frame grabber (IMAQ PCI-1428, National Instrument, Austin, TX) installed in a desktop PC computer was used to capture the sample images. This system was capable of capturing up to 7.5 images s−1.

Baseline data collection

All the tests were performed under natural sunlight. Multispectral images of soybean leaves at different severity levels were collected as baseline data. Specifically, images of four soybean leaves were collected from four layers of a rust infected plant from top to bottom, representing four different levels of severity (Fig. 2). The lower level leaflet represented the most severe rust infection with extensive dark brown or reddish brown lesions. The top layer leaflet contained the least rust infection with no visible lesions. There were 24 leaf samples from 6 rust infected soybean plants being measured. The images of each sample were taken twice using the multispectral camera in a greenhouse environment on a sunny day at the University of Illinois at Urbana-Champaign. The reflectance spectrums of those plants, as well as of selected individual leaves, were measured using the spectroradiometer. Each reflectance measurement was repeated three times and the average was taken as the final result. There were 48 images and 72 reflectance data acquired in all. Four representative leaflets with different levels of rust infection were chosen for reflectance spectrum analysis and multispectral image analysis.

Fig. 2
figure 2

Illustration of four leaf images collected from a soybean plant

Full size image

Sensitive bands investigation

One fundamental task in developing optical sensors for crop disease detection is to find out the sensitive bands of the sensor to the diseases to be detected. To locate such bands for detecting soybean rust in terms of leaf reflectance to natural sunlight, the spectroradiometer was used to measure the leaf reflectance over a wide band from 350 to 1050 nm that covers from visible to near infrared bands. Three measurements were made for each sample, and the average value was taken as the characteristic curve. The reflectance spectrums of the four representative leaflets with different levels of rust infection were used for analysis.

To investigate the possibility of directly using the reflectance spectrum for detecting rust infection, a collection of vegetation indices, originally developed for assessing crop growth conditions, such as Normalized Difference Vegetation Index (NDVI) [19], Green Normalized Difference Vegetation Index (GNDVI) [20], Ratio Vegetation Index (RVI) [21], Green Ratio Vegetation Index (GRVI) [22], Difference Vegetation Index (DVI) [23], and Normalized Difference Green Index (NDGI) [19], were examined to test their capabilities on distinguishing infection severities. All those trials used the same bands as developed for crop growing assessment, namely the reflectance at G band (550 nm), R band (670 nm) and/or NIR band (800 nm). The index which has the potential to detect rust should present a noticeable difference on its values collected from leaves with different infection severities.

Image processing methods for detecting and quantifying rusting areas

The core of this research was to develop a systematic method capable of sensitive and reliable detection, and to quantify rust areas on soybean leaves to support on-line rust diagnosis based on multispectral images.

The base images of infected plants were collected as a whole using the multispectral CCD camera under the natural sunlight. The infected leaflets of different severities were separated from the original plant images for minimizing the image processing load, as well as for eliminating the unnecessary complicity.

A three-step approach, namely infected area separation, lesion color identification and rust severity quantification, was developed. Each step would output a characteristic parameter for quantifying the corresponding parameter. Those parameters were ratio of infected area (RIA), lesion color index (LCI) and rust severity index (RSI), respectively.

The infected area separation starts at segmenting lesion pixels in a leaf image using a threshold function defined as follows:

$$ GL(x,y,i) = \left\{ \begin{array} {lll} H\left( {x,y,i} \right)\quad {\text{if}}\,\left( {255 - R} \right)^{2} + \left( {255 - G} \right)^{2} + B^{2} \ge GL_{\text{threshold}} \\ I\left( {x,y,i} \right)\quad {\text{if}}\,\left( {255 - R} \right)^{2} + \left( {255 - G} \right)^{2} + B^{2} < GL_{\text{threshold}} \end{array} \right. $$
(1)

where \( GL(x,y,i) \) is the gray level of a pixel at point (x, y) in channel i, \( H(x,y,i) \) and \( I(x,y,i) \) are gray levels of pixels (x, y) located in either the health or the infected regions in channel i. R, G and B are the gray levels of pixel (x, y) in red, green and blue channel, and noted as channels 1, 2 and 3, respectively.

As shown in Eq. 1, if the gray level of a pixel satisfies \( (255 - R)^{2} + (255 - G)^{2} + B^{2} < GL_{\text{threshold}},\) this pixel is classified as an infected pixel. Otherwise, it is classified as a healthy pixel. Because the RGB values of pure green and yellow colors are (0, 255, 0) and (255, 255, 0), a threshold value was determined through calculating \( (255 - R)^{2} + (255 - G)^{2} + B^{2} \) by reducing the gray values in R, G and B channels until it can reliably segment the infected and healthy pixels.

After pixel segmentation, an RIA could be determined in terms of the ratio of total number of infected pixels to the whole leaf pixels in channel i as follows:

$$ IA = \frac{{\sum\limits_{j} {I_{j} } \left( {x,y,i} \right)}}{{\sum\limits_{j} {H_{j} } \left( {x,y,i} \right) + \sum\limits_{j} {I_{j} } \left( {x,y,i} \right)}} \times 100\% $$
(2)

While the common believe is that the degree of disease severity increases with the infected area, the color of the infected spots is another common criterion of infection severity by human eye. To incorporate both the infection area and infected spots area in severity quantification, a LCI was defined to serve for this purpose in this research. From extensive image analysis, it was found that the value of \( \left( {R - G} \right) \) offered an important indication on color analysis. A pixel appears green color if \( \left( {R - G} \right) < 0,\) or brown if \( \left( {R - G} \right) > 0.\) The larger the \( \left| {R - G} \right| \) value, the darker the color will be. Based on this fact, a LCI was defined as follows:

$$ LCI = \frac{R - G}{{\sqrt {R^{2} + G^{2} + B^{2} } }} $$
(3)

The obtained LCI values over the entire band of pixel gray level will result in a histogram of lesion color distribution with \( {\text{LCI}} = 0 \) being a demarcation point, which can be used as an important indicator for rust severity assessment.

For more effectively utilizing the obtained LCI distribution in rust severity assessment, a new index, the RSI, was defined as follows:

$$ RSI = \frac{{\sum\limits_{k} {\begin{array}{*{20}c} {LCI_{k} } & {\forall LCI_{k} > 0} \\ \end{array} } }}{{\sum\limits_{k} {\begin{array}{*{20}c} {LCI_{k} } & {\forall LCI_{k} \le 0} \\ \end{array} } }} $$
(4)

Higher RSI values indicate increased disease severity.

Results and discussion

Reflectance spectrum analysis

Figure 3 shows the typical reflectance spectrums of four leaflets with different levels of rust infection. Each of those spectrums was the average of three measurements on the four representative leaflets. From general observation, the overall reflectance level decreased as rust infection becomes severer. It also showed that there was a peak reflectance appeared around 550 nm range (G band). Such a phenomenon was caused by the chlorophyll content and nitrogen content of leaves [24, 25]. The differences among reflectance levels of leaves with different rust severity became more distinguishable in the NIR bands. Therefore, the NIR band of reflectance was the potential band suitable for rust detection.

Fig. 3
figure 3

Reflectance spectrums of four representative leaflets with different rust severity

Full size image

To evaluate the possibility of using existing vegetation indices, such as NDVI, GNDVI, NDGI, RVI, GRVI and DVI, the values of those indices were calculated to detect soybean rust and determine its degree of severity. Table 1 summarizes the values of difference indices corresponding to different rust severities. However, all those indices, except for DVI, did not represent a definite correlation with infection severity. While DVI did show a positive correlation with rust severity levels, this research did not collect sufficient samples to allow a reliable elimination of other attributes, such as crop nutritional health. Therefore, further studies are needed to confirm the feasibility of using DVI as an effective tool for soybean rust detection.

Table 1 Values of vegetation indices corresponding to different rust infection levels
Full size table

Multispectral image analysis

Since convincible rust infection detection using vegetation index based spectrum analysis approach was not obtained, we investigated the multispectral image analysis approach for accomplishing the same purpose. The critical step for reducing computational load in image processing was to separate the leaflet of interest from whole plant image. At image preparation step, leaflet images were separated manually by removing both background and other portion of soybean plant using Photoshop 7.0 (Adobe Systems Incorporated, New York, NY). Images of soybean leave with severe, medium, light and non-visible rust levels were prepared for methodology development.

The next step was to segment the infected and healthy pixels in leaflet image using Eq. 1 . The key for such segmentation was to determine the threshold value of the gray level for each degree of rust severity. All the threshold values were manually selected based on a guideline of clearly segmenting the infected pixels from healthy ones. After the infected and healthy pixels being segmented, the RIA could be calculated using Eq. 2.

As an example, when analyzing a leaflet image with medium rust severity, the threshold value for segmenting the infected and healthy pixels was selected as 50,000. This threshold value could effectively segment the infected spots from the leaflet as illustrated in Fig. 4. The number of infected pixels and healthy pixels were 17,916 and 159,095, respectively. By adding up the segmented pixels of infected and healthy pixels, the corresponding RIA was about 10.1%. Similar results were obtained from leaflet images with similar degree of infection based on visual inspections.

Fig 4
figure 4

Illustration of segmentation of rust infected and healthy pixels from a leaflet image. a Leaflet image, b Disease area image, c Healthy area image

Full size image

By adjusting the threshold values, specifically, 20,000 for severe infection, 60,000 for light infection and 62,000 for non-visible infection, the average RIAs were 58.8%, 1.7% and 0.1% for severe, light, and non-visible levels of rust infection, respectively.

It was also noted that not only could this developed system segment the rust infection pixels, it could also segment out powdery mildew infected pixels. During the investigation, one leaflet image with light level of rust infection was infected by powdery mildew. By applying a similar approach as segmenting rust infected pixels, a method for segmenting the powdery mildew infected pixels was also developed. Because the color of the powdery mildew disease is white, by applying pure white RGB value of (255, 255, 255) into \( (255 - R)^{2} + (255 - G)^{2} + (255 - B)^{2} < GL_{\text{threshold}},\) the resulting pixels were classified as powdery mildew infected pixel. The average gray value of the leaflet image in three channels was 55, 82, 41, which resulted in a \( (255 - R)^{2} + (255 - G)^{2} + (255 - B)^{2} \) value as 115,725 for powdery mildew infection. By reducing this value until the powdery mildew infected pixels were segmented out as much as possible, a threshold was finally set at 92,000 for powdery mildew infection. As shown in Fig. 5, the multi-class pixel segmentation could selectively segment both the rust infected and powdery mildew infected pixels. The number of rust and powdery mildew infected pixels are 5,674 and 26,958, respectively. Visual inspection verified that this method could provide a satisfactory segmentation for both rust and powdery mildew inspection. The powdery mildew infected area (PMIA) was 8.1%.

Fig. 5
figure 5

Illustration of segmentation of rust and powdery mildew infected pixels from a leaflet image. a Leaflet image, b rust disease area image, c white spot area image

Full size image

Visual observation indicated that rust infected areas on a soybean leaflet increases with the degree of severity. However, it is insufficient to determine the severity of infection solely based on infection areas. A LCI and distribution was created to quantify the attribution of lesion color by using Eq. 3. Calculate LCI values of all pixels segmented in infected leaflets and display them in a histogram (Fig. 6). When setting \( {\text{LCI}} = 0 \) as the demarcation point, all \( {\text{LCI}} < 0 \) indicates the colors of pixels are located in gray green region, whereas all \( {\text{LCI}} > 0 \) denotes tan or dark brown pixels. So this histogram distribution could visibly reveal whether a leaflet is more green than brown. By applying Eq. 4, a RSI can be determined by the area ratio of \( {\text{LCI}} > 0 \) over \( {\text{LCI}} < 0 \) as an additional measure of rust severity. In the example of a leaflet with medium degree of rust infection, the resulting RSI value was 1.13.

Fig. 6
figure 6

Histogram distribution of pixel LCI values obtained from a medium infected leaflet

Full size image

After applying the same procedures to all samples with different severity levels of rust infection, a positive correlation was found between rust severity, measured by RSI, and the total infected area. Table 2 summarizes four representative leaflets with different levels of rust infection. From these results, it was found that LCI histogram provided a good indicator for rust infection as well as the degree of severity. When a leaflet was not infected with rust pathogen, all pixels should have a negative LCI value, resulting in the LCI histogram distribution in the left side of the demarcation point. For a leaflet was severely infected, many pixels would have a positive LCI value, resulting in LCI distribution in the right side of the demarcation point. For LCI histogram was evenly distributed on both sides of demarcation point, a medium level infection could be quantified.

Table 2 Summary of multispectral image processing and outcomes for soybean rust detection
Full size table

These results indicated that with the defined characteristic parameters, it was technically possible to detect soybean rust infection, and assess its degree of severity based on multispectral images of soybean leaflets. It also showed a potential of using the same system, supported by different color indices, to detect other soybean diseases, such as the powdery mildew disease.

Conclusion

A collection of soybean leaflets with different degrees of rust severity were used in both reflectance spectrum and multispectrum analyses. Obtained results verified that reflectance values differed by the rust intensity. The leaflets with the most rust had the lowest values of reflectance. Among the different channels, the reflectance values in NIR (centered at 800 nm) band showed the most obvious differences related to rust severity. While DVI values showed a positive correlation to rust severity, it requires more research to verify the correlation between DVI and rust severity.

After analyzing the multispectral images of infected leaflets, three image processing parameters (RIA, LCI and RSI) were defined and formulated to describe the degree of rust severity. Among those parameters, RIA was used to measure the percentage of leaf area being infected with rust pathogen. LCI was used to measure the color of leaflet pixels. The RSI was developed to quantify the severity of rust infection. Laboratory-scale test results verified that this set of defined parameters, especially the RSI, positively correlate with the severity of rust infection.

This paper reports the preliminary results in developing a multispectral imaging sensor for detecting soybean rust infection. More comprehensive studies are needed to verify sensible range and accuracy of the developed methods in different environments. A machine-mount in-field multispectral image sensing device will be constructed for performing field tests.