Keywords

1 Introducing the Normalized Difference Vegetation Index (NDVI)

Crops sit at the base of food chains, absorbing sunlight to fuel the photosynthetic reactions. The sunlight that plants absorb is referred to as Photosynthetically Active Radiation (PAR) and is measured as irradiance, in units of light quanta, integrated over the wavelength interval of 400 to 700 nm [1]. At wavelengths greater than about 750 nm incident light is preferentially scattered by leaves. This scattering is the reason that plants appear as bright objects in infrared photography. The transition between absorption and scattering causes the sharp jump in reflectance between the visible and near-infrared region of the reflectance spectrum known as the red edge (Fig. 1).

Fig. 1
figure 1

Reflectance of potato canopy and soil. Canopy spectra present low reflectance in the visible due to absorption by pigments, and increased reflectance in the NIR due to scattering. The NDVI quantifies this difference. The soil spectrum is relatively constant across the shown interval, hence results in a much smaller NDVI relative to the potato. The Red and NIR band centres correspond to the METER NDVI band locations

Full size image

The passive remote sensing of crops has a long and storied history which is entwined with the development of Earth Observing spectral radiometers in the latter half of the twentieth century (see [2] for a recent perspective). Early instruments measured radiance in a few multispectral bands spanning the visible and near-infrared region of the spectrum. It was evident that algebraically combining band data from these radiometers, to form so called vegetation indices and band ratios, increased the signal to noise content of the data greatly enhancing the observation of vegetation from space [3]. Studies in the 1970s and 1980s demonstrated how such data could be used to estimate vegetation parameters from space, including the leaf area index (LAI), defined as the ratio of the one-sided surface area of leaves to surface area of ground [4]. Global maps of vegetation phenology soon followed [5].

Note that in this early work there was a distinction between simple band ratios which are the division of bands, and vegetation indices (VIs) which are band functions that feature differencing as the main operation [6]. Myneni et al.’s [7] theoretical work showed a functional relationship between VIs and the first derivative of the reflectance spectrum. The advantage of using the first derivative of a spectrum rather than the measured zero order values of a given wavelength band is that confounding variation due to constant offsets between observations (e.g., in time) of single bands, which could occur due to instrumental or target related factors, cancel out in differences [7, 8].

The best-known vegetation index is the normalized difference vegetation index (NDVI), which quantifies the strong contrast between photosynthetic light absorption in the visible region of the spectrum and scattering in the near-infrared (NIR) region [9, 10]. In simple terms, NDVI puts a single number on the red edge jump in the canopy spectrum, and is typically calculated using NIR and Red reflectance values as:

$$ NDVI=\frac{NIR- Red}{NIR+ Red} $$
(1)

The difference to which NDVI gives its name is in the numerator of Eq. 1, the denominator normalises difference values between −1 and 1. The normalisation makes NDVI potentially easier to interpret compared to the unbounded range possible with, for example, the simple ratio NIR/Red. The normalisation has also been suggested to reduce the effect of sensor degradation [3]. Crippen [11] suggested an interesting reason for the relative success of NDVI, he proposed that the NDVI was “self-perpetuating” as the established standard, having gathered sufficient user inertia.

The main limitation of the NDVI, which was already apparent in early applications [10], is referred to as the saturation effect, and denotes a non-linear asymptotic flattening, or loss of sensitivity, of the curve between NDVI and LAI (or biomass). The loss of sensitivity typically starts at LAI values ranging from 2–4 depending on the crop [12]. Further shortcomings of the NDVI were gradually revealed in the 1980s using physically based methods.

Seller’s [6, 13] adapted the two stream (dual direction) radiative transfer formalism used in atmospheric science to model maize NDVI as a function of vegetation structural and optical parameters such as the LAI. His work explored the non-linear relationship between NDVI and LAI, finding that non-linearity was amplified by bare ground in the sensor field of view. However, and unlike for LAI, Seller’s [6] went on to demonstrate a linear relationship between the fraction of absorbed PAR (fAPAR) and NDVI. This result makes intuitive sense as the fraction of light absorbed by a canopy will also saturate at a given leaf area. Taken together his results can be interpreted as suggesting that NDVI is useful as a measure of near instantaneous productivity, which depends on fAPAR, but of limited use for LAI, or total biomass, in most green crops or forests due to the saturation effect.

It is tempting to assume that if NDVI is linearly related to fAPAR, then NDVI should also be related to foliar chlorophyll, which is the main light absorbing molecule, or nitrogen content which is used to build leaf proteins.Footnote 1 However, as with LAI, the relationship between (red band) NDVI and chlorophyll content saturates at low chlorophyll values [14]. As Seller’s [13] work showed, it is more likely that NDVI is influenced by canopy structural factors such as the leaf angle distribution, especially at low LAI values. The dissociation between pigments and NDVI, was also confirmed in latter studies [15] who found no relationships of merit between pigment content and NDVI.

The limitations described above, which are further confounded by atmospheric effects, fractional cover sensitivity and variance in soil colour and brightness, motivated the development of new and more complicated, in terms of mathematical formulae, VIs, designed to address these shortcomings [3, 12, 16]. These include the perpendicular and orthogonal vegetation indices where reflectance values in NIR-Red space are projected onto the so-called soil line, with greater distances (projections) from the line representing increased vegetation fractions [3, 17].

An additional advance that was used as the foundation of NASA’s MODIS productivity algorithm, was the Enhanced Vegetation Index (EVI) that corrects for soil effects and is less sensitive to saturation than the NDVI [18]. More recent techniques for tracking productivity from space include Sun-induced Fluorescence (SIF) [19], and the NIRv index which is an adjusted form of NDVI whose derivation is rooted in Seller’s [6] theory and which aims to minimise the effects of background variation in the signal [20].

A further development related to vegetation indices is the use of multi-angular observations to characterise the anisotropy (directional dependency) of observations. For a sensor above a sunlit scene, the observable radiance is a function of the inherent optical properties of the objects within the scene, their structure and also the view and solar geometry, and the ratio of diffuse to direct radiation. The Bidirectional Reflectance Distribution Function (BRDF) is a theoretical concept that formalizes the directional dependency as a function of view and direct beam incident light angles. Most measurements of NDVI are therefore subject to directional artefacts, and care must be taken to compare data observed with differing geometrical configurations. However, as well as being a possible hindrance, reflectance anisotropy can also be utilised to retrieve structural information, on for example canopy clumping, from multi-angle reflectance data [21].

Although no longer at the cutting edge of satellite remote sensing, the use of NDVI persists and may even be growing [22, 23]. New remote and proximal (close to canopy) NDVI platforms and sensors differ from the relatively coarse resolution satellites of old, and include high resolution uncrewed aerial vehicles (UAVs) [24, 25], field based robots [26], close contact spectral sensors mounted on mobile phenotyping platforms [27], active NDVI field sensing [22], and fleets of Earth Observing CubeSats [28]. Applications are also migrating from the traditional global photosynthesis prediction and change detection [2] to the rapidly evolving field scale commercial crop analytics, nutrient, and yield prediction [22] and phenotyping [27] disciplines which include a significant commercial element. A parallel and integrated development, is the uptake of data driven analytical modelling methods, referred to as machine learning, to relate NDVI or other optical data to crop parameters [29]. Such methods are useful as they can handle the vast amounts of data generated by high resolution imaging spectroscopy sensors to selectively arrive at accurate predictive models for e.g., chlorophyll content or LAI retrieval [30].

The emergence of the NDVI in new applications is probably due to the relative simplicity of the formula and the ease of measurement, requiring only an NIR and visible sensitive instrument. Obviously from the discussion above, the interpretation of NDVI is far from straightforward. Hence there is a gap that requires bridging between decades of physically based knowledge derived from satellite remote sensing and the state of the art in field and plot scale data, which tends to a data driven focus. This chapter digs into the issue of relating the historical satellite-derived theory [6] to new crop applications and sensors. This is achieved by analysing proximal data acquired at high temporal and spectral resolution in two agricultural sites in Finland, to attempt to answer the question: what does NDVI and spectral reflectance data really tell us about crops in the field? For the sake of simplicity, the chapter is focused on point based spectral reflectance sensing. However, the lessons learned here are also applicable to well calibrated imaging spectroscopy data.

2 Methods

2.1 Sites, Sensors and Supporting Observations

Data were acquired from spectral reflectance sensors using proximal field and UAV platforms at two agricultural experimental sites in Finland shown in Fig. 2. The sensors were designed to capture temporal and spectral variation, and the main sensor and site characteristics are listed in Table 1. The temporal dimension was investigated using data collected at the Qvidja research site, in South Western Finland in 2019 and the spectral dimension was investigated using data collected at the Viikki experimental field site in Helsinki in 2018, using a UAV platform.

Fig. 2
figure 2

NDVI instrumentation and site locations. Left frame shows proximal METER NDVI sensor above grasses at Qvidja site. The central frame shows site locations in southern Finland overlaid on a PROBA-V satellite NDVI retrieval from July 2018. The right frame shows the UAV carrying the Piccolo-Doppio payload flying over the potato crop at Viikki, Helsinski

Full size image
Table 1 Site and NDVI sensor characteristics. Note that there were many additional instruments operating at both sites, as both sets of observations were collected during heavily instrumented field campaigns
Full size table

The Qvidja estate site is an experimental grass site located in southwestern Finland. At this site, mainly timothy and meadow fescue grasses (Phleum pratense L. and Festuca pratensis Huds. respectively) grown in small 4 by 4 m plots underwent differing fertilization treatments classified into four groupings: no fertilization, 50% of optimal mineral fertilization, optimal mineral fertilization, and organic fertilization (For brevity, these categories are referred to as: No fertilization, 50%, Optimised and Organic fertilization.). The plots were harvested at multiple times of year. METER (Pullman, WA, USA), formerly Decagon, NDVI Spectral Reflectance Sensors were deployed at the site during summer 2019. One sensor was placed over each treatment using a wood support structure with a view zenith angle of 45 ° which resulted in a ground instantaneous field of view (IFOV) major axis length of 2.9 m. These devices use the GSM network to transfer data from the field to a cloud service, which is subsequently visualised in a web browser and downloaded as text files for further processing. According to the METER specifications, the red and NIR bands are centred at 650 and 810 nm, with 10 nm FWHM. Data were collected at a 5-minute interval, and daily averaged between 10:00 and 17:00. Supporting LAI measurements and SPAD chlorophyll readings were also taken and are briefly mentioned in the results.

The second experimental site was located within the University of Helsinki’s Viikki campus in Helsinki, Finland. Here the potato variety Lady Felicia (Solanum tuberosum L.) was grown as row crops from seed in 2018. Two differing treatments were imposed: a paired drought treatment and a multi-level nutrient treatment. In the drought treatment, there were 10 plots of 6 m by 6 m, including a 1 m buffer, in total with a paired (treatment and control) sampling design where one of each pair was irrigated and the corresponding pair was under drought treatment. In the nutrient plots, which were the same dimensions, there were four levels of nutrient addition. The four nutrient addition levels varied between two different levels of nitrogen fertilizer (YaraBela Suomensalpietari, Yara International, Norway) and two different levels of general macronutrient fertilizer (Yara Mila Hevi3, Yara International, Norway) and were replicated four times. Due to the exceptionally good weather in Summer 2018, no rain exclusion was necessary to achieve the required drought effect.

For chlorophyll concentration ([Chl.]) sampling, leaves were picked during the measurement period and frozen until analysis. The frozen leaves were mixed with dimethyl sulfoxide (DMSO), which has been shown to be an effective solvent for chlorophyll and pigment analysis [31]. The samples were then homogenized and extracted in an oven for 4 h at 50 °C before analysis with a Shimadzu UV-1800 spectrometer (Shimadzu Corporation, Japan). [Chl.] was estimated as total chlorophyll a and b on a leaf area basis.

UAV flights were conducted by Finnish Geospatial Research Institute’s (FGI) drone laboratory using a custom-built UAV based on a Gryphon Dynamics quadcopter frame, a Pixhawk autopilot and Applanix APX-15-EI UAV positioning system. The main payload of the UAV was a Piccolo Doppio (PD) Dual Field of View (DFOV) spectrometer system which uses the Cos-conical approach. This system is principally designed to retrieve SIF but also measures visible and near infrared radiance and irradiance, which is the purpose to which it was deployed here. The PD is based around two Ocean Optics spectrometers (Ocean Optics/Insight, Dunedin, FL, USA), a Flame and QE Pro and a bifurcated fibre optic assembly manufactured by Alker Ltd. (Alker Fibre Optic Specialists Ltd., Surrey, UK). The PD DFOV system collects incident irradiance through a cosine corrected diffuser fore-optic attached to one fibre optic leg and upwelling radiance through a bare tipped optical fibre. In the present study both the irradiance and radiance fibre optic cables were mounted to a stabilising gimbal (Photohigher, Wellington, New Zealand) and upwelling radiance was collected in the nadir view. Here data is presented from the Flame spectrometer, which has a usable spectral range of 400 to 950 nm, spectral sampling interval of 0.4 nm and a sampling band width (FWHM) of 1.3 nm. More details concerning the PD can be found in MacArthur et al. [32] and Atherton et al. [33].

The PD UAV flights were conducted on 25th July 2018 by hovering approximately 9 m above ground level for each of the plots. A 9 m height above canopy results in a top of canopy field of view diameter of 4 m, given the bare fiber IFOV angle of 25°. Repeats of 25 spectral radiance and irradiance samples were collected above each plot and subsequently averaged to estimate per plot reflectance. Per flight median optimised integration times were between 7–8 milliseconds for downwelling and were between 12–14 milliseconds for upwelling.

Hemispherical-conical reflectance factors (HCRF) were estimated as the ratio of upwelling radiance spectra, multiplied by π, to downwelling irradiance spectra [34]. Note that when measuring reflectance in the field, the observation depends on the atmospheric conditions, and in particular the diffuse to direct ratio of irradiance which influences the sampling of the BRDF [35, 36]. As such, measuring irradiance and radiance a small distance above the canopy target affects the resulting spectra due to atmospheric scattering and absorption in the path to the target. These points should be kept in mind when comparing data across sites and instruments.

The UAV was also used to collect RGB imagery which was processed to point clouds using AgiSoft Professional software (AgiSoft LLC, St. Petersburg, Russia). The imagery was collected using two Sony A7R II digital cameras with Sony FE 35 mm f/2.8 ZA Carl Zeiss Sonnar T* lens. They were mounted at +15° and − 15° oblique angles in a stabilized rack. The flights were carried out at a flight height of 50 m that resulted in a ground sample distance of 0.64 cm. A double grid setup with six north-south flight lines and nine east-west flight lines were used to obtain a minimum of nine overlapping images over the entire area of interest. Protocols describing the photogrammetric processing chain developed by FGI can be found in Viljanen et al. [37].

2.2 Data Processing and Analysis

Data collected at site 1 (Qvidja) required little extra processing except for daily statistics and quality control which were conducted in R 3.5.3 programming language [38]. A field cross calibration was conducted between radiance and irradiance sensors using a near lambertian Spectralon panel on June 5th and these coefficients used to calculate reflectance. At site 2 (Viikki), NDVI was calculated by first interpolating reflectance spectra to a 1 nm wavelength scale and then applying Eq. 1 to PD data using the reflectance values at wavelengths 650 nm and 810 nm.

Leaf angle distributions were estimated from UAV retrieved photogrammetric point cloud normal vectors. The Average Leaf Inclination Angle (ALA) per plot, referenced to the upwards (zenith) pointing vector, was computed as a summary statistic from these angular distributions. A description of this new approach can be found in Xu et al. [39]. In this chapter, the Singular Value Decomposition method was not used, rather the pre-computed vectors from Agisoft software as a higher correlation was found between leaf angles and spectral data using the latter approach. Data below a single reference height (4.4 m above coordinate system zero level) were excluded due to shadow and soil contamination. Similar results were obtained when no height threshold was applied, however the resulting correlations with spectral data were not as strong as when using the height filtered data. Linear correlations between spectral data and other variables (e.g., ALA) were calculated using Pearson’s correlation coefficient.

3 Results and Discussion

3.1 Temporal Variability at Site 1

Figure 3 shows the time course of NDVI measurements during summer 2019 at the Qvidja site. There were two harvest periods which are marked on the figure at the start of June and mid-way through August. Post-harvest NDVI values were around 0.6–0.7 rising to maxima of around 0.8 for the organic and optimized fertilized sites a few weeks after the harvest period. The fertilized sites show elevated NDVI for most of the time course. At end of the season, variance in each of the time-series is increased which could be due to the lower light levels at the end of the year.

Fig. 3
figure 3

Time-series of proximally sensed NDVI of grasses undergoing nutrient treatments at Qvidja estate site, Southwestern Finland. Data are daily averaged

Full size image

In these grasses, NDVI clearly tracks growth post-harvest and differentiates between fertilized and untreated plots. As corroborating evidence, LAI increases were measured over the experimental period with highest values in the fertilized plots; data from a SPAD chlorophyll meter showed no increasing trends over the sampling period (data not shown). It is therefore probable that changes in NDVI related to an associated increase in the fraction and total amount of absorbed PAR by photosynthetic elements which was determined by the fractional coverage and also related, probably asymptotically, to LAI [6].

Variation in the NDVI is driven by canopy scattering which occurs principally in the NIR, and soil reflectance which occurs across all wavelengths [6, 13]. To explore these issues further, the focus shifts to the second site where variability across space using a UAV-based hyperspectral instrument was observed.

3.2 Spectral-Spatial Variability at Site 2

When retrieved proximally from a UAV platform, there was a negative relationship between Average Leaf inclination Angle (ALA) and NDVI in the potato crops at site 2 (Pearson’s r =  − 0.77, Fig. 4). Further, those plots under greatest water deficit presented the largest ALA and smallest NDVI values. This results chimes with previous research where the link between canopy structure and NDVI under water limited conditions has been highlighted, and where NDVI is referred to as a structural vegetation index [40].

Fig. 4
figure 4

Average Leaf inclination Angle (ALA) per plot Vs NDVI for a combination of water and nutrient treatment potato plots. Drought treatment plots with wilted leaves tend to the bottom right of the plot, with relatively high ALA and low NDVI values. NDVI error bars are one sample standard deviation of batch means

Full size image

Water limitation causes a lack of leaf turgidity, commonly referred to as wilting, resulting in greater leaf inclination angles and hence larger plot-wise ALA. This shift towards an erectophile distribution influences the NDVI due to the increased canopy gap fraction both within and between crop rows. Increasing the gap fraction exposes a greater proportion of soil, increasing the contribution of soil reflectance to the total signal. An increase in gap fraction also increases the distance between scattering and absorbing leaf elements and changes the canopy shadowing properties. It is not possible to separate between all these factors using the NDVI alone, here hyperspectral data is instructional.

ALA is correlated with canopy scale reflectance (Fig. 5) across large parts of the visible and NIR spectrum. However, both the strength and sign of the correlation (Fig. 5 lower panel) is dependent on wavelength. Visible reflectance is positively correlated with ALA, whereas in the NIR the sign switches to a negative correlation of increased magnitude. This can be explained by the interacting effects of light scattering by soil and vegetation. In sections of the visible region, soil is of higher reflectivity than vegetation (Fig. 1). Increasing the ALA means that the sensor FOV contains a higher proportion of soil background, resulting in the positive correlation. In contrast, in the NIR region scattering in the sensor FOV is dominated by canopy leaves [6]. An increase in ALA reduces the probability that a photon is scattered in the direction of the nadir pointing sensor, resulting in a negative correlation at those wavelengths greater than approximately 700 nm. These results combine to decrease reflectance under water stressed conditions in the NIR, due to lessening leaf scattering, but increase reflectance in much of the visible region due to increased soil scattering. This result is in contradiction to Moran et al. [41] who found water related scattering decreases in the visible and NIR regions.

Fig. 5
figure 5

Spectral correlation with structural (ALA) and leaf level ([Chl.]) variables. Top panel shows the square of Pearson’s correlation coefficient between spectral reflectance (HCRF) and variables, and bottom panel shows Pearson’s correlation between spectral reflectance (HCRF) and variables. Vertical lines mark the location of NDVI bands

Full size image

Returning to the NDVI, then it appears that variation in the index is caused by the interplay of soil and canopy scattering. Although these mechanisms have opposite signs of correlation with the structural variable ALA, they both decrease the NDVI relative to increasing ALA (loss of turgidity) due to the mathematical formulation of the index. This is because NDVI can be reduced due to either an increase in red reflectance, caused by soil scattering, or by a decrease in NIR reflectance, caused by a reduction in leaf scattering. However, just because NDVI responds to ALA does not mean NDVI is an optimal remote estimator of ALA. Note that NIR reflectance has slightly higher correlation with ALA relative to NDVI (Fig. 5 lower panel). Further, Zou and Mõttus [42] found that although NDVI was related to ALA in crops, other indices demonstrated stronger correlations.

The two mechanisms discussed above, which control the NDVI in the potato crop, do not directlyFootnote 2 relate to photosynthetic light absorption at the leaf scale. In this case variation in leaf optical properties, and by extension chlorophyll, is largely immaterial to NDVI which is under the control of variance in canopy scale topology. The breakdown of the relationship between pigment content and NDVI under water limited conditions has previously been noted and alternative indices applied for chlorophyll and nitrogen estimation such as the MERIS terrestrial chlorophyll index that focus specifically on the red edge region [15]. Note that, in addition to the relevance to field data discussed in the current chapter, retrieval of chlorophyll from space is currently a major goal in satellite remote sensing [43].

Clearly broadband NDVI cannot be used to estimate leaf properties such as chlorophyll content in the water limited case. However, the results shown in Fig. 5 suggest that a sensor with differing band location could potentially be used to infer pigment content. Figure 5 shows a clear correlation between canopy reflectance and foliar chlorophyll concentration in the red edge close to 700 nm, and in the green region where correlation with ALA is minimal. In contrast, the NDVI bands fall squarely in the soil-structure dominated spectral regions (Fig. 5 top panel). As an interesting aside, the empirical results shown in the top panel of Fig. 5 closely resemble the model predictions found in Fig. 1 of Zou and Mõttus [44].

To summarise the above, NDVI does respond to water deficit. However, this response is due to structural changes in the canopy, not variance in pigments or leaf optical properties. These results suggest that, whilst care must be taken in interpretation, NDVI can indeed be used as an indicator of water stress capturing water related variation in soil and canopy scattering properties in the red and NIR bands respectively.

4 Conclusions and Outlook

At both sites, the NDVI conveyed information relating to canopy structure. At the first site, the NDVI followed post-harvest grass development, differentiating between nutrient treatments (Fig. 3). At the second site, NDVI responded to water limitation in potatoes via variation in canopy scattering but was unrelated to pigment content (Figs. 4 and 5). Therefore, reflectance in the red-edge or green region is required to relate canopy observations to leaf pigments, especially under conditions of structural variance caused by water limitation.

At the two study sites, area integrated point data were used to investigate variance in canopy reflectance in the temporal and spatial dimensions. Such data are useful as a robust empirical benchmark and are potential candidates for the validation of CubeSat-based NDVI and LAI retrievals. However, the future of field spectroscopy lies in high spatial resolution imagery [24]. Once properly calibrated, such systems can probe variance in ecosystem processes at the individual plant scale across the field. So how best to capitalize on our current knowledge going forward in this direction?

The application of mature radiative transfer codes to estimate biophysical parameters (e.g., pigment content) from multi-angular UAV data is an avenue under current investigation [45]. However, it is worth emphasising that the spatial scales that data is now generated at are unprecedented and call for the development of new physically based tools and models. Verrelst [30] discusses options for hybrid imaging spectroscopy approaches combining physically based remote sensing with data driven methods which is an interesting avenue of future research. The method of radiosity [46, 47], which models the canopy as a collection of planar canopy elements throughout 3D space, is a worthwhile avenue of research to revisit in this direction. Following from this, perhaps the most interesting direction of research is the extension of imaging spectroscopy into the 3rd spatial dimension and the opportunities that this affords [48].