A review of the state of the art in agricultural automation. Part IV: Sensor-based nitrogen management technologies

This paper was originally written for presentation at the 2018 ASABE Annual International Meeting, Detroit, Michigan, July 29-August 1, 2018 and can be located as Antille, D. L., Lobsey, C. R., McCarthy, C. L., Thomasson, J. A., Baillie, C. P. (2018). A review of the state of the art in agricultural automation. Part IV: Sensor-based nitrogen management technologies. ASABE Paper No.: 1801593. St. Joseph, MI.: American Society of Agricultural and Biological Engineers. DOI: 1013031/aim.201801593.

Take home message

Crop nitrogen (N) management is one of many important agricultural applications that can benefit from crop sensing. The technologies in this field are advancing rapidly, including: (1) sensor-carrying platforms, (2) the sensors themselves, and (3) the analytical techniques used to derive actionable information from the data.

A review of commercially and semi-commercially available platforms was undertaken to inform sensor mounting, with particular focus on unmanned aerial vehicle (UAV) sensor platforms and unmanned ground vehicle (UGV) sensor platforms. The UAV and UAG platforms provide indirect and direct measurements for crop monitoring and N mapping with the goals of being low-cost, on-site, and versatile.

Optical crop sensing techniques and systems for N management are also discussed, because destructive sampling and laboratory analyses are expensive and often not practical for site-specific management of N. The optical properties of the plant are significant because they are related to water content, leaf senescence, disease, and nutrient status, which can inform farming decisions. Additionally, Red, Green and Blue (RGB) imaging can provide a plant height assessment for multiple measurements, including: yield potential, biomass, density, uniformity, and planter skips.

The work reported in this paper includes a comparison of various optical sensors for plant measurements, including: vis-NIR, machine vision, and 3D-imaging, with camera varieties such as multispectral, fluorescence, hyperspectral, thermal, and visible. Key recommendations have been provided for the development of data aggregation and decision support tools including the data sources to be used in development of machine learning models, software/data standardisation efforts, and corporate collaborations regarding big data. In conjunction with the sensors and their platforms, this advancing field of management technology can provide intelligent sensing and intelligent decisions.

Introduction

This paper, the fourth in a series of four, includes a comparison of various optical sensors for plant measurements, including: vis-NIR, machine vision, and 3D-imaging, with camera varieties such as multispectral, fluorescence, hyperspectral, thermal, and visible, respectively. The other three papers in this series (Baillie et al., 2018a-b; Thomasson et al., 2018) critically review commercially-available sensing technologies for optimization of machine operations and farm inputs, the on-farm communications and connectivity, and the agricultural machinery navigation systems, respectively.

The current reference technique for measurement of plant nitrogen (N) status is plant tissue sampling following by analysis of total N using Kjeldahl-digestion and Dumas-combustion (Oxenham et al., 1983), which are both time consuming and costly laboratory techniques (Muñoz-Huerta et al., 2013). To quantify N deficiency, the N content in plant material is then referenced to critical threshold concentrations. These are defined as the minimum N content required for optimal growth or beyond which no additional yield would be expected (e.g., Rochester, 2011). These values are determined experimentally (using field trials) and are dependent on the crop growth stage or above ground biomass (e.g., Justes et al., 1994). The N status can be also represented as a N nutrition Index (NNI) calculated as a ratio of the actual measured N content and the critical N content for the given dry weight (Mistele and Schmidhalter, 2008). The analysis of plant tissue to determine N status has some advantages over soil testing in that the complexities of other soil, plant and environmental factors that will ultimately influence plant uptake are effectively accounted for by measuring the plant response directly. For macronutrients, good correlation is usually found between plant concentrations and nutrient status as measured by yield (van Maarschalkerweerd and Husted, 2015). Analysis of plant tissue however is regarded as impractical for site-specific management of N. Soil mineral N is highly variable (McBratney and Pringle, 1999) and although not practical, it has been suggested that analysis and differential fertilizer application should occur at a spatial resolution of 1-m to maximize benefits of site specific N management, recognizing that variability in yield potential and fertilizer needs exist at this resolution (Raun, 2005). There has therefore been a considerable effort in the development of rapid, low cost optical sensing techniques for analysis of plant N status.

Optical crop sensing techniques for nitrogen management

Optical sensing provides a cost effective and rapid technique for the measurement of plant biophysical and biochemical status. The optical properties of the plant are related to water content, leaf senescence, disease and nutrient status (Muñoz-Huerta et al. 2013). However, optical sensing techniques are typically indirect, and are unable to measure N content of the plant directly (Raun et al., 2002). They are instead based on measurement of compounds such as chlorophyll, which can be used as an indicator of N status (Živčák et al., 2014a-b). Sensing techniques can be defined by the region in the electromagnetic spectrum in which they operate and their measurement range (e.g., tractor mounted) or remote levels (e.g., airborne or satellite platforms). Sensors can rely on existing ambient lighting (passive) or use active light sources to improve measurements in variable lighting conditions and also allow operation at night. Sensors may be deployed on airborne vehicles (e.g., UAVs) or may be limited to ground vehicles (e.g., UGVs and tractors). When compared with other sensing techniques (e.g., soil sensing), optical plant sensing is relatively easy to perform and a range of sensing techniques and sensors are commercially available (Table 1). Optical sensor measurements are rapid and low cost and can therefore provide the high spatial and temporal resolution required for N management.

Reflectance (multispectral)

Reflectance sensors in the visible-near infrared range utilize the principle of healthy plant tissue absorbing light in the photosynthetic active radiation (PAR) region of the spectrum, and reflecting in the infrared region (Galambošová et al., 2014). Sensors typically measure reflectance in the red wavelength range that corresponds primarily to absorption by chlorophyll A, and NIR wavelengths at approximately 800-nm. Reflectance sensors can be active, which allows operation at night and improves response under variable lighting conditions. They can also operate passively and so the technique is suitable for remote sensing (UAV or satellite). Some commercial sensors (e.g., OptRx, CropSpec) and the RapidEye satellite also utilize the ‘Red Edge’ band (690-730 nm) at the inflection point in the absorption spectrum, which can improve response in high-plant density and later growth stages. Using the reflectance measurements in these different bands, vegetation indices are calculated relating to the plant vigor, biomass or chlorophyll content. These indices are then related to the N status of the plant.  A number of different indices, such as the normalized difference vegetation index (NDVI) have been developed for these purposes (e.g., Rodriguez et al., 2006; Chen et al., 2010). Sensors can be mounted on a tractor for on-the-go measurement and therefore they have seen considerable attention for N management (Table 1). An overview of the commercially available reflectance sensors and their application to N management is provided by Whelan (2015).

Table 1. Optical plant sensing for nitrogen management.

Sensing mechanism

Measured properties (Indices)

Wave length range

Energy

Sensing range (level)

Autonomous deployment

Sensing resolution at field scale1

Measurement of N status2

Commercial products

References

vis-NIR Reflectance (multispectral)

Chlorophyll
(NDVI, SRI, VI …)

Vis-850nm

Active / Passive

Leaf / canopy / remote

UAV / UGV / Satellite

+++

++

GreenSeeker,
CropCircle,
CropSpec,
Yara N-Sensor,
OptRx,
Field Scout

Rodriguez et al. (2006), Raun et al. (2005)

vis-NIR Transmittance

Chlorophyll,
Polyphenols
(NBI)

650nm and 940nm

Active

Leaf

UGV

+

++++

SPAD
Duelex
Cropscan 3000H

Prost and Jeuffroy (2007), Debaeke et al. (2006), Chen et al. (2010)

Fluorescence

Chlorophyll, Polyphenols
(Anth, Chl, Flav, NBI)

UV (excitation)- vis-NIR

Active / Passive

Leaf / canopy

UGV

++

+++

Mulitplex,
MiniVeg

Cendrero-Mateo et al. (2015), Cerovic et al. (2012), Tremblay et al. (2012), Tartachnyk et al. (2006), Chaerle et al. (2007)

vis-NIR Reflectance (hyperspectral)

Nutrient status, water  content
(empirical calibration)

Vis-2500nm

Active / Passive

Leaf / canopy

UGV

++

+++

Fieldspec (ASD), Ocean Optics, ...

Hansen & Schjoerring (2003), Kusnierek & Korsaeth (2015), Chen et al. (2010)

Machine vision (visible)

Plant height,
Stalk diameter,
Structure,
Leaf area

Vis

Passive (typically)

Leaf / canopy / remote

UGV / UAV

++++

++

Licor canopy analyser for LAI

Sadeghi-Tehran et al. (2017), Li et al. (2010), Casadesus & Villegas (2013)

Machine vision (multispectral)

 

Vis-1000nm

Active /

Passive

Leaf / canopy / remote

UAV / UGV

++++

+++

Survey2 (MAPIR)
RedEdge/Parrot Sequoia (Micasense)

Tilling et al. (2007), Vigneau et al. (2011)

Machine vision (fluorescence)

Chlorophyll, Polyphenols,
Biotic stress
(Anth, Chl, Flav, NBI)

UV (excitation)- vis-NIR

Active

Leaf / canopy / (remote)

UGV

+

+++

Walz (IMAGING-PAM M-Series)

Bauriegel and Herppich (2014), Kuckenberg et al. (2009)

Machine vision (thermal)

Water stress,
Biotic stress

LWIR

Passive

Leaf / canopy / remote

UAV / UGV

++++

+

FLIR, Xenics

Tilling et al. (2007), Costa et al. 2014

3D Imaging (LIDAR, TOF, stereo vision, laser profiling…)

Plant height, stalk diameter, structure, leaf area, tiller counting

Vis, IR

Active / Passive

Leaf / canopy

UAV / UGV

++++ (UAV plant height)
++ (other)

+

Photogrammetry software for point cloud generation (Pix4D, Agisoft),
SICK scanners, + others

Duan et al. (2016)

1Sensing resolution is a function of sensor speed, sensor resolution, potential for remote deployment etc. (+ offers low or poor resolution, ++++ indicates high resolution, potential for 1~m2 spatial resolution as suggested by Raun et al. (2005). 2Indicates the usefulness (or directness) for measurement of N status or informing N management decisions (+ indicates indirect measurement, low correlation, ancillary information only while ++++ indicates high correlation to plant N status).

Rodriguez et al. (2006) performed canopy reflectance measurement using a spectrometer (FieldSpec, ASD, 350-2500 nm) to evaluate a range of reflectance indices for the measurement of N stress, calculated from shoot N and shoot dry weight. The canopy chlorophyll content index (CCCi) and modified spectral ratio (mSRPi) indices were able to explain 68% and 69% of the variation in N stress, despite variation in water stress and canopy density. Chen et al. (2010) evaluated a number of vegetation indices calculated from hyperspectral reflectance measurements for the measurement of plant N concentration in wheat and corn. The results of the regression analysis (R2) were generally low for all indices in wheat (0.17-0.56) although measurements were diverse and included multiple growth stages. A new N index was later proposed, the Double-peaked Canopy Nitrogen Index (DCNI), which could explain 72% and 44% of the variation in plant N concentration for corn and wheat, respectively. In a later study (Chen, 2015) correlation of spectral indices with N concentration in wheat statistics improve on a growth stage basis with approximately 60% and 80% of the variation in N content explained by all indices at the Feekes 4 and Feekes 7-8 growth stages, respectively. Mechanistic models relating spectral indices and N nutrition index (NNI) validated well (R2: 0.82-0.94) and were independent of phenology.

Mistele and Schmidhalter (2008) used a spectrometer to measure wheat canopy reflectance and calculate the red edge inflection point (REIP). REIP and the N nutrition index NNI were highly correlated (R2 =0.95). The NIR/NIR indices (R760/R730) was reported to be the most reliable index for measurement of N status (Erdle et al., 2011). The response of the NDVI indices become saturated and was unable to distinguish between high N treatments. While active sensors provide greater versatility in operation, the wide spectral information that can be obtained from passive sensors may be needed for phenotyping certain traits (Erdle et al., 2011).

There are limitations in the use of reflectance sensor for measurement of plant N content. Reflectance sensors are ultimately a measure of biomass/chlorophyll content, and this can be slow to change in response to most environmental changes and is not solely related to stress (Tremblay et al., 2012). The chlorophyll content and biomass of the crop may only be indirectly related to the adequacy of N supply, and therefore empirical calibration of applied N to sensor response during crop growth is necessary for inference. Methods for predicting N requirements and calculating N response index using reflectance sensors are described by Whelan (2015). Calculating a response index is typically performed using reference strips where fertilizer is applied in non-limiting rates and sensor response in this area is used to inform application rates in the remainder of the field. Limitations for this approach are selecting a reference strip area that is representative of the field (e.g., Samborski et al., 2017) and that it is effective only when N is the main growth limiting factor (e.g., Zillmann et al., 2006). Further limitations of this sensor technique for N management include chlorophyll saturation where sensors are unable to distinguish between adequate and excess supply of N (Ruiz Diaz et al., 2008) and they are sensitive only when the fraction of vegetation cover is low (Maier and Günther, 2003). The available commercial sensors also differ in their use of active or passive techniques, wavelength ranges and the signal output can also differ with measurements based on different vegetation indices. Crop sensors need to be calibrated for different cultivars, sites and seasons (Craigie et al., 2013).

Transmittance (multispectral)

Measurement of chlorophyll can also be performed using transmission based (absorption) methods. The basic principle is as for the reflectance based methods described earlier. Commercial sensors are available, including the SPAD chlorophyll sensor (Minola Osaka Company, Ltd., Japan), which measures absorption using red (650-nm) and infrared (940-nm) LEDs in a leaf clip arrangement. SPAD measurements correlate well (R2 >0.70) with plant N concentration in both corn and wheat combined and over multiple years, and growth stages (Chen et al., 2010). Prost and Jeuffroy (2007) evaluated the use of SPAD measurements in the assessment of wheat N status as an alternative to the N nutrition index (NNI) and found that a significant relationship (R2 = 0.89) exists between SPAD measurements and NNI (at flowering), which was independent of cultivar. Debaeke et al. (2006) related normalized SPAD measurements to NNI and found them to be closely related irrespective of year, cultivar or growth stage.

Measurement of flavonoids in the leaf epidermis can also be used as an indicator of N status because plants produce these compounds under stress conditions (e.g., N stress) (Muñoz-Huerta et al., 2013). UV light is also absorbed by these compounds in the leaf epidermis and so flavonoid content can be estimated using absorption methods (e.g., Duelex® sensor, Force-A, Orsay, France) (Tremblay et al., 2009; Cerovic et al., 2012). The Duelex® sensor uses both fluorescence and light transmission through the leaf to calculate both chlorophyll content and epidermal UV-absorbance (Flv). This forms the basis of the N balance index (NBI) calculated as a ratio of the chlorophyll to flavonoid content. A limitation of transmission techniques is that they are contact-based measurements, and therefore not suitable for on-the-go or large area sensing. Similarly, automated measurements are also difficult to perform. The technique measures chlorophyll concentration as a proxy for N content and so limitations in this assumption will also apply (as for reflectance measurements).

Transmission methods can also be applied post-harvest for the measurement of grain protein and water content, and commercial sensors are available (e.g. the CropScan 3000H) for on harvester measurement and mapping. Combined with yield data, these measurements provide useful information on N use and may be used to inform variable rate N management (Long et al., 2005).

Chlorophyll fluorescence

The principles of chlorophyll fluorescence sensing are described in Tremblay et al. (2012). Of the incident solar radiation in the photosynthetically active region (PAR) (400-700 nm), 75% is absorbed by leaves with the majority of this energy dissipated thermally and a small fraction (3%) converted to organic matter. On absorbing energy at a given wavelength chlorophyll molecules also re-emit this energy at a longer wavelength through electron excitation (fluorescence). Measurement of chlorophyll fluorescence at these wavelengths is related to leaf chlorophyll content (Tremblay et al., 2012). Chlorophyll fluorescence can be measured in two ways: (1) by using the variable (dynamic) changes in fluorescence corresponding to changes in the photosynthetic reactions (Kuckenberg et al., 2009; Tremblay et al., 2012), and (2) using the ratio of fluorescence emissions at different wavelengths, which are more suitable for in-field application and measurements at the canopy level. These measurements can be passive, using sun-induced fluorescence or active; e.g., laser-induced fluorescence (Schächtl et al., 2005). Commercial sensors are available, including the Multiplex® (Force-A, Orsay, France) and the MiniVeg-N laser-induced fluorescence sensor (Fritzmeier, Germany). Chlorophyll fluorescence can also be used to measure polyphenolic compounds (e.g., flavonoids) compounds in the leaf epidermis. UV light is also absorbed by these compounds in the leaf epidermis and by comparing chlorophyll fluorescence induced by both UV and red light; the flavonoid content can be estimated (e.g., Quemada et al., 2014).

When used for sensing plant N status, an increase in fluorescence emission indicates a reduction in photosynthetic efficiency, which increases at the early stage of most stress conditions (Chaerle et al., 2007). As measurements are based on fluorescence signals (and not reflected), fluorescence sensing can provide highly sensitive information on plant N status that is independent of soil interference, leaf area or biomass status (Tremblay et al., 2012). Tremblay et al. (2012) suggest that fluorescence sensors could be used for non-invasive detection of stress long before reflectance based measurement of chlorophyll content or biomass, which are long-term effects of stress and also potentially a response to other environment changes. The sensing technique has seen limited application in N management, possibly due to the relative ease of reflectance-based measurements (e.g., for on-the-go). When using variable fluorescence measurements, dark adaptation is required, which is time consuming (10-20 minutes).

Reflectance (hyperspectral)

Spectroscopic analysis of the canopy using visible and near-infrared reflectance (vis-NIR) could provide greater information relating to plant status than indices derived using discrete wavelengths (Meng and Dennison, 2015). For plant analysis, several N absorption features exist below 2500-nm; however, water absorption bands in the NIR region limit use on plant samples that have not been processed (dried) and therefore field application (Chen et al., 2010). Several spectrometers are commercially available and have been used for plant analysis (e.g., FieldSpec, Analytical Spectral Devices (ASD), Boulder, Colorado). Spectrometers are generally expensive, with some exceptions such as the DLP® NIRscanTM sensors (Texas Instruments) covering the 900-2490 nm range. A potential advantage of spectrometers for canopy reflectance measurement is that full spectral information can be retained and multiple vegetation indices estimated simultaneously. Using chemometric approaches, it is also possible to simultaneously derive estimates of water status to separate N from water stress occurring simultaneously (e.g., Kusnierek and Korsaeth, 2015; Lamptey et al., 2017), and potentially the status of other nutrients that are relevant to N decision support (e.g., van Maarschalkerweerd and Husted, 2015).

Machine vision

Most optical sensing mechanisms can also be applied to machine vision (Hague et al., 2000). Image sensors are available that cover visible and near-infrared ranges (e.g., multispectral and hyperspectral cameras). These can be used to image the crop or plant at wavelengths surrounding the ‘red edge’ region, and thus capable of computing the vegetation indices of other reflectance-based optical sensors (e.g., GreenSeeker). Imaging has advantages over these sensors: (1) cameras deployed on remote platforms (UAV) can be used to capture canopy reflectance or thermal measurements with full field coverage at high spatial resolution, (2) ground-based machine vision sensors can better isolate soil influence on the signal by first segmenting leaves and plants within the image, and (3) the image resolution offered by cameras can also be used to resolve structure information at the plant and leaf level that could be utilized for N management of the crop. Several camera systems are commercially available at relatively low cost, and lightweight cameras have been released for UAV application. For example, the Survey2 by MAPIR (San Diego, California) and the RedEdge by MicaSense (Seattle, Washington) that can be installed with a range of filter options for crop vegetation imaging (e.g., NDVI).

Figure 1 (after Li et al., 2010) shows the use of a standard digital camera for measurement of crop N status. Camera measurement of the crop canopy cover correlated well with N status and also with NDVI of the GreenSeeker and Yara N-Sensor, respectively. However, correlation with N status was strong only during the vegetative and early stem elongation phases.

This is a set of two images - a digital camera photo of wheat crop and a digital image showing the wheat crop with segmentation of leaf area.

Figure 1. A digital camera image of wheat crop with segmentation of leaf area (after Li et al. 2010).

Thermal imaging can be used to measure the leaf surface temperature, which is influenced by the cooling effects of transpiration due to stomatal conductance (Irmak et al., 2008). This mechanism responds to a number of stress conditions both abiotic and biotic (Chaerle et al., 2007; Yeboah et al., 2017). Water or nutrient deficiency causes increase in leaf temperature by disrupting the transport of nutrient and water throughout the plant, causing the stomata to close to prevent further moisture loss (Mee et al., 2016).

Tilling et al. (2007) used ground-based spectral (vis-NIR) and airborne multispectral and thermal imagery to quantify N and water stress in wheat (Figure 2). The Canopy Chlorophyll Content Index (CCCI) using the ground-based spectrometer was well correlated with a Nitrogen Stress Index (NSI) derived from plant samples, but correlation was lower with airborne spectral measurements. A major limitation of remote sensing is the influence of the soil background when partially vegetated, but the thermal sensing of water stress could be useful for decisions on N application (Tilling et al., 2007).

This is a set of two airborne images showing Canopy Chlorophyll Content Index (CCCI) (left image) derived using a 3-band multispectral camera (670, 720, 790-nm) and thermal imagery (right image) (after Tilling et al., 2007).

Figure 2. Airborne images of Canopy Chlorophyll Content Index (CCCI) (left image) derived using a 3-band multispectral camera (670, 720, 790-nm) and thermal imagery (right image) (after Tilling et al., 2007).

The measurement of plant physical properties, or structure, can be used as an indicator of plant N status, yield prediction, and N demand. Measurement of leaf area is important for predicting crop growth and yield, and can be well correlated with canopy N content (e.g., Yin et al., 2003; Gebbers et al., 2011; Tavakoli et al., 2014). LAI can be detected remotely using UAV platforms. For N management, Hunt et al. (2010) used a standard digital camera (without infrared blocking filter) and modified with an interference filter to derive a Green Normalized Difference Vegetation Index (GNDVI). With images captured from the UAV platform correlation of GNDVI to LAI was R2 >0.80, but saturated above a LAI of 2.5 m2 m-2.

Sadeghi-Tehran et al. (2017) developed a computer vision framework for the automated monitoring of ear emergence and flowering in wheat using an RGB camera in-field conditions. The accuracy in detection of ear emergence was reported as >90%, depending on the size of training dataset, and accuracies ranged from 80% to 93% for detection of flowering time. Both hyperspectral and chlorophyll fluorescence sensing techniques can also be implemented in computer vision providing information on plant transpiration and photosynthesis relating to N, water and disease status.

Vigneau et al. (2011) evaluated the use of a hyperspectral camera (push broom CCD) with a spectral range from 400 nm to 1000 nm at 3.7 nm resolution. The camera was tractor-mounted on a gantry at a height of 1-m above the crop canopy, and the camera motion along the gantry was used to provide the second imaging dimension. An empirical calibration was used to estimate the leaf N concentration using a chemometric approach from hyperspectral data. Models developed using measurement of individual leaves in both greenhouse and field environment showed good predictability (R2 >0.85).

Machine vision sensors can be used for detection of biotic stress due to pathogens or insect damage. While this is not a direct measurement of plant N status, such information could be useful in decision support systems; for example, by validating the assumption of N stress as the primary factor affecting other optical measurements of plant chlorophyll or biomass (Bauriegel and Herppich, 2014). Biotic stress may also induce yield loss that is assumed to be N-related, machine vision sensors may therefore assist in identifying the cause of yield loss before using this data in N and crop modelling.

3D imaging

3D Imaging of crops using for example LIDAR, stereo cameras or time of flight (TOF), can provide plant physical or structural information. Application of these techniques to date is mostly in high throughput screening and plant phenotyping for breeding. However, in a field context this information could be used normalize measurements from other sensors, that is, account for biomass and growth stage, to support decisions in N management. Commercial sensors are available, as well as software for photogrammetry and point cloud generation. For plant phenotyping, Duan et al. (2016) used multiview camera imaging to reconstruct 3D images of wheat plants (point clouds) using the Multi-View Stereo and Structure From Motion (MVS-SFM) algorithm (Figure 3). From the 3D point cloud, Duan et al. (2016) were able to automatically determine several phenotyping traits, including: tiller and leaf number, plant height and leaf properties useful for N management and yield prediction.

This is an image showing measuring plant structure using multiview camera imaging.

Figure 3. Measuring plant structure using multiview camera imaging (after Duan et. al., 2016).

Optical sensing of nitrogen stress: confounding factors and sensor fusion

Spectral features assist the detection of plant nutritional-related stress when only one factor is causing such stress (Mee et al., 2017). This makes it challenging when trying to discriminate sources of stress influencing plant physiological performance at the same time. This is the case when sources of stress can have similar impacts on the plant’s physiological response (e.g., N and water) consequently leading to the observation of similar spectral responses (Thomason et al., 2007). Peteinatos et al. (2015) evaluated the use of multiple optical sensors, including reflectance (spectrometer), and calculated multiple indices: the CLAAS Isaria® crop sensor, and the Multiplex® to identify stress due to water and N deficiency, weed and fungal infection in wheat. In their study, it was hypothesized that these sensors could be used to detect individual stressors in the presence of others. It was found that some indices, such as the FRFUV provided by the Multiplex® sensor, responded only to N stress whereas indices such as FERRG and ANTHRG responded only to fungal infection. Other indices from the Multiplex® and HandySpec® sensor respond to multiple stress factors. Variability in sensor response due to plant growth stage was more significant than variability due to stress factors (Peteinatos et al., 2015). Therefore, growth stage needs to be considered to define a stress threshold.

Discussion

Analysis of plant tissue to determine nitrogen (N) status has some advantages over soil testing in that the complexities of other soil, plant and environmental properties influencing plant uptake of N are effectively accounted for by measuring the plant directly (e.g., Antille et al., 2015, 2017). For macronutrients, good correlation is usually found between plant concentrations and nutrient status as measured by yield (e.g., Antille et al., 2014; van Maarschalkerweerd and Husted, 2015). Optical sensors typically measure compounds (such as chlorophyll) that relate well to N status. However, this relationship can break-down when soil N availability is high, and as several other factors affect chlorophyll synthesis, it could be difficult to make environmentally-sound decisions on N application based on optical sensors (Tremblay et al., 2009; Galambošová et al., 2014).

Considering the range of sensing techniques available, reflectance-based measurements are easy to perform, show reasonably good correlation with plant N status, can be performed remotely providing high resolution, and several commercial sensors are available. Consequently, site-specific management of N has traditionally focused on reflectance-based technologies. However, a limitation of these technologies is the inconsistency in the relationship with N status based on crop growth stage, biomass and other growth factors, therefore requiring calibration on a site and seasonal basis. There are also different vegetation indices reported in the literature, and there is little consistency in the reported performance of such indices. Alternative sensing techniques, such as transmission-based measurements appear to be more reliable for estimating N status (e.g., SPAD measurements). However, it may not be possible to apply this technique at the spatial resolution required for N management (e.g., 1-m). Transmission-based measurements applied post-harvest (e.g., on-harvester protein-sensing) could also provide valuable information for N modelling, and in estimating N removal from soil.

For the measurement of N status, fluorescence-based techniques have some advantages over reflectance (Živčák et al., 2014a-b), such as no influence of soil background and measurement of polyphenols. This latter enables the calculation of the N buffer index (NBI). Therefore, these may offer improvements in measurement of N status, although the additional measurement and sensor requirements (close range) may not warrant the use of such sensors alone. It had been reported that an on-the-go version of the Multiplex® sensor (A-Force, France) was under development, but this version has since been discontinued. However, factors limiting deployment of sensor technologies, such as active fluorescence sensing or transmission methods, may be addressed within a future autonomous framework; for example, with such sensor deployed using UGV to provide calibration/ground-based referencing of remote sensor measurement.

Using machine vision sensing of reflectance or fluorescence has potential to address some of the limitations in other techniques, such as segmenting of plants prior to calculation of indices and removing soil background effects, although sensor and computation requirements are greater. Deployed on airborne platforms (e.g., UAV), image sensors can provide reflectance measurements at high spatial and temporal resolution. When deployed on a ground vehicle, the camera can measure plant physical properties such as leaf area and height, potentially enabling independent estimates of biomass and chlorophyll content. At the leaf level, machine vision may also allow detecting biotic stresses. While not directly applicable to N decision support, these measurements could be used in diagnosing yield reduction, improving the quality of data applied to modelling and machine learning.

It is also difficult to directly compare sensing techniques for the measurement of crop N status, with the accuracy reported by studies undertaken in different conditions. However, a general classification of sensor performance for N stress measurement was presented in Table 1, which could be used to assist sensor selection.

Conclusions and recommendations

Optical sensor measurements of nitrogen (N) status are indirect, sensors respond to plant stresses that could be induced by a number of biotic and abiotic factors. Sensor measurements are also often dependent on other dominant factors, such as growth stage. A multi-sensor approach may provide a more accurate or reliable indication of N status, and better isolate those factors.

Optical sensing should also be applied post-harvest (e.g., for protein monitoring), and this could provide information on N uptake during crop growth, and quantifying N removed by the crop. There is a requirement for engagement with the tractor / harvesting manufacturers (e.g., John Deere, CNH, CLAAS) to further develop these applications.

Acknowledgements

The research, which informs this article, was made possible by the significant contribution of Australian grain growers through the support of the Grains Research and Development Corporation. The authors would like to thank them for their continued support. This paper is an extract from the final report for Project USQ00022 ‘Future FARM’ commissioned to the authors by GRDC.

Disclaimer

Reference to commercial names or products in this work is solely for the purpose of providing accurate information and does not represent endorsement or otherwise by the authors or their organisations. The graphical information presented in this review is fully credited to the original sources.

References

Antille, D. L., Godwin , R. J., Sakrabani, R., Seneweera, S., Tyrrel, S. F., Johnston, A. E. (2017). Field-scale evaluation of biosolids-derived organomineral fertilizers applied to winter wheat in England. Agronomy Journal, 109(2), 654-674. DOI: 10.2134/agronj2016.09.0495.

Antille, D. L., Hoekstra, N. J., Lalor, S. T. J. (2015). Field-scale evaluation of calcium ammonium nitrate, urea, and urea treated with N-(N-butyl) thiophosphoric triamide applied to grassland in Ireland. Communications in Soil Science and Plant Analysis, 46(11), 1345-1361. DOI: 10.1080/00103624.2015.1033540.

Antille, D. L., Sakrabani, R., Godwin, R. J. (2014). Effects of biosolids-derived organomineral fertilizers, urea, and biosolids granules on crop and soil established with ryegrass (Lolium perenne L.). Communications in Soil Science and Plant Analysis, 45(12), 1605-1621. DOI: 10.1080/00103624.2013.875205.

Baillie, C. P., Thomasson, J. A., Lobsey, C. R., McCarthy, C. L., Antille, D. L. (2018a). A review of the state of the art in agricultural automation. Part I: Sensing technologies for optimization of machine operation and farm inputs. ASABE Paper No.: 1801589. St. Joseph, MI.: 2018 ASABE Annual International Meeting, American Society of Agricultural and Biological Engineers. DOI: 10.13031/aim.201801801589.

Baillie, C. P., Lobsey, C. R., Antille, D. L., McCarthy, C. L., Thomasson, J. A. (2018b). A review of the state of the art in agricultural automation. Part III: Agricultural machinery navigation systems. ASABE Paper No.: 1801591. St. Joseph, MI.: 2018 ASABE Annual International Meeting, American Society of Agricultural and Biological Engineers. DOI: 10.13031/aim.201801801591.

Bauriegel, E., Herppich, W. (2014). Hyperspectral and chlorophyll fluorescence imaging for early detection of plant diseases, with special reference to Fusarium spec. infections on wheat. Agriculture, 4(1), 32-57. DOI: 10.3390/agriculture4010032.

Casadesus, J., Villegas, D. (2014). Conventional digital cameras as a tool for assessing leaf area index and biomass for cereal breeding. Journal of Integrative Plant Biology, 56(1), 7-14. DOI: 10.1111/jipb.12117.

Cendrero-Mateo, M. P., Moran, M. S., Papuga, S. A., Thorp, K. R., Alonso, L., Moreno, J., . . . Wang, G. (2016). Plant chlorophyll fluorescence: active and passive measurements at canopy and leaf scales with different nitrogen treatments. Journal of Experimental Botany, 67(1), 275-286. DOI: 10.1093/jxb/erv456.

Cerovic, Z. G., Masdoumier, G., Ghozlen, N. B., Latouche, G. (2012). A new optical leaf-clip meter for simultaneous non-destructive assessment of leaf chlorophyll and epidermal flavonoids. Physiol. Plant, 146(3), 251-260. DOI: 10.1111/j.1399-3054.2012.01639.x.

Chaerle, L., Leinonen, I., Jones, H. G., Van Der Straeten, D. (2007). Monitoring and screening plant populations with combined thermal and chlorophyll fluorescence imaging. Journal of Experimental Botany, 58(4), 773-784. DOI: 10.1093/jxb/erl257.

Chen, P. (2015). A comparison of two approaches for estimating the wheat nitrogen nutrition index using remote sensing. Remote Sensing, 7(4), 4527-4548. DOI: 10.3390/rs70404527.

Chen, P., Haboudane, D., Tremblay, N., Wang, J., Vigneault, P., Li, B. (2010). New spectral indicator assessing the efficiency of crop nitrogen treatment in corn and wheat. Remote Sensing of Environment, 114(9), 1987-1997. DOI: 10.1016/j.rse.2010.04.006.

Costa, J. M., Grant, O. M., Chaves, M. M. (2013). Thermography to explore plant-environment interactions. Journal of Experimental Botany, 64(13), 3937-3949. DOI: 10.1093/jxb/ert029.

Craigie, R., Yule, I., P, M. (2013). Crop sensing for nitrogen management. FAR FOCUS Issue 9, pp. 32. Christchurch, New Zealand: Foundation for Arable Research. Available [Here].

Debaeke, P., Rouet, P., Justes, E. (2006). Relationship between the normalized SPAD index and the nitrogen nutrition index: Application to durum wheat. Journal of Plant Nutrition, 29(1), 75-92. DOI: 10.1080/01904160500416471.

Duan, T., Chapman, S. C., Holland, E., Rebetzke, G. J., Guo, Y., Zheng, B. (2016). Dynamic quantification of canopy structure to characterize early plant vigour in wheat genotypes. Journal of Experimental Botany, 67(15), 4523-4534. DOI: 10.1093/jxb/erw227.

Erdle, K., Mistele, B., Schmidhalter, U. (2011). Comparison of active and passive spectral sensors in discriminating biomass parameters and nitrogen status in wheat cultivars. Field Crops Research, 124(1), 74-84. DOI: 10.1016/j.fcr.2011.06.007.

Feng, W., Yao, X., Zhu, Y., Tian, Y. C., Cao, W. X. (2008). Monitoring leaf nitrogen status with hyperspectral reflectance in wheat. European Journal of Agronomy, 28(3), 394-404. DOI: 10.1016/j.eja.2007.11.005.

Force-A (2017), DUALEX Scientific - user’s manual. Retrieved from [Here].

Galambošová, J., Macák, M., Živčák, M., Rataj, V., Slamka, P., Olšovská, K. (2014). Comparison of spectral reflectance and multispectrally induced fluorescence to determine winter wheat nitrogen deficit. Advanced Materials, 1059: 127-133. DOI: 10.4028/www.scientific.net/AMR.1059.127.

Gebbers, R., Ehlert, D., Adamek, R. (2011). Rapid mapping of the leaf area index in agricultural crops. Agronomy Journal, 103(5), 1532-1541. DOI: 10.2134/agronj2011.0201.

Hague, T., Marchant, J. A., Tillett, N. D. (2000). Ground based sensing systems for autonomous agricultural vehicles. Computers and Electronics in Agriculture, 25(1-2), 11-28. DOI: 10.1016/s0168-1699(99)00053-8.

Hansen, P., Schjoerring, J. (2003). Reflectance measurement of canopy biomass and nitrogen status in wheat crops using normalized difference vegetation indices and partial least squares regression. Remote Sensing of Environment, 86(4), 542-553.

Hunt Jr., E. R., Dean Hively, W., Fujikawa, S. J., Linden, D. S., Daughtry, C. S. T., McCarty, G. W. (2010). Acquisition of NIR-green-blue digital photographs from unmanned aircraft for crop monitoring. Remote Sensing, 2(1), 290-305. DOI: 10.3390/rs2010290.

Irmak, S., Mutiibwa, D., Irmak, A., Arkebauer, T. J., Weiss, A., Martin, D. L., Eisenhauer, D. E. (2008). On the scaling up leaf stomatal resistance to canopy resistance using photosynthetic photon flux density. Agricultural and Forest Meteorology, 148(6-7), 1034-1044. DOI: 10.1016/j.agrformet.2008.02.001.

Justes, E., Mary, B., Meynard, J.-M., Machet, J.-M., Thelier-Huché, L. (1994). Determination of a critical nitrogen dilution curve for winter wheat crops. Annals of botany, 74(4), 397-407.

Konica Minolta (2009). Chlorophyll meter SPAD-502Plus. Technical Specifications. Retrieved from [Here].

Kuckenberg, J., Tartachnyk, I., Noga, G. (2009). Temporal and spatial changes of chlorophyll fluorescence as a basis for early and precise detection of leaf rust and powdery mildew infections in wheat leaves. Precision Agriculture, 10(1), 34-44. DOI: 10.1007/s11119-008-9082-0.

Kusnierek, K., Korsaeth, A. (2015). Simultaneous identification of spring wheat nitrogen and water status using visible and near infrared spectra and Powered Partial Least Squares Regression. Computers Electron. Agric., 117, 200-213. DOI: 10.1016/j.compag.2015.08.001.

Lamptey, S., Li, L., Xie, J., Zhang, R., Yeboah, S., Antille, D. L. (2017). Photosynthetic response of maize to nitrogen fertilization in the semiarid Western Loess Plateau of China. Crop Science, 57(5), 2739-2752. DOI: 10.2135/cropsci2016.12.1021.

Li, Y., Chen, D., Walker, C. N., Angus, J. F. (2010). Estimating the nitrogen status of crops using a digital camera. Field Crops Research, 118(3), 221-227. DOI: 10.1016/j.fcr.2010.05.011.

Long, D. S., Engel, R. E., Carpenter, F. M. (2005). On-combine sensing and mapping of wheat protein concentration. Crop Management, 4(1), 0-0.

Maier, S. W., Günther, K. P., Stellmes, M. (2003). Sun-induced fluorescence: A new tool for precision farming. Digital imaging and spectral techniques: Applications to precision agriculture and crop physiology (digitalimaginga), 209-222.

McBratney, A., Pringle, M. (1999). Estimating average and proportional variograms of soil properties and their potential use in precision agriculture. Precision Agriculture, 1(2), 125-152.

Mee, C. Y., Balasundram, S. K., Mohd Hanif, A. H. (2017). Detecting and Monitoring Plant Nutrient Stress Using Remote Sensing Approaches: A Review. Asian Journal of Plant Sciences, 16, 1-8.

Meng, R., Dennison, P. E. (2015). Spectroscopic analysis of green, desiccated and dead Tamarisk canopies. Photogrammetric Engineering and Remote Sensing, 81(3), 199-207. DOI: 10.14358/PERS.81.3.199.

Mistele, B., Schmidhalter, U. (2008). Estimating the nitrogen nutrition index using spectral canopy reflectance measurements. European Journal of Agronomy, 29(4), 184-190. DOI: 10.1016/j.eja.2008.05.007.

Moges, S. M., Raun, W. R., Mullen, R. W., Freeman, K. W., Johnson, G. V., Solie, J. B. (2005). Evaluation of green, red, and near infrared bands for predicting winter wheat biomass, nitrogen uptake, and final grain yield. Journal of Plant Nutrition, 27(8), 1431-1441. DOI: 10.1081/pln-200025858.

Moron, A., García, A., Sawchik, J., Cozzolino, D. (2007). Preliminary study on the use of near‐infrared reflectance spectroscopy to assess nitrogen content of undried wheat plants. Journal of the Science of Food and Agriculture, 87(1), 147-152.

Munoz-Huerta, R. F., Guevara-Gonzalez, R. G., Contreras-Medina, L. M., Torres-Pacheco, I., Prado-Olivarez, J., Ocampo-Velazquez, R. V. (2013). A review of methods for sensing the nitrogen status in plants: advantages, disadvantages and recent advances. Sensors (Basel), 13(8), 10823-10843. DOI: 10.3390/s130810823.

Oxenham, D. J., Catchpoole, V. R., Dolby, G. R. (1983). A comparison of dry combustion and acid digestion for the determination of nitrogen in soil and plant material. Communications in Soil Science and Plant Analysis, 14(2), 153-160. DOI: 10.1080/00103628309367351.

Peteinatos, G., Korsaeth, A., Berge, T., Gerhards, R. (2016). Using optical sensors to identify water deprivation, nitrogen shortage, weed presence and fungal infection in wheat. Agriculture, 6(2), 24.

Prost, L., Jeuffroy, M.-H. (2007). Replacing the nitrogen nutrition index by the chlorophyll meter to assess wheat N status. Agronomy for Sustainable Development, 27(4), 321-330. DOI: 10.1051/agro:2007032.

Quemada, M., Gabriel, J., Zarco-Tejada, P. (2014). Airborne hyperspectral images and ground-level optical sensors as assessment tools for maize nitrogen fertilization. Remote Sensing, 6(4), 2940-2962. DOI: 10.3390/rs6042940.

Raun, W. R., Solie, J. B., Johnson, G. V., Stone, M. L., Mullen, R. W., Freeman, K. W., Thomason, W. E., Lukina, E. V. (2002). Improving nitrogen use efficiency in cereal grain production with optical sensing and variable rate application contribution from the Oklahoma Agric. Exp. Stn. Agronomy Journal, 94(4), 815-820. DOI: 10.2134/agronj2002.8150.

Raun, W. R., Solie, J. B., Stone, M. L., Martin, K. L., Freeman, K. W., Mullen, R. W., . . . Johnson, G. V. (2005). Optical Sensor‐Based Algorithm for Crop Nitrogen Fertilization. Communications in Soil Science and Plant Analysis, 36(19-20), 2759-2781. DOI: 10.1080/00103620500303988.

Rochester, I. J. (2011). Assessing internal crop nitrogen use efficiency in high-yielding irrigated cotton. Nutrient Cycling in Agroecosystems, 90(1), 147-156. DOI: 10.1007/s10705-010-9418-9.

Rodriguez, D., Fitzgerald, G. J., Belford, R., Christensen, L. K. (2006). Detection of nitrogen deficiency in wheat from spectral reflectance indices and basic crop eco-physiological concepts. Australian Journal of Agricultural Research, 57(7), 781. DOI: 10.1071/ar05361.

Rorie, R. L., Purcell, L. C., Karcher, D. E., King, C. A. (2011). The assessment of leaf nitrogen in corn from digital images. Crop Science, 51(5), 2174-2180. DOI: 10.2135/cropsci2010.12.0699

Ruiz Diaz, D. A., Hawkins, J. A., Sawyer, J. E., Lundvall, J. P. (2008). Evaluation of in-season nitrogen management strategies for corn production. Agronomy Journal, 100: 1711-1719. DOI: 10.2134/agronj2008.0175.

Sadeghi-Tehran, P., Sabermanesh, K., Virlet, N., Hawkesford, M. J. (2017). Automated method to determine two critical growth stages of wheat: heading and flowering. Front. Plant Sci., 8, 252. DOI: 10.3389/fpls.2017.00252

Samborski, S. M., Gozdowski, D., Walsh, O. S., Kyveryga, P., Stłpieł, M. (2017). Sensitivity of sensor-based nitrogen rates to selection of within-field calibration strips in winter wheat. Crop and Pasture Science, 68(2), 101-114. DOI: 10.1071/cp16380.

Schächtl, J., Huber, G., Maidl, F.-X., Sticksel, E., Schulz, J., Haschberger, P. (2005). Laser-induced chlorophyll fluorescence measurements for detecting the nitrogen status of wheat (Triticum aestivum L.) canopies. Precision Agriculture, 6(2), 143-156.

Tartachnyk, I. I., Rademacher, I., Kühbauch, W. (2006). Distinguishing nitrogen deficiency and fungal infection of winter wheat by laser-induced fluorescence. Precision Agriculture, 7(4), 281-293. DOI: 10.1007/s11119-006-9008-7.

Tavakoli, H., Mohtasebi, S., Alimardani, R., Gebbers, R. (2014). Evaluation of different sensing approaches concerning to non-destructive estimation of leaf area index (LAI) for winter wheat. International Journal on Smart Sensing & Intelligent Systems, 7(1): 337-359.

Thomason, W. E., Phillips, S. B., Raymond, F. D. (2007). Defining useful limits for spectral reflectance measures in corn. Journal of Plant Nutrition, 30(8), 1263-1277.

Thomasson, J. A., Baillie, C. P., Antille, D. L., McCarthy, C. L., Lobsey, C. R. (2018). A review of the state of the art in agricultural automation. Part II: On-farm agricultural communications and connectivity. ASABE Paper No.: 1801590. St. Joseph, MI.: 2018 ASABE Annual International Meeting, American Society of Agricultural and Biological Engineers. DOI: 10.13031/aim.201801801590.

Tilling, A. K., O'Leary, G. J., Ferwerda, J. G., Jones, S. D., Fitzgerald, G. J., Rodriguez, D., Belford, R. (2007). Remote sensing of nitrogen and water stress in wheat. Field Crops Research, 104(1-3), 77-85. DOI: 10.1016/j.fcr.2007.03.023.

Tremblay, N., Wang, Z., Bélec, C. (2009). Performance of Dualex in spring wheat for crop nitrogen status assessment, yield prediction and estimation of soil nitrate content. Journal of Plant Nutrition, 33(1), 57-70. DOI: 10.1080/01904160903391081.

Tremblay, N., Wang, Z., Cerovic, Z. G. (2012). Sensing crop nitrogen status with fluorescence indicators. A review. Agronomy for Sustainable Development, 32(2), 451-464. DOI: 10.1007/s13593-011-0041-1.

van Maarschalkerweerd, M., Husted, S. (2015). Recent developments in fast spectroscopy for plant mineral analysis. Front. Plant Sci., 6, 169. DOI: 10.3389/fpls.2015.00169.

Vigneau, N., Ecarnot, M., Rabatel, G., Roumet, P. (2011). Potential of field hyperspectral imaging as a non-destructive method to assess leaf nitrogen content in Wheat. Field Crops Research, 122(1), 25-31. DOI: 10.1016/j.fcr.2011.02.003.

Wang, T.-C., Ma, B., Xiong, Y.-C., Saleem, M. F., Li, F.-M. (2011). Optical sensing estimation of leaf nitrogen concentration in maize across a range of water-stress levels. Crop and Pasture Science, 62(6), 474-480.

Whelan, B. (2015). Proximal Crop Reflectance Sensors – A guide to their capabilities and uses. Retrieved from [Here].

Yeboah, S., Zhang, R., Cai, L., Li, L., Xie, J., Luo, Z., Wu, J., Antille, D. L. (2017). Soil water content and photosynthetic capacity of spring wheat as affected by soil application of nitrogen-enriched biochar in a semiarid environment. Photosynthetica, 55(3), 532-542. DOI: 10.1007/s11099-016-0672-1.

Yin, X., Lantinga, E. A., Schapendonk, A. H., Zhong, X. (2003). Some quantitative relationships between leaf area index and canopy nitrogen content and distribution. Annals of Botany, 91(7), 893-903.

Zillmann, E., Graeff, S., Link, J., Batchelor, W. D., Claupein, W. (2006). Assessment of cereal nitrogen requirements derived by optical on-the-go sensors on heterogeneous soils. Agronomy Journal, 98(3), 682-690.

Živčák, M., Olšovská, K., Slamka, P., Galambošová, J., Rataj, V., Shao, H. B., Brestič, M. (2014a). Application of chlorophyll fluorescence performance indices to assess the wheat photosynthetic functions influenced by nitrogen deficiency. Plant, Soil and Environment, 60(5), 210-215.

Živčák, M., Olšovská, K., Slamka, P., Galambošová, J., Rataj, V., Shao, H. B., Kalaji, H. M., Brestič, M. (2014b). Measurements of chlorophyll fluorescence in different leaf positions may detect nitrogen deficiency in wheat. Zemdirbyste, 101(4), 437-444. DOI: 10.13080/z-a.2014.101.056.

Contact details

Craig Baillie
Centre for Agricultural Engineering (CAE)
Institute for Advanced Engineering & Space Sciences (IAESS)
Ph: 07 4631 2071
Mb: 0428 750 060
Email: craig.baillie@usq.edu.au

Diogenes L. Antille
Ph: +61 (07) 4631 2948
Email: Dio.Antille@usq.edu.au

® Registered trademark

GRDC Project Code: USQ00022,