Take home messages

• Green-on-green spot spraying products are starting to roll out in North America, Europe and Australia
• The primary factor limiting more widespread availability is the time required to adapt the spray system to different locations and weeds
• Research is focussing on new crop/weed types, aerial mapping, and new machine learning techniques
• Recent disruptive artificial intelligence (AI) technologies are starting to be used by both providers and end-users to be more productive.

Background

This paper aims to summarise the cu­rrent state of product development for green-on-green spot spraying. Green-on-green spot spraying is an important innovation for agriculture, offering the potential for reduced chemical use, enhanced weed control, and data-driven decision making. Both original equipment manufacturers (OEMs) such as John Deere (see Figure 1) and third-party suppliers are developing green-on-green spot spray systems for Australia, North America, and Europe.

By looking at the locations and weed/crop types that manufacturers are targeting, we can gain insight into what will be available in the short term in Australia. The scientific community is also continually active in adapting the latest technologies to agricultural applications, and so it is also worth reviewing what researchers are focussing on in the field of weed management.

Figure 1. See and Spray Ultimate from John Deere, a green-on-green spot spraying solution.     Source: Agfri (2022).

Current state of play

Boom sprayers

Table 1 shows the current or near-release products at the time of publication. Only Bilberry is currently available in Australia, with two systems available each for the U.S. and Europe this year. Precision Planting has also announced an entry into the sprayer market, which includes vision systems to identify weeds (Precision Planting, 2023).

Table 1. Boom sprayer green-on-green products.

The deep learning models that are used for detecting weeds in these vision systems have been less adaptable to new locations than expected, which has resulted in new products being rolled out only one country or region at a time. Each new location requires new data collection, model training, and validation. This may place Australia behind other regions (e.g., North America, South America) to receive access to these new spot spraying systems.

One other notable trend is the investment by original equipment manufacturers (OEMs) in companies that specialise in crop or weed sensing. John Deere was the first to move in 2018 with the purchase of Blue River Technologies (which then contributed to See & Spray™ Ultimate). AGCO has collaborated with Bosch BASF and in 2021 invested in Greeneye™ Technology (AGCO, 2023; AGCO, 2021). Case New Holland (CNH) has also invested in this space, first with Raven Technologies and then with Augmenta (Augmenta, 2023).

Aerial mapping/spraying

Unmanned aerial vehicles (UAVs) are another tool used in weed management. UAVs were first used as a form of rapid aerial survey, using consumer grade models with colour cameras to produce a weed map which is in turn loaded onto a sprayer. The Single Shot system (Single Shot, 2023) is one example of this operating as a service to growers.

Spray drones have also recently emerged as an alternate method of herbicide delivery, such as the DJI Agras series (see Figure 2) or XAG P30 (DJI, 2023a; XAG Australia, 2023). These products can accept prescription maps but cannot yet sense and spray in the one pass.

Figure 2. The Agras T30 spray drone from DJI.  Source: DJI (2023b).

Over the horizon – industry

There are some insights that can be gleaned from looking at the current release cadence of green-on-green products to predict what might be available in Australia in the coming years. The deep learning detection of weeds in the vision system is the limiting factor, slowing down wider availability of these products, specifically the requirement to retrain or adapt the vision system for each new region.

The region-specific nature of the deep learning weed detection can be observed by looking at the roadmap for Greeneye™, where the system is being made available only for certain parts of the U.S. Greeneye™ are rolling the product out to other states over time, and it is the author’s speculation that this is because extra time is required to validate the vision detection for the other states.

With expansion to new regions requiring extra R&D, Australia will be in line to get access to the products already available in the U.S. It is reasonable to expect that the crop and weed types currently covered by See & Spray™ Ultimate will eventually be available in Australia (See & Spray™ Select was launched in Australia). Greeneye™ have announced that they are currently adapting their vision systems for the Brazil market, and so Australian growers may be waiting a little longer to have a brand-agnostic green-on-green solution.

In addition to new regions, the manufacturers are also of course developing the weed detection to work for new weeds and in new crops. The list of available crop/weed combinations in Table 1 is hardly comprehensive, and additional crops and weeds will trickle in over the coming years. For the product already available in Australia (i.e., Bilberry Weedetect), we will need to look directly to the manufacturer to see what new crop/weed types are coming next, rather than looking at overseas commercial releases.

The last form of expansion for the boom sprayers are new applications beyond weeds. Nearly every manufacturer listed in Table 1 has declared a roadmap that includes precision application of nutrients, fungicides and pesticides based on vision technology from the same cameras that are now doing weed spot spraying (Precision Farming Dealer, 2023). Emergence mapping is another function mentioned by Precision Planting (2023) and Bosch BASF (Precision Farming Dealer, 2023).

Over the horizon – scientific community

Research from the scientific community has often laid the foundation for technologies that are later integrated into industry products. As an example, the use of neural networks to perform object detection was first proposed by Girshick et al. (2014) and is foundational as to how the green-on-green spot spray systems work. This section will cite a few instances of recent work to give an overview of current activity in precision weed management.

With R&D on cameras on a spray boom covered by the manufacturers, research in the scientific community is centred more on aerial imagery and smaller ground vehicles such as manual weeding vehicles. Examples of this work are from Islam et al. (2021) on an Australian chilli farm, Khan et al. (2021) on strawberry and pea fields, and Osorio et al. (2020) in lettuce. Some of the typical novel contributions of this research are:

• Testing existing deep learning models on new crop/weed combinations
• Exploring a novel version of a deep learning model that might be more effective for weed detection
• Collecting and making available a dataset for other researchers to use (a list of available datasets was summarised by Coleman et al., 2022)

Data augmentation is a process in training a deep learning weed detection model that makes it resistant to small changes in image content. Sapkota et al. (2022) explored a similar technique of expanding a dataset of images with synthetic images created from mixing and merging the foreground and background of various images. Improving data augmentation techniques means that models may be more effective when being transferred between regions, easing the load on manufacturers to do so much localisation.

A recent review of weed sensing technologies was performed by Rai et al. (2023), which is a good starting point for further reading.

AI and productivity

Parallel to these advances in the application of artificial intelligence (AI), the world has also experienced massive disruption from other recent advances in AI – starting with the public test release of ChatGPT. These new tools have enabled software developers (and many other workers) to make new and unexpected leaps in productivity, which is relevant both to the manufacturers and end-users.

For the developers, the number of tasks that can be performed by software has dramatically increased. Large language models (LLMs) like ChatGPT can now be tasked with writing code, scanning documents, and interpreting photos (OpenAI, 2023), and AI functions that more directly assist with managing deep learning datasets are likely to exist soon. As a result, the barriers to adapting the vision systems for new regions or crops/weeds will be managed more easily thanks to the new tools at the developer’s disposal.

The introduction of LLMs to the public will also have benefits for the end-users. If the LLM is given access to the manual for a sprayer, it will become a very effective means for growers to troubleshoot issues with machines.

Conclusion

Weed management technologies continue to evolve and we are now seeing multiple providers with products available in Australia, the U.S. and Europe. We can expect the following from the manufacturers:

• Products that are launching overseas (particularly in the U.S.) are likely to be available in Australia in the future.
• Products will continually be adding new crop/weed combinations to their vision systems, which might be available as a software update.
• Applications for the camera systems other than controlling herbicide (fungicide, pesticide, nutrients) are in development.

The general perception of AI and what it is capable of changed in late 2022, and this could be a large boost to manufacturers’ ability to deliver on their vision of green-on-green spot spraying across the globe.

References

AGCO (2021) ACGO invests in Greeneye Technology, an innovator in smart farming solutions for precision spraying, press release

AGCO (2023) AGCO and Bosch BASF Smart Farming announce joint development and commercialization of smart spraying capabilities, press release

Afgri Equipment (2022) Deere release See & Spray Ultimate in US, press release

Augmenta (2023) Augmenta acquired by CNH Industrial, press release

BASF (2023) Smart Spraying Solution from Bosch BASF Smart Farming enters first series production at Dammann for Germany and Hungary, press release.

Coleman GRY, Bender A, Hu K, Sharpe SM, Schumann AW, Wang Z, Bagavathiannan MV, Boyd NS and Walsh MJ (2022) Weed detection to weed recognition: reviewing 50 years of research to identify constraints and opportunities for large-scale cropping systems. Weed Technology, 36, 741-757

DJI (2023a) Agras T30 A new flagship for digital agriculture, website

DJI (2023b) DJI Agras T30, photo

Goldacres (2022) Weedetect - Bilberry, product brochure

Girshick R, Donahue J, Darrell T and Malik J (2014) Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, 23-28 June 2014, 580-587.

Greeneye Technology (2021) Greeneye announces commercial launch of AI precision spraying technology proven to cut herbicide use by 78%, U.S. launch press release

Islam N, Rashid MM, Wibowo S, Xu C-Y, Morshed A, Wasimi SA, Moore S and Rahman SM (2021) Early Weed Detection Using Image Processing and Machine Learning Techniques in an Australian Chilli Farm. Agriculture 11 (5), 387

John Deere (2023a) John Deere launches See & Spray™ Ultimate technology, press release

John Deere (2023b) See & Spray™ Ultimate, web page

Khan S, Tufail M, Khan MT, Khan ZA and Anwar S (2021) Deep learning-based identification system of weeds and crops in strawberry and pea fields for a precision agriculture sprayer. Precision Agriculture 22, 1711-1727

OpenAI (2023) GPT-4 technical report

Osorio K, Puerto A, Pedraza C, Jamaica D and Rodríguez L (2020) A deep learning approach for weed detection in lettuce crops using multispectral images. AgriEngineering 2(3), 471-488

Precision Planting (2022) Sprayer technology press release

Precision Farming Dealer (2023) Inside the Smart Spraying collaboration between AGCO & Bosch BASF Smart Farming, podcast

Rai N, Zhang Y, Ram BG, Schumacher L, Yellavajjala RK, Bajwa S and Sun X (2023) Applications of deep learning in precision weed management: A review. Computers and Electronics in Agriculture 206, 107698

Sapkota BB, Popescu S, Rajan N, Leon RG, Reberg-Horton C, Mirsky S and Bagavathiannan MV (2022) Use of synthetic images for training a deep learning model for weed detection and biomass estimation in cotton. Scientific Reports 12, 19580

Single Agriculture (2023) Single Shot drone weed mapping technology, website

XAG Australia (2020) P30 2020 technical specs

Contact details

Derek Long
University of Southern Queensland
487-535 West St, Darling Heights, Toowoomba 4350
Ph: 07 4631 1515
Email: Derek.long@usq.edu.au

Date published: July 2023