Automation: opportunities for adoption in agriculture
Automation: opportunities for adoption in agriculture
Author: Jacob Humpal and Craig Baillie (Centre for Agricultural Engineering, University of Southern Queensland) | Date: 24 Feb 2022
Take home messages
- Automation is a stepping-stone on the path to autonomy.
- Automation reduces risk by increasing precision and repeatability.
- Opportunities for farm automation are here now.
Background
Gains in productivity are being made through increased control of inputs and through precise timing of management operations, indicating a potential use case for increased integration of automation and autonomous systems on-farm.
Original equipment manufacturers (OEMs), such as John Deere and Case IH, are primarily developing machinery with an increasing level of automation leading to full autonomy.
However, there is a lack of consumer available functionality which provides active perception and decision making to on-farm machinery. This limits the potential of the automated technologies to situations where an operator can be present, limiting operations both by the number of automation capable and equipped machines and by the number of available skilled operators.
While automated systems are widely used in agriculture today, a focus will need to be on bringing active perception and intelligent decision-making tools to automated systems, allowing for fully autonomous systems to be viable and actively adopted in agriculture. This will allow the removal of the operator, enabling operations to be completed with increased efficiency.
Fundamentals of automation for agriculture
Defining automation to autonomy
The initial step towards on-farm autonomous machinery is automation. Automation allows a machine to complete a series of functions automatically. This differs to autonomy where the machine must complete a task without human assistance while having sufficient skill to complete that task in changing environments. This difference is illustrated when looking at a machine which can be operated using an automatic guidance system on pre-defined tracks and a machine which can follow those pre-defined tracks, identify a perceived obstacle and make a decision regarding the interaction of the current operation and that obstacle (Figure 1).
Figure 1. A) An automatic machine requiring human input to make a decision and avoid an obstacle. B) An autonomous machine identifying an obstacle and making a decision regarding interaction with that obstacle.
Autonomy is driven by perception which can be broken down into two groups:
- Locative Perception where the machine knows something about its surroundings and its place in those surroundings.
- Task Perception where the machine knows something about what it is doing and how well this task is being done.
Machine objectives
The key features which enable tractor autonomy are defined in Baillie et al. (2020, 2017) and include the automated systems which work together to provide an autonomous solution, such as automated tractor guidance systems, variable rate technology (VRT), sensing and perception systems and in-field communication.
Currently, OEMs and third-party equipment manufacturers do not offer commercial autonomous navigation perception or sensing options for integration into their existing automation technology stacks. These options are presently limited to expensive, single-use technologies from high-tech start-ups. However, perception technologies currently exist within OEM and third-party technology stacks for other uses and these companies are actively investing in and exploring autonomous use cases, as evidenced by John Deere’s recent acquisition of Bear Flag Robotics (John Deere, 2021a) and John Deere, Case New Holland (CNH), and Kubota’s development of autonomous tractor concepts (Case IH, 2021a; John Deere, 2021b; Wilson, 2020) (Figure 2).
Figure 2. Kubota X autonomous tractor (Topspeed Media, 2021).
Current state-of-the-art
Currently, the majority of OEM and third-party provider offerings revolve around automated guidance systems and VRT, with some expansion into machine vision and perception as operator assistance tools. All major OEMs provide an option for automated tractor guidance through GPS and RTK correction systems (Case IH, 2021b; CLAAS, 2018a; Deutz Fahr, 2021; Fendt, 2021; John Deere, 2021c) as well as VRT application (Case IH, 2021c; CLAAS, 2018b; John Deere, 2021d). However, there is limited product development of perception technologies by OEMs for active obstacle avoidance. Current perception technologies developed by OEMs are focused on in-row wheel guidance and harvester optimisation.
Locative perception
CNH provides a range of sensing and perception technologies through their technology stacks with a focus on in-crop row guidance and harvester optimisation. AFS RowGuide™ provides mechanical deflection-based row guidance (Case IH, 2021d), while Cruise Cut provides laser-based harvester guidance through the detection of the crop edge (Case IH, 2021b).
CLAAS’s range of sensing and perception technologies have also been developed primarily around in-crop row guidance and harvest optimisation. CLAAS provides three separate sensor-based automatic steering options for their harvesters, AUTO PILOT, CAM PILOT and LASER PILOT (CLAAS Group, 2021). CLAAS AUTO PILOT uses two mechanical deflection-based sensors to determine the position of maize rows relative to the harvester. CLAAS CAM PILOT builds on this using a stereo camera to control automatic steering in grass harvesting operations. Finally, CLAAS LASER PILOT uses a LiDAR to guide the harvester along the cut edge of cereal crops.
John Deere provides sensing options for in-crop guidance through AutoTrac TM Vision. AutoTracTM Vision uses a single frame-mounted camera using machine vision to keep sprayer wheels in between crop rows (John Deere, 2021e).
SAME Deutz-Fahr (SDF) developed Driver Extended Eyes to assist the operator in navigating machine blind spots. This technology uses an intelligent control unit to actively monitor up to six cameras, extending the view of the driver to the left and right and automatically preventing tractor movement if a human obstacle is detected by cameras (SDF Group, 2021). SDF also uses their machine vision camera technology to provide automatic trailer hitch coupling using environmental recognition to control tractor movement. These cameras actively identify the position of an implement in relation to the tractor and the control system calculates the best path for coupling to the implement, automatically performing all necessary movements (SDF Group, 2021).
Raven Precision, recently acquired by CNH Industrial (Raven Precision, 2021a), has also developed in-row guidance systems and offers the VSN® Visual Guidance Sensor, a stereo vision camera-based machine vision system which can navigate crop rows up to full canopy closure (Raven Precision, 2021b). Raven’s AutoBoom® XRT uses radar and optical obstacle detection to maintain optimal spray height (Raven Precision, 2021c).
Raven Precision is on the leading edge of full on-farm autonomy with their OMNiDRIVETM and OMNiPOWERTM systems. Raven’s OMNiDRIVETM system is an autosteer system which allows an operator to monitor and control a driverless tractor from the cab of the harvester (Raven Precision, 2021d). The autonomous tractor can return to a predetermined unloading area when the grain cart is full. Raven’s OMNiPOWERTM is a self-propelled power platform which allows fully autonomous control of field tasks which can be remotely commanded and supervised. However, OMNiPOWERTM requires specially made implements and currently only supports a nutrient applicator spreader and a VRT spray system (Raven Precision, 2021e).
Task perception
CNH developed AFS Harvest Command for Case IH machines which provides automatic adjustments to the harvester in real-time as conditions change (Case IH, 2021e). CNH also provides IntelliFillTM, a 3D camera-based system to detect trailer edges and monitor forage harvester filling on New Holland machines (New Holland Agriculture, 2021).
CLAAS offers variable rate harvesting control and automatic trailer filling using a 3D camera to detect trailer edges and monitor and adjust the upper grain discharge chute during cart fill (CLAAS, 2021).
John Deere also provides sensing options for automatic grain-cart filling and real-time spot spraying. John Deere’s Active Fill Control controls the harvester discharge chute similar to CLAAS’ system, enabling automated cart filling (John Deere, 2021f). John Deere’s See & SprayTM Select uses an integrated camera and machine vision algorithms to automatically detect green plants within fallow ground, triggering a chemical application (Burwood-Taylor, 2017).
Future opportunities
As OEMs prepare for the future, they are looking towards technologies such as big data, artificial intelligence, unmanned aerial vehicles (UAVs) and electrification. CNH has produced two autonomous tractor concept vehicles to date. One tractor was a futuristic concept cab-less Case IH while the New Holland NHDrive concept resembles a retrofit tractor with a cab which allows operator control. Both CNH vehicles use AI, GPS/RTK, LiDAR, radar and RGB camera sensors to run autonomously. John Deere has developed both a concept autonomous one axle electric-drive tractor and an autonomous electric-drive sprayer, both concepts were unveiled at Agritechnica 2019. The electric-drive sprayer is fully autonomous with a 30-foot boom, 560-litre spray tank, independent four-wheel steering and tracks to reduce ground pressure (Francis, 2019). John Deere has indicated that they will consider improvements to battery life, speed and autonomous filling (Real Agriculture, 2019). John Deere has actively pursued these battery improvements as evidenced by their recent acquisition of majority ownership in Kreisel Electric, a high-density, high-durability battery manufacturer (John Deere, 2021g). Autonomous filling was revealed at Agritechnica 2019 on John Deere’s semi-autonomous tractor concept as part of their Automation to Autonomy core technology stack for the future of farming (John Deere, 2021b). The spray tank on this machine is capable of being filled completely automatically to reduce operator exposure to pesticides. John Deere’s autonomous electric one axle tractor concept removes operating emissions and greatly reduces noise levels (John Deere, 2021b). John Deere has also developed a Command Cab, a future tractor cab concept which John Deere views as an intermediate step from automation to autonomy, integrating big data to provide a mobile command centre for the farm (John Deere, 2021b). Finally, John Deere has announced the development of a bolt-on package of hardware and software that uses machine learning to create a fully autonomous tractor which will begin production in late 2022 (Vincent, 2022). If successful, John Deere may have the first widespread, OEM developed autonomous tractor kit on the market.
Multiple manufacturers are also developing autonomous UAV spray systems. John Deere is developing an autonomous swarm drone sprayer concept. The drones are equipped with John Deere’s weed identification technologies and each carry a 10.6L spray tank which is filled automatically at a field boundary station (John Deere, 2021b). Kubota is also developing pesticide spraying drones, integrating environmental data into their current decision system software and aim to optimise planting patterns through AI decision systems (Kubota, 2021). Kubota has developed a three-step roadmap to autonomy (Kubota, 2021). This revolved around Kubota’s Agri Robo Tractor launch in 2017, followed by the Agri Robo Combine Harvester in 2018 under the plan. Kubota further plans to implement unmanned operation across multiple fields and enable remote monitoring of equipment travel on farm roads. While Case IH has not yet signalled anything around the development of UAV spray systems, they are actively collaborating with DroneDeploy® to provide a drone package which monitors crop health for rapid decision-making (Case IH, 2021f).
Discussion
Current OEM automation technologies enable the majority of features which are required for full machine autonomy. However, they are limited by lack of perception integration options. Major OEMs are developing autonomous concepts which incorporate AI-driven perception systems (namely LiDAR, radar, stereo cameras). Additionally, these perception technologies exist within the OEMs for other uses, such as increasing operator safety or reducing fatigue.
To determine how sophisticated autonomy enabling technologies, such as perception, need to be an operating design domain (ODD) needs to be defined. This ODD specifies what the machine will be tasked for and therefore informs the technical requirements. Currently, there is an opportunity to better define specific applications for autonomy and the ODD with the OEMs. This will allow OEMs to refine the use case and the cost benefit to farmers.
In addition to the technical impediments to autonomy, there are other barriers and, in particular, the safe operation of this equipment and where the various responsibilities exist for the technology users, dealers, service providers and manufacturers. The code of practice for autonomous machinery, led by Grain Producers Australia in collaboration with industry partners , a world first, has been developed to better define these responsibilities (GPA, TMA, SPAA, 2021).
The integration of AI-driven machine perception for navigation by OEMs into their technology stacks and the development of ODDs within these OEMs will likely be the catalyst to allow for the uptake of true on-farm automation. This will, in turn, bring regulatory and legislative attention to on-farm automation technologies and allow for regulatory requirements to begin to be seriously discussed. The Code of Practice has helped put Australian agriculture on the front food in this regard. The gradual implementation of electrification will also inform the pathway to autonomy.
Conclusion
True on-farm autonomous vehicles are limited in use as they are often developed by high-tech start-ups and are targeted towards the specialty or high-value crop markets. OEM driven machine autonomy is limited by lack of AI-driven perception integration options for autonomous navigation and the lack of defined ODDs within these OEMs. With current GPS autosteer, end-of-run sequence automation and communication capabilities, current OEM and third-party options can be integrated to make a machine which can run fully autonomously. However, OEMs put safety mechanisms in place to ensure operator presence as the current machines lack perception integration. These same OEMs are currently developing autonomous concepts but have yet to make them available to consumers, instead focusing on the development of integrated technology stacks, progressively increasing machine automation. Further hurdles to full autonomy include the lack of a direct path to market and the lack of standard regulatory requirements. An AI-driven perception technology, likely developed by an OEM or major third-party provider after the definition of an ODD, which is compatible with current technology stacks, will be the catalyst for the transition from on-farm automation to true autonomy. These developments are beginning to be seen with John Deere’s announcement of a commercial autonomous kit for their 8R series tractor in January 2022 (Vincent, 2022). John Deere have defined the ODD for the specific use case, operating a chisel plow in US broadacre cropping systems and notably connected to the farmer via a smart phone to provide supervision. It is expected that the ODD will be developed as the capabilities of the technology advance and incorporate other use cases, including adoption of that technology in Australia.
Acknowledgements
The research undertaken as part of this project is made possible by the significant contributions of growers through both trial cooperation and the support of the GRDC, the author would like to thank them for their continued support. We would like to acknowledge Sugar Research Australia and the Cotton Research and Development Corporation who have contributed funding to the previous work undertaken by the University of Southern Queensland’s Centre for Agricultural Engineering.
References
Baillie C, Lobsey, C, McCarthy C, Antille D, Thomasson J, Xu Z, Sukkarieh S (2017) Developments in autonomous tractors. Grains Research & Development Corporation
Baillie C, Torrance L, Long D, Pothula A, Brett P, Humpal J (2020) Review of technologies, regulations and operating standards for field based autonomous agricultural machinery. Grains Research & Development Corporation
Burwood-Taylor L (2017) John Deere acquires ‘see & spray’ robotics startup Blue River Technology for $305m. Ag Funder News
Case IH (2021a) Autonomous farming
Case IH (2021b) Guidance and steering
Case IH (2021c) Section & rate control
Case IH (2021d) AFS RowGuideTM
Case IH (2021e) AFS harvest commandTM
Case IH (2021f) Case IH UAV
CLAAS (2018a) Automatic steering systems
CLAAS (2018b) Variable-rate drilling | base fertiliser application | liming
CLAAS (2021) Operator assistance systems
CLAAS Group (2021) Satellite-based GNSS steering systems
GPA, TMA, SPAA (2021) Code of practice: agricultural mobile field machinery with autonomous functions in Australia
Deutz Fahr (2021) SDF guidance
Fendt (2021) Guidance and steering systems
Francis S (2019) John Deere showcases autonomous electric tractor and other new tech.
John Deere (2021a) John Deere acquires Bear Flag Robotics to accelerate autonomous technology on the farm
John Deere (2021b) Future of farming
John Deere (2021c) Guidance
John Deere (2021d) GreenstarTM 2 rate controller
John Deere (2021e) AutotracTM vision
John Deere (2021f) Active fill control
John Deere (2021g) John Deere to acquire majority ownership in Kreisel Electric
Kubota (2021) Kubota smart agriculture
New Holland Agriculture (2021) IntellifillTM system
Raven Precision (2021a) CNH Industrial completes the acquisition of Raven Industries
Raven Precision (2021b) VSN® visual guidance
Raven Precision (2021c) Autoboom® XRT
Raven Precision (2021d) OmnidriveTM by Raven
Raven Precision (2021e) OmnipowerTM by Raven
Real Agriculture (2019) John Deere unveils concept autonomous sprayer
SDF Group (2021) Other technologies
Vincent J (2022) John Deere’s self-driving tractor lets farmers leave the cab — and the field. The Verge
Wilson K (2020) Kubota shows what is to come
Contact details
Dr Jacob Humpal
P9 Handley Street, Darling Heights QLD 4350
0455 220 027
jacob.humpal@usq.edu.au
@caeusq
GRDC Project Code: USQ2006-004RTX,
Was this page helpful?
YOUR FEEDBACK