Using image analysis to map snails

Key Messages

• Image analysis of field photographs taken with mobile phones attached to agricultural equipment has the potential to be used to economically make maps of snail distribution and density in a paddock.
• This data can be used to make snail management decisions that reduce their economic impact.
• Machine learning deep neural networks were used to produce models that would quickly detect snails in images.

Aims

Snail control is estimated to cost the Western Australia (WA) grains industry $1.86 million dollars per annum. If these pests are not controlled, then damage is estimated to cost $19.25 million dollars in lost yield and quality in grain crops (Murray et al, 2013). In WA, snails are mainly an issue in southern grain growing regions. The three main pest species are the small pointed snail (Prietocella barbara), white Italian snail (Theba pisana) and the vineyard snail (Cernuella virgata). These were introduced from Europe and are now present along the south coast of WA with the small pointed snails having a wider distribution than any other mollusc pest in WA. This species was used in this research.

These snail species feed on emerging crops causing retardation of crops or seedling losses, especially in canola, but can also be harvested with the grain causing contamination and loads can be rejected or downgraded if snail numbers are above receival standards (CBH, 2017). Removal of snails from harvested grain is problematic if the snail is the same size as a single grain. Consequently, to avoid possible contamination issues and crop loss, snail control is recommended to occur before crop germination.

The most common control method employed by growers for snail control is the application of baits at a single rate over the entire paddock (McDonald et al, 2017). The efficacy of baits on snails depends on the rate that is applied, the evenness of distribution and whether the snails are mobile and actively feeding (McDonald et al, 2017). However, a single bait application may not provide adequate control and a follow up bait application is usually required.

If a map could be produced showing the density and distribution of snails across a paddock, growers could target their controls more effectively. This could be done by using variable rate technologies to apply higher densities of bait to sections of the paddock in which snails are greatest in number. It is estimated that using variable rates to apply baits, rather than a single rate across a paddock could make savings of more than $10 per hectare (Moore et al, 2018).

The location of snails is an important factor when determining how to map them. When actively moving, small pointed snails are more likely to be found on the ground and can be more easily be detected with vehicle mounted cameras when moving across a paddock.

The methodology employed to map the snails was shaped by growers who attended meetings held by the Regional Cropping Solutions Network in the Albany and Esperance Port Zones in 2017. These growers reported that they wanted inexpensive technologies for snail control and technologies that could be adapted for their farming system. For this reason, the focus of this research was to produce maps from visual images taken of snails on the ground using readily available equipment.

Method

Image capture

An app (SnapMaps) was produced that would record the GPS location and trigger a mobile phone camera when it was pointing towards the ground. This allowed the phone to be mounted onto any wheel so that it would take a photo with each revolution without motion blur. SnapMaps together with the smartphone and mounting on the wheel was called the WheelCam.

The WheelCam was used to capture images and data when attached to an autonomous rover that was set on a predefined path using Mission Planner. Mission Planner is open source software (http://ardupilot.org/planner/) used mainly for controlling drones. In Mission Planner the area of interest is outlined using outside waypoints and then internal waypoints are generated within the area at set distances apart. The Gairdner site was in a snail infested barley crop that the Department of Primary Industries and Regional Development (DPIRD) was monitoring. A two hectare, sub site was selected after harvest and tracks at 10m row spacings were generated with Mission Planner (Figure 1). This information was then sent to the autonomous rover using a radio link and the rover followed this path using on board autonomous steering. In other situations, not reported here, the WheelCam is attached to an agricultural vehicle like a harvester, sprayer or cultivator and the grid is determined by the work pattern (Moore et al. 2018).

Snail image analysis

After image capture is complete, images are downloaded from the smart phone for processing. In this case they were used to determine snail prevalence.

Data collected at Gairdner was analysed manually initially. Snails in these images were counted and a kriged map of their distribution was produced using AgLeader Spatial Management Software (SMS) (Figure 2).

Images for two training sets (local and general) were extracted; the first (local) contained 70 images from the 611 images from Gairdner which were taken under good lighting conditions during the middle of the day. The second (general) training set was taken from 646 images from all sites and included a wider range of situations including 6 sites and laboratory images, at all times of the day and night and over many months. This was used to determine if a more general model would provide similar precision to the local model derived from images from the Gairdner site only.

LabelImg (Tzutalin 2015) was used to manually record where snails were positioned in the training images. This graphical image annotation tool creates a companion xml file for each image with the bounding boxes that contain each selected snail in the image. It was written in Python and uses Qt for its graphical interface. Annotations are saved as XML files in PASCAL VOC format. This format is used by ImageNet and is suitable for image analysis using TensorFlow.

These training images were forwarded to MapIzy Pty Ltd who used machine learning with a deep neural network with 16 layers(YOLO) and TensorFlow with keras to detect small pointed snails using the first and then the second training set (Figure 3). It was then re analysed with a more complex model called YOLO3 (Figure 4).

The location data for each image was extracted using a Python program to retrieve the latitude and longitude from the Exif data attached to each image. This location data was matched with the number of snails detected by the image analysis and then kriged and mapped using AgLeader SMS. This provided a map of the location and density of the snails detected in the area sampled.

This was validated by comparing the number of snails in each image that were manually counted with those that were predicted by image analysis. This was done for the first 150 images of the 611 image set. This data was also statistically analysed using regression analysis of the predicted number of snails compared to the number manually counted for the whole dataset. Genstat was used to determine the regression equations and correlation between the various estimates of snail numbers.

Results

At Gairdner, the WheelCam allowed the smart phone to capture 611 usable images while it was attached to the autonomous rover. The locations of the images were transferred to Google maps to help the growers visually orient the area mapped and help them transfer the locations to equipment control programs (Figure 1).

Manual analysis of photos showed the highest number of snails in an image was 21 and the average number of snails per image was 0.87 which calibrated to 3.5 snails/m². The SMS was used to produce a contoured snail density map (Figure 2). This could be used to generate a snail bait prescription map.

Figure 1. WheelCam photo locations and Grid locations at Gairdner River super imposed on a Google Maps image of the area.

Figure 2 shows manual counts of snails with areas of high densities greater than 20 snails/m² colored red and low densities colored green. Approximately one quarter of the area surveyed had a snail density greater than the threshold for canola of 20 snails/m² (Moore and Moore, 2018).

Figure 2. Contoured snail density map created by kriging of manual snail counts and using an automatic variogram at a three meter grid at Gairdner

The number of snails detected by image analysis using the initial model with local data was a poor fit and discarded. This could be caused by the model simply remembering the training data which limits it usefulness for predicting the targets in new situations.

The initial model with the general training data was then applied to the Gairdner image set and the variogram is shown in Figure 3. This was a better visual fit but was still not that representative of the manual counts in Figure 2.

Figure 3. Contoured snail density map created by kriging of snail densities detected with the image analysis with the initial model with general training data and using an automatic variogram at a three meter grid at Gairdner.

The regression equation between the number of snails manually counted and those detected with the initial model for image analysis was:

No. of snails counted = 0.48 * No. of snails predicted by the initial model (r² = 0.28) (P>0.001).

This indicates that the model predicted about twice the number of snails that were actually counted, i.e. there were a lot of false positives.

When the complex model was applied with general training data the variogram in Figure 3 was generated. This was visually more similar to the manual variogram than the previous model.

Figure 4. Contoured snail density map created by kriging of snail densities detected with the image analysis with the complex model with general training data and using an automatic variogram at a three meter grid at Gairdner.

The regression equation between the number of snails manually counted and those detected with the new complex model for image analysis was:

No. of snails counted = 0.80 * No. of snails predicted by the complex model (r² = 0.41) (P>0.001).

The new complex model gave better overall results both visually and by the regression coefficient of 0.41. There was a lower number of false detections indicated by the higher coefficient of 0.80 in the regression equation.

Conclusion

Hardware and associated systems to create high resolution digital images are comparatively inexpensive and create the opportunity to collect paddock scale data on snail populations. The WheelCam effectively collected this geospatially referenced, high resolution digital image data for analysis and mapping. The analysis of the images using computer algorithms to detect snails was used to map the distribution and density of snails.

When applied in the field, this approach quantified significant variation in density and distribution of small pointed snails, when images from a 2ha section of snail-infested cropping paddock were assessed. The resulting map of variation and pattern of snail density could readily justify modifying bait programs to incorporate variable application rate.  Theoretical calculations suggest this would enhance the economic benefit by both improving the level of control and reducing the overall treatment cost. For instance, in Figure 2 approximately a quarter of the area surveyed has a snail infestation that is likely to cause damage to canola seedlings. If the grower was using metaldehyde baits that cost $1.54 per kg (Moore & Moore, 2018) and the paddock was treated with the minimum label rate of five kg per hectare then the total cost for the two hectares would be $15.40. If the grower used the mapping information and the snail density threshold and applied the maximum label rate of 7.5 kg/ha on the 0.5ha that was heavily infested, then both better snail control and a saving of $4.80 per hectare would be expected. This assumes that there is no residual benefit from baiting areas that are less than threshold values, an assumption which should be tested.

Alternatively, the use of maps for snail distribution could be used to avoid heavily infested areas during harvest to reduce the risk of grain contamination or alternative crops that are more tolerant of snails could be planted in infested areas.

A current impediment to the practical application of paddock scale snail mapping is the precision with which machine learning algorithms can routinely interpret image data that reliably predict snail numbers. When computer analysis of the images was used to detect snails and map their distribution, the accuracy depended on both the training data used and the complexity of the neural network image analysis model.  Initial results using the present methodology show potential for this technology’s routine adaptation to snail baiting in commercial grain farming, but improvements are required to determine the optimal combination of training data and to model various situations.

Acknowledgments

This work was supported by funding from Boosting Grains R&D through the Royalties for Regions program as an initiative of DPIRD on behalf of the State Government of Western Australia.

Much of the computer code is derived from the generosity of open source programmers and organisations.

Growers have provided valuable input and the use of their land.

The participation of Ms Alice Butler is in conjunction with project DAW00256 Building crop protection and production agronomy R&D capacity in regional Western Australia, with the support of the Grains Research and Development Corporation.

Project Numbers:

88801589, CT-47. Boosting grains flagship project- Improving efficiencies of slug and snail control.

DAW00256 Building crop protection and production agronomy R&D capacity in regional Western Australia

Reviewed by Rob Loughman

References

CBH, 2017: Retrieved January 17, 2018, from Grains Industry Western Australia (GIWA): 2017 2018 CBH Public Receival Standards WA as at October 2017.pdf

McDonald, K. A., Butler, S. Micic, 2017: Snail Management Guide for WA Farmers. Albany: Stirlings to Coast Farmers.

Moore J, Micic S, Babativa Rodriguez C, Butler A, Skinner G (2018) Image analysis of slugs and snails in broadacre agriculture. GRDC 2018 Grains Research Updates.

Moore C.B., & Moore J.H. (2018) HerbiGuide - The Pesticide Expert on a Disk. Version. 32.0, (PO Box 44, Albany, Western Australia, 6331, HerbiGuide).

Murray D, Clarke M, D Ronning (2013) The current and potential costs of invertebrate pests in grain crops. Publication AEP001. Grains Research and Development Corporation, Canberra, ACT, Australia.

Tzutalin 2015: LabelImg. GitHub