Precision weed detection via colour and depth data fusion in real-time for automatic spot spraying

Precision weed detection via colour and depth data fusion in real-time for automatic spot spraying

Broadacre and row crop farming in Australia uses no-till and minimum-till farming systems which have led to the overuse of specific herbicides for weed control,
causing resistance to those specific herbicides to build-up in weeds. Automatic weed spot spraying can help reduce resistance build-up by specifically detecting weeds for targeted control with alternative herbicides, hence breaking the resistance cycle. Existing commercial weed spot sprayers are only capable of distinguishing green from brown, i.e. plant material in a fallow situation, and image
analysis research for weed discrimination typically was not developed for commercial on-farm conditions. The research in this thesis has developed a real-time, real-world machine vision spot spray system that can operate at groundspeeds up to 20 km/h and discriminate green from green (i.e. weed from crop) under commercial conditions for two very different crop types and size scales, specifically
sugarcane (grass-like) and pyrethrum (broadleaf).

Occlusion of a weed leaf by another leaf or plant is a major impediment for real-world operation of a machine vision weed spot sprayer. A Depth Colour Segmentation Algorithm (DCSA) has been developed which combines depth data and colour image data to segment individual leaves from each other, based on pixel connectedness in height and colour, providing an accuracy when occluded of greater than 99%. The DCSA has a �filtering capability that can reduce the amount of data requiring further analysis by an observed 83% for sugarcane and 53% for pyrethrum.

Existing feature extraction techniques have been evaluated in the thesis and have been shown to be unsatisfactory in discriminating weed from crop especially when the weed and crop are similar species, e.g. grass-like weed (guinea grass) from grass-like crop (sugarcane). Depth features were added to the extracted features of a local binary pattern function, improving the accuracy from 63% to 90% for pyrethrum identification, and showing that depth data combined with 2D data can improve the discrimination result. Additional real-world custom algorithms have been
developed to achieve an identification accuracy of 87% (where 86% of the weed was occluded) with a 3.5% false positive rate for sugarcane. The Depth, Colour, Size and Spatial (DCSS) algorithm developed for pyrethrum achieved
98% accuracy for pyrethrum identification with a 1.2% false positive rate.

Real-time functionality has been obtained by the development of a Synchronised Parallel Processing
(SPP) technique. The SPP technique maintains a high frame rate (which determines the maximum groundspeed) by assigning the workload in a permanently allocated pipeline synchronised by the incoming video image. Calculations for sugarcane and pyrethrum show that speeds up to 18.5 and 17.2 km/h respectively are achievable based on the algorithms developed and a higher core count CPU
(six cores were used in the calculation) would achieve higher groundspeeds. The gains from the additional processing availability provided by SPP can be used to achieve a higher groundspeed, or undertake additional image analysis, if required.

It is concluded that the machine vision components developed in this thesis comprise a real-time,
real-world machine vision spot sprayer that can operate at commercial groundspeeds up to 20 km/h and discriminate weed from crop.