Orchard mapping and mobile robot localisation using on-board camera and laser scanner data fusion

PhD Thesis


Shalal, Nagham Jamil Dawood. 2016. Orchard mapping and mobile robot localisation using on-board camera and laser scanner data fusion. PhD Thesis Doctor of Philosophy. University of Southern Queensland.
Title

Orchard mapping and mobile robot localisation using on-board camera and laser scanner data fusion

TypePhD Thesis
Authors
AuthorShalal, Nagham Jamil Dawood
SupervisorLow, Dr Tobias
McCarthy, Dr Cheryl
Hancock, Professor Nigel
Institution of OriginUniversity of Southern Queensland
Qualification NameDoctor of Philosophy
Number of Pages213
Year2016
Abstract

Agricultural mobile robots have great potential to effectively implement different agricultural tasks. They can save human labour costs, avoid the need for people having to perform risky operations and increase productivity. Automation and advanced sensing technologies can provide up-to-date information that helps farmers in orchard management. Data collected from on-board sensors on a mobile robot provide information that can help the farmer detect tree or fruit diseases or damage, measure tree canopy volume and monitor fruit development. In orchards, trees are natural landmarks providing suitable cues for mobile robot localisation and navigation as trees are nominally planted in straight and parallel rows.

This thesis presents a novel tree trunk detection algorithm that detects trees and discriminates between trees and non-tree objects in the orchard using a camera and 2D laser scanner data fusion. A local orchard map of the individual trees was developed allowing the mobile robot to navigate to a specific tree in the orchard to perform a specific task such as tree inspection. Furthermore, this thesis presents a localisation algorithm that does not rely on GPS positions and depends only on the on-board sensors of the mobile robot without adding any artificial landmarks, respective tapes or tags to the trees.

The novel tree trunk detection algorithm combined the features extracted from a low cost camera's images and 2D laser scanner data to increase the robustness of the detection. The developed algorithm used a new method to detect the edge points and determine the width of the tree trunks and non-tree objects from the laser scan data. Then a projection of the edge points from the laser scanner coordinates to the image plane was implemented to construct a region of interest with the required features for tree trunk colour and edge detection. The camera images were used
to verify the colour and the parallel edges of the tree trunks and non-tree objects. The algorithm automatically adjusted the colour detection parameters after each test which was shown to increase the detection accuracy. The orchard map was constructed based on tree trunk detection and consisted of the 2D positions of the individual trees and non-tree objects. The map of the individual trees was used as an a priority map for mobile robot localisation. A data fusion algorithm based on an Extended Kalman filter was used for pose estimation of the mobile robot in different paths (midway between rows, close to the rows and moving around trees in the row) and different turns (semi-circle and right angle turns) required for tree inspection tasks. The 2D positions of the individual trees were used in the correction step of the Extended Kalman filter to enhance localisation accuracy.

Experimental tests were conducted in a simulated environment and a real orchard to evaluate the performance of the developed algorithms. The tree trunk detection algorithm was evaluated under two broad illumination conditions (sunny and cloudy). The algorithm was able to detect the tree trunks (regular and thin tree trunks) and discriminate between trees and non-tree objects with a detection accuracy of 97% showing that the fusion of both vision and 2D laser scanner technologies produced robust tree trunk detection. The mapping method successfully localised all the trees and non-tree objects of the tested tree rows in the orchard environment. The mapping results indicated that the constructed map can be reliably used for mobile robot localisation and navigation. The localisation algorithm was evaluated against the logged RTK-GPS positions for different paths and headland turns. The average of the RMS of the position error in x, y coordinates and Euclidean distance were 0.08 m, 0.07 m and 0.103 m respectively, whilst the average of the RMS of the heading error was 3:32°. These results were considered acceptable while driving along the rows and when executing headland turns for the target application of autonomous mobile robot navigation and tree inspection tasks in orchards.

ANZSRC Field of Research 2020400799. Control engineering, mechatronics and robotics not elsewhere classified
300202. Agricultural land management
Byline AffiliationsSchool of Mechanical and Electrical Engineering
Permalink -

https://research.usq.edu.au/item/q38q5/orchard-mapping-and-mobile-robot-localisation-using-on-board-camera-and-laser-scanner-data-fusion

Download files


Published Version
Final thesis version-signed.pdf
File access level: Anyone

  • 1948
    total views
  • 341
    total downloads
  • 1
    views this month
  • 0
    downloads this month

Export as

Related outputs

A review of autonomous navigation systems in agricultural environments
Shalal, N., Low, T., McCarthy, C. and Hancock, N.. 2013. "A review of autonomous navigation systems in agricultural environments." SEAg 2013: Innovative Agricultural Technologies for a Sustainable Future. Barton, Australia 22 - 25 Sep 2013
Orchard mapping and mobile robot localisation using on-board camera and laser scanner data fusion – Part B: mapping and localisation
Shalal, Nagham, Low, Tobias, McCarthy, Cheryl and Hancock, Nigel. 2015. "Orchard mapping and mobile robot localisation using on-board camera and laser scanner data fusion – Part B: mapping and localisation." Computers and Electronics in Agriculture. 119, pp. 267-278. https://doi.org/10.1016/j.compag.2015.09.026
Orchard mapping and mobile robot localisation using on-board camera and laser scanner data fusion – Part A: tree detection
Shalal, Nagham, Low, Tobias, McCarthy, Cheryl and Hancock, Nigel. 2015. "Orchard mapping and mobile robot localisation using on-board camera and laser scanner data fusion – Part A: tree detection." Computers and Electronics in Agriculture. 119, pp. 254-266. https://doi.org/10.1016/j.compag.2015.09.025
A preliminary evaluation of vision and laser sensing for tree trunk detection and orchard mapping
Shalal, Nagham, Low, Tobias, McCarthy, Cheryl and Hancock, Nigel. 2013. "A preliminary evaluation of vision and laser sensing for tree trunk detection and orchard mapping." Katupitiya, Jayantha, Guivant, Jose and Eaton, Ray (ed.) Australasian Conference on Robotics and Automation (ACRA 2013). Sydney, Australia 02 - 04 Dec 2013 Australia.