Object tracking in augmented reality remote access laboratories without fiducial markers

PhD Thesis


Smith, Mark. 2019. Object tracking in augmented reality remote access laboratories without fiducial markers. PhD Thesis Doctor of Philosophy. University of Southern Queensland. https://doi.org/10.26192/XSFE-4A84
Title

Object tracking in augmented reality remote access laboratories without fiducial markers

TypePhD Thesis
Authors
AuthorSmith, Mark
SupervisorKist, Alexander
Maxwell, Andrew
Institution of OriginUniversity of Southern Queensland
Qualification NameDoctor of Philosophy
Number of Pages303
Year2019
Digital Object Identifier (DOI)https://doi.org/10.26192/XSFE-4A84
Abstract

Remote Access Laboratories provide students with access to learning resources without the need to be in-situ (with the assets). The technology endows users with access to physical experiments anywhere and anytime, while also minimising or distributing the cost of operation for expensive laboratory equipment. Augmented Reality is a technology which provides interactive sensory feedback to users. The user experiences reality through a computer-based user interface with additional computer-generated information in the form applicable to the targeted senses.

Recent advances in high definition video capture devices, video screens and mobile computers have driven resurgence in mainstream Augmented Reality technologies. Lower cost and greater processing power of microprocessors and memory place the resources in the hands of developers and users alike, allowing education institutes to invest in technologies that enhance the delivery of course content. This increase in pedagogical resources has already allowed the phenomenon of education at a distance to reach students from a wide range of demographics, improving access and outcomes in multiple disciplines. Incorporating Augmented Reality into Remote Access Laboratories resources has the benefit of improving overall user immersion into the remote experiment, thus improving student engagement and understanding of the delivered material.

Visual implementations of Augmented Reality rely on providing the user with seamless integration of the current environment (through mobile device, desktop PC, or heads up display) with computer generated artificial visual artefacts. Virtual objects must appear in context to the current environment, and respond in a realistic period, or else the user suffers from a disjointed and confusing blend of real and virtual information. Understanding and interacting with the visual scene is controlled through Computer Vision algorithms, and are crucial in ensuring that the AR systems co-operate with the data discovered through the systems.

While Augmented Reality has begun to expand in the educational environment, currently, there is still very little overlap of Augmented Reality technologies with Remote Access Laboratories. This research has investigated Computer Vision models that support Augmented Reality technologies such that live video streams from Remote Laboratories are enhanced by synthetic overlays pertinent to the experiments. Orientation of synthetic visual overlays requires knowledge of key reference points, often performed by fiducial markers. Removing the equipment’s need for fiducial markers and a priori knowledge simplifies and accelerates the uptake and expansion of the technology.

These works uncover hybrid Computer Vision models which require no prior knowledge of the laboratory environment, including no fiducial markers or tags to track important objects and references. Developed models derive all relevant data from the live video stream and require no previous knowledge regarding the configuration of the physical scene. The new image analysis paradigms, (Two-Dimensional Colour Histograms and Neighbourhood Gradient Signature) improve the current state of markerless tracking through the unique attributes discovered within the sequential video frames. Novel methods are also established, with which to assess and measure the performance of Computer Vision models. Objective ground truth images minimise the level of subjective interference in measuring the efficacy of CV edge and corner detectors.

Additionally, locating an effective method to contrast detected attributes associated with an image or object, has provided a means to measure the likelihood of an image match between video frames. In combination with existing material and new contributions, this research demonstrates effective object detection and tracking for Augmented Reality systems within a Remote Access Laboratory environment, with no requirement for fiducial markers, or prior knowledge of the environment. The models that have been proposed in the work can be generalised to be used in any cyber-physical environment that facilitates peripherals such as cameras and other sensors.

Keywordsaugmented reality, remote access laboratories, computer vision, image Processing, Object Tracking, Human Computer Interface
ANZSRC Field of Research 2020469999. Other information and computing sciences not elsewhere classified
Byline AffiliationsSchool of Mechanical and Electrical Engineering
Permalink -

https://research.usq.edu.au/item/q5xx2/object-tracking-in-augmented-reality-remote-access-laboratories-without-fiducial-markers

Download files


Published Version
M Smith 23 Jul 19.pdf
File access level: Anyone

  • 308
    total views
  • 162
    total downloads
  • 3
    views this month
  • 4
    downloads this month

Export as

Related outputs

Objective Construction of Ground Truth Images
Smith, Mark, Maiti, Ananda, Maxwell, Andrew and Kist, Alexander A.. 2021. "Objective Construction of Ground Truth Images." Auer, Michael E. and May, Dominik (ed.) 17th International Conference on Remote Engineering and Virtual Instrumentation (REV 2020). Athens, United States 26 - 28 Feb 2020 Switzerland. Springer. https://doi.org/10.1007/978-3-030-52575-0_24
Colour histogram segmentation for object tracking in remote laboratory environments
Smith, Mark, Maiti, Ananda, Maxwell, Andrew and Kist, Alexander. 2020. "Colour histogram segmentation for object tracking in remote laboratory environments." Auer, Michael E. and Ram B., Kalyan (ed.) 16th International Conference on Remote Engineering and Virtual Instrumentation: Cyber-Physical Systems and Digital Twins (REV 2019). Bangalore, India 03 - 06 Feb 2019 Switzerland. Springer. https://doi.org/10.1007/978-3-030-23162-0_49
Applying Augmented Reality to New or Existing Remote Access Laboratories
Smith, Mark, Maiti, Ananda, Maxwell, Andrew and Kist, Alexander A.. 2019. "Applying Augmented Reality to New or Existing Remote Access Laboratories." Cardoso, Alberto and Restivo, Maria Teresa (ed.) 5th Experiment@International Conference (exp.at'19). Madeira, Portugal 12 - 14 Jun 2019 United States. https://doi.org/10.1109/EXPAT.2019.8876551
Using Unity 3D as the augmented reality framework for remote access laboratories
Smith, Mark, Maiti, Ananda, Maxwell, Andrew D. and Kist, Alexander A.. 2019. "Using Unity 3D as the augmented reality framework for remote access laboratories." Auer, Michael E. and Langmann, Reinhard (ed.) 15th International Conference on Remote Engineering and Virtual Instrumentation (REV 2018): Smart Industry and Smart Education. Duesseldorf, Germany 21 - 23 Mar 2018 Switzerland. Springer. https://doi.org/10.1007/978-3-319-95678-7_64
Object detection resource usage within a remote real-time video stream
Smith, Mark, Maiti, Ananda, Maxwell, Andrew D. and Kist, Alexander A.. 2017. "Object detection resource usage within a remote real-time video stream." Auer, Michael E. and Zutin, Danilo G. (ed.) 14th International Conference on Remote Engineering and Virtual Instrumentation (REV 2017): Online Engineering & Internet of Things:. New York, USA 15 - 17 Mar 2017 Switzerland. https://doi.org/10.1007/978-3-319-64352-6_25
Augmented reality and natural user interface applications for remote laboratories
Maiti, Ananda, Smith, Mark, Maxwell, Andrew D. and Kist, Alexander A.. 2018. "Augmented reality and natural user interface applications for remote laboratories." Auer, Michael E., Azad, Abul K. M., Edwards, Aurther and de Jong, Ton (ed.) Cyber-physical laboratories in engineering and science education. Switzerland. Springer. pp. 79-109
Key aspects of integrating augmented reality tools into peer-to-peer remote laboratory user interfaces
Maiti, Ananda, Kist, Alexander and Smith, Mark. 2016. "Key aspects of integrating augmented reality tools into peer-to-peer remote laboratory user interfaces." 13th International Conference on Remote Engineering and Virtual Instrumentation (REV 2016). Madrid, Spain 24 - 26 Feb 2016 Piscataway, NJ. United States. https://doi.org/10.1109/REV.2016.7444434
Augmented and mixed reality features and tools for remote laboratory experiment
Smith, Mark, Maiti, Ananda, Maxwell, Andrew D. and Kist, Alexander A.. 2016. "Augmented and mixed reality features and tools for remote laboratory experiment." International Journal of Online Engineering. 12 (7), pp. 45-52. https://doi.org/10.3991/ijoe.v12i07.5851