Error-resilient multi-view video plus depth based 3-D video coding

PhD Thesis

Gao, Pan. 2016. Error-resilient multi-view video plus depth based 3-D video coding . PhD Thesis Doctor of Philosophy. University of Southern Queensland.

Error-resilient multi-view video plus depth based 3-D video coding

TypePhD Thesis
AuthorGao, Pan
SupervisorXiang, Wei
Institution of OriginUniversity of Southern Queensland
Qualification NameDoctor of Philosophy
Number of Pages188

Three Dimensional (3-D) video, by definition, is a collection of signals that can provide depth perception of a 3-D scene. With the development of 3-D display
technologies and interactive multimedia systems, 3-D video has attracted significant interest from both industries and academia with a variety of applications. In order to provide desired services in various 3-D video applications, the multiview video plus depth (MVD) representation, which can facilitate the generation of virtual views, has been determined to be the best format for 3-D video data.

Similar to 2-D video, compressed 3-D video is highly sensitive to transmission errors due to errors propagated from the current frame to the future predicted frames. Moreover, since the virtual views required for auto-stereoscopic displays are rendered from the compressed texture videos and depth maps, transmission
errors of the distorted texture videos and depth maps can be further propagated to the virtual views. Besides, the distortions in texture and depth show different
effects on the rendering views. Therefore, compared to the reliability of the transmission of the 2-D video, error-resilient texture video and depth map coding
are facing major new challenges.

This research concentrates on improving the error resilience performance of MVD-based 3-D video in packet loss scenarios. Based on the analysis of the propagating behaviour of transmission errors, a Wyner-Ziv (WZ)-based error-resilient algorithm is first designed for coding of the multi-view video data or depth data. In this scheme, an auxiliary redundant stream encoded according to WZ principle
is employed to protect a primary stream encoded with standard multi-view video coding codec. Then, considering the fact that different combinations of texture and depth coding mode will exhibit varying robustness to transmission errors, a rate-distortion optimized mode switching scheme is proposed to strike the optimal trade-off between robustness and compression effciency. In this approach,
the texture and depth modes are jointly optimized by minimizing the overall distortion of both the coded and synthesized views subject to a given bit rate. Finally, this study extends the research on the reliable transmission of view synthesis prediction (VSP)-based 3-D video. In order to mitigate the prediction position error caused by packet losses in the depth map, a novel disparity vector correction algorithm is developed, where the corrected disparity vector is calculated from the depth error. To facilitate decoder error concealment, the depth
error is recursively estimated at the decoder.

The contributions of this dissertation are multifold. First, the proposed WZbased error-resilient algorithm can accurately characterize the effect of transmission
error on multi-view distortion at the transform domain in consideration of both temporal and inter-view error propagation, and based on the estimated distortion,
this algorithm can perform optimal WZ bit allocation at the encoder through explicitly developing a sophisticated rate allocation strategy. This proposed algorithm is able to provide a finer granularity in performing rate adaptivity
and unequal error protection for multi-view data, not only at the frame level, but also at the bit-plane level. Secondly, in the proposed mode switching scheme, a
new analytic model is formulated to optimally estimate the view synthesis distortion due to packet losses, in which the compound impact of the transmission distortions of both the texture video and the depth map on the quality of the
synthesized view is mathematically analysed. The accuracy of this view synthesis distortion model is demonstrated via simulation results and, further, the estimated distortion is integrated into a rate-distortion framework for optimal
mode switching to achieve substantial performance gains over state-of-the-art algorithms. Last, but not least, this dissertation provides a preliminary investigation
of VSP-based 3-D video over unreliable channel. In the proposed disparity vector correction algorithm, the pixel-level depth map error can be precisely estimated at the decoder without the deterministic knowledge of the error-free reconstructed depth. The approximation of the innovation term involved in depth error estimation is proved theoretically. This algorithm is very useful to conceal
the position-erroneous pixels whose disparity vectors are correctly received.

Keywords3-D video; multi-view video
ANZSRC Field of Research 2020400899. Electrical engineering not elsewhere classified
Byline AffiliationsSchool of Mechanical and Electrical Engineering
Permalink -

Download files

Published Version
File access level: Anyone

  • 1532
    total views
  • 175
    total downloads
  • 2
    views this month
  • 2
    downloads this month

Export as

Related outputs

Error-resilient multi-view video coding for next generation 3-D video broadcasting
Gao, Pan, Xiang, Wei, Billingsley, John and Zhang, Yongbing. 2013. "Error-resilient multi-view video coding for next generation 3-D video broadcasting." 4th International Conference on Information and Communication Technology Convergence (ICTC 2013): Future Creative Convergence Technologies for New ICT Ecosystems. Jeju Island, South Korea 14 - 16 Oct 2013 United States.
A rate–distortion optimized error-resilient algorithm for multi-view video coding
Gao, Pan, Xiang, Wei, Peng, Qiang and Billingsley, John. 2016. "A rate–distortion optimized error-resilient algorithm for multi-view video coding." Circuits, Systems and Signal Processing. 35 (1), pp. 301-323.
Disparity Vector Correction for View Synthesis Prediction-Based 3-D Video Transmission
Gao, Pan and Xiang, Wei. 2015. "Disparity Vector Correction for View Synthesis Prediction-Based 3-D Video Transmission." IEEE Transactions on Multimedia. 17 (8), pp. 1153-1165.
Rate-distortion optimized mode switching for error-resilient multi-view video plus depth based 3-D video coding
Gao, Pan and Xiang, Wei. 2014. "Rate-distortion optimized mode switching for error-resilient multi-view video plus depth based 3-D video coding." IEEE Transactions on Multimedia. 16 (7), pp. 1797-1808.
Error-resilient multi-view video coding using Wyner-Ziv techniques
Gao, Pan, Peng, Qiang and Xiang, Wei. 2015. "Error-resilient multi-view video coding using Wyner-Ziv techniques." Multimedia Tools and Applications. 74 (18), pp. 7957-7982.