Automatic disparity search range estimation in deep learning stereo

Automatic disparity search range estimation in deep learning stereo

This research concerns the deep stereo networks used for inferring depth from images captured with a stereo pair of cameras. Central to the process is the measurement of disparity between the images, with the computational effort depending on the limit of the 'disparity search range (DSR)' over which the search is to be performed. Like the traditional stereo techniques, deep learning stereo methods also require users to specify the upper bound of DSR, commonly known as the ‘Maximum Disparity’, manually. Selecting a substantially lower or a higher maximum disparity than necessary for a given scene can lead to disparity estimation errors or performance degradations during disparity inference.

This thesis presents an automatic disparity search range estimation technique for deep learning stereo which can be seamlessly embedded into the stereo algorithm itself without requiring pre-processing or explicit configuration by the users. The method incorporates a novel metric referred to as the Sum of New Cost Extrema (SNCE). The stated metric can be estimated on a per-layer basis during the cost volume construction phase of the stereo method. This can serve as the criterion on which to decide whether to continue the cost volume creation process at a given disparity, or to terminate it. In this way, the maximum disparity for a given scene is determined.

The SCNE metric is further utilised in a deep stereo algorithm which produces accurate disparity maps while estimating the disparity search range automatically without user intervention. The memory efficient design of the algorithm makes it possible to optimise memory for standard desktop computers and consumer-grade graphics processing hardware. Evaluations conducted using the benchmark stereo datasets indicate that the algorithm is able to produce accurate results for synthetic and real stereo image sequences without requiring users to set any parameter values during disparity inference, setting a new state-of-the-art benchmark in stereo disparity estimation. Results indicate improvements in performance due to the computational efficiency arising from the optimal cost volume sizes. When used with standard stereo image sequences, the algorithm performed up to 50% faster compared to a reference implementation with a fixed cost volume size. Extended testing with common stereo evaluation metrics on various real-world stereo datasets showed that the algorithm can produce accurate results under varying scene conditions. Additional tests on images captured with a custom-built stereo camera confirmed the generalization capabilities of the algorithm while demonstrating the possibility of achieving complete independence from user specified parameters values when inferring depth from real world stereo imagery.