Normal Assisted Stereo Depth Estimation
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2020
Illustration of results of separate learning and joint learning of depth and normal. While the normal prediction is smooth and accurate, existing state-of-the-art stereo depth prediction result is noisy. Our method improves the prediction quality significantly by joint learning of depth and normal and enforcing consistency
Abstract
Accurate stereo depth estimation plays a critical role in various 3D tasks in both indoor and outdoor environments. Recently, learning-based multi-view stereo methods have demonstrated competitive performance with limited number of views. However, in challenging scenarios, especially when building cross-view correspondences is hard, these methods still cannot produce satisfying results. In this paper, we study how to enforce the consistency between surface normal and depth at training time to improve the performance. We couple the learning of a multi-view normal estimation module and a multi-view depth estimation module. In addition, we propose a novel consistency loss to train an independent consistency module that refines the depths from depth/normal pairs. We find that the joint learning can improve both the prediction of normal and depth, and the accuracy and smoothness can be further improved by enforcing the consistency. Experiments on MVS, SUN3D, RGBD and Scenes11 demonstrate the effectiveness of our method and state-of-the-art performance.
Fast Forward
Resources
Cite Us
@InProceedings{Kusupati_2020_CVPR,
author = {Kusupati, Uday and Cheng, Shuo and Chen, Rui and Su, Hao},
title = {Normal Assisted Stereo Depth Estimation},
booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2020}
}