標題: MVSNet plus plus : Learning Depth-Based Attention Pyramid Features for Multi-View Stereo
作者: Chen, Po-Heng
Yang, Hsiao-Chien
Chen, Kuan-Wen
Chen, Yong-Sheng
資訊工程學系
電子工程學系及電子研究所
Department of Computer Science
Department of Electronics Engineering and Institute of Electronics
關鍵字: Feature extraction;Three-dimensional displays;Solid modeling;Computational modeling;Image reconstruction;Estimation;Training;Multi-view stereo;deep learning;3D model reconstruction;feature aggregation;plane sweep algorithm
公開日期: 1-Jan-2020
摘要: The goal of Multi-View Stereo (MVS) is to reconstruct 3D point-cloud model from multiple views. On the basis of the considerable progress of deep learning, an increasing amount of research has moved from traditional MVS methods to learning-based ones. However, two issues remain unsolved in the existing state-of-the-art methods: (1) only high-level information is considered for depth estimation. This may reduce the localization accuracy of 3D points as the learned model lacks spatial information; and (2) most of the methods require additional post-processing or network refinement to generate a smooth 3D model. This significantly increases the number of model parameters or the computational complexity. To this end, we propose MVSNet++, an end-to-end trainable network for dense depth estimation. Such an estimated depth map can further be applied to 3D model reconstruction. Different from previous methods, in the proposed method, we first adopt feature pyramid structures for both feature extraction and cost volume regularization. This can lead to accurate 3D point localization by fusing multi-level information. To generate smooth depth map, we then carefully integrate instance normalization into MVSNet++ without increasing model parameters and computational burden. Furthermore, we additionally design three loss functions and integrate Curriculum Learning framework into the training process, which can lead to an accurate reconstruction of 3D model. MVSNet++ is evaluated on DTU and Tanks & Temples benchmarks with comprehensive ablation studies. Experimental results demonstrate that our proposed method performs favorably against previous state-of-the-art methods, showing the accuracy and effectiveness of the proposed MVSNet++.
URI: http://dx.doi.org/10.1109/TIP.2020.3000611
http://hdl.handle.net/11536/155205
ISSN: 1057-7149
DOI: 10.1109/TIP.2020.3000611
期刊: IEEE TRANSACTIONS ON IMAGE PROCESSING
Volume: 29
起始頁: 7261
結束頁: 7273
Appears in Collections:Articles