NeuralUDF: Learning Unsigned Distance Fields
for Multi-view Reconstruction of Surfaces with Arbitrary Topologies

Abstract

We present a novel method, called NeuralUDF, for reconstructing surfaces with arbitrary topologies from 2D images via volume rendering. Recent advances in neural rendering based reconstruction have achieved compelling results. However, these methods are limited to objects with closed surfaces since they adopt Signed Distance Function (SDF) as surface representation which requires the target shape to be divided into inside and outside. In this paper, we propose to represent surfaces as the Unsigned Distance Function (UDF) and develop a new volume rendering scheme to learn the neural UDF representation. Specifically, a new density function that correlates the property of UDF with the volume rendering scheme is introduced for robust optimization of the UDF fields. Experiments on the DTU and DeepFashion3D datasets show that our method not only enables high-quality reconstruction of non-closed shapes with complex typologies, but also achieves comparable performance to the SDF based methods on the reconstruction of closed surfaces.

Comparisons on objects with open surfaces

To validate the ability of NeuralUDF to model open surfaces, we conduct comparisons with current SDF based methods on DeepFashion3D dataset. As you can see, the SDF based methods struggle to reconstruct the objects with open surfaces, and model such objects with closed surfaces, thus leading to erroneous geometries. In contrast, due to the flexibility of UDF representation, our method successfully model the open surfaces.

Comparisons on objects with closed surfaces

Although our model is only trained on DTU, our model still generalizes well to unseen scenes of BlendedMVS dataset.

More results

Citation

		@inproceedings{long2023neuraludf,
		  title={Neuraludf: Learning unsigned distance fields for multi-view reconstruction of surfaces with arbitrary topologies},
		  author={Long, Xiaoxiao and Lin, Cheng and Liu, Lingjie and Liu, Yuan and Wang, Peng and Theobalt, Christian and Komura, Taku and Wang, Wenping},
		  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
		  pages={20834--20843},
		  year={2023}
		}