Abstract

We propose a method for 3D point cloud surface fitting. This method, called DeepFit, incorporates a neural network to learn point-wise weights for weighted least squares polynomial surface fitting. The learned weights act as a soft selection for the neighborhood of surface points thus avoiding the scale selection required of previous methods. To train the network we propose a novel surface consistency loss that improves point weight estimation. The method enables extracting normal vectors and other geometrical properties, such as principal curvatures, the latter were not presented as ground truth during training. We achieve state-of-the-art results on a benchmark normal and curvature estimation dataset, demonstrate robustness to noise, outliers and density variations, and show its application on noise removal.

Cite

arXiv (preprint):

@article{ben2020deepfit,
  title={DeepFit: 3D Surface Fitting via Neural Network Weighted Least Squares},
  author={Ben-Shabat, Yizhak and Gould, Stephen},
  journal={arXiv preprint arXiv:2003.10826},
  year={2020}
}

ECCV 2020 (oral) (will update once paper is published)

Video

Important links: [preprint] [short video] [full video] [code]

Surface fitting – additional related content

If you found DeepFit interesting, you may also like my Nesti-Net paper on normal estimation for 3D point clouds that tackles a similar problem but instead of selecting points, it chooses the whole neighbourhood radius. Another relevant paper is 3DmFV paper on deep learning on 3D point clouds which tackles the first challenge in the DeepFit’s surface fitting pipeline which is applying deep learning on point clouds. A few years ago I made a summary of 3D point cloud classification methods. In this work, we chose to use PointNet, however, this block can be easily replaced with 3DmFV-Net (we actually tested it and got similar results), PointNet++, kd-network or any newer architecture. This obviously introduces a tradeoff between efficiency and accuracy which is inherent to all of these methods.

To stay up to date with my latest research: Follow me on twitter and subscribe to my YouTube channel.