SampleNet: Differentiable Point Cloud Sampling

In this episode of the Talking Papers Podcast, I hosted Itai Lang to chat about his paper “SampleNet: Differentiable Point Cloud Sampling”, published in CVPR 2020.

In this paper, they propose a point soft-projection to allow differentiating through the sampling operation and enable learning task-specific point sampling. Combined with their regularization and task-specific losses, they can reduce the number of points to 3% of the original samples with a very low impact on task performance.

I met Itai for the first time at CVPR 2019. Being a point-cloud guy myself, I have been following his research work ever since. It is amazing how much progress he has made and I can’t wait to see what he comes up with next. It was a pleasure hosting him on the podcast.

AUTHORS

Itai Lang, Asaf Manor, Shai Avidan

 

ABSTRACT

There is a growing number of tasks that work directly on point clouds. As the size of the point cloud grows, so do the computational demands of these tasks. A possible solution is to sample the point cloud first. Classic sampling approaches, such as farthest point sampling (FPS), do not consider the downstream task. A recent work showed that learning a task-specific sampling can improve results significantly. However, the proposed technique did not deal with the non-differentiability of the sampling operation and offered a workaround instead. We introduce a novel differentiable relaxation for point cloud sampling that approximates sampled points as a mixture of points in the primary input cloud. Our approximation scheme leads to consistently good results on classification and geometry reconstruction applications. We also show that the proposed sampling method can be used as a front to a point cloud registration network. This is a challenging task since sampling must be consistent across two different point clouds for a shared downstream task. In all cases, our approach outperforms existing non-learned and learned sampling alternatives. Our code is publicly available.

 

📚 Learning to Sample

📚 Farthest Point Sampling

LINKS AND RESOURCES

💻Code

📚 Paper

To stay up to date with Itai’s latest research, follow him on:

🎓Google Scholar

🐦Twitter

Recorded on February 15th 2022.


CONTACT

If you would like to be a guest, sponsor or just share your thoughts, feel free to reach out via email: talking.papers.podcast@gmail.com

SUBSCRIBE AND FOLLOW

🎧Subscribe on your favourite podcast app: https://talking.papers.podcast.itzikbs.com

📧Subscribe to our mailing list: http://eepurl.com/hRznqb

🐦Follow us on Twitter: https://twitter.com/talking_papers

🎥YouTube Channel: https://bit.ly/3eQOgwP