Differentiable Stereopsis:
Meshes from multiple views using differentiable rendering

Shubham Goel

Georgia Gkioxari

Jitendra Malik

University of California, Berkeley

Facebook AI Research

In CVPR 2022

[Paper]

[Code]

Given few input views with masks and noisy camera poses, Differentiable Stereopsis reconstructs the shape and texture of the underlying object while fixing the camera poses.

We propose Differentiable Stereopsis, a multi-view stereo approach that reconstructs shape and texture from few input views and noisy cameras. We pair traditional stereopsis and modern differentiable rendering to build an end-to-end model which predicts textured 3D meshes of objects with varying topologies and shape. We frame stereopsis as an optimization problem and simultaneously update shape and cameras via simple gradient descent. We run an extensive quantitative analysis and compare to traditional multi-view stereo techniques and state-of-the-art learning based methods. We show compelling reconstructions on challenging real-world scenes and for an abundance of object types with complex shape, topology and texture.

Overview

Approach

Results

Paper

Goel, Gkioxari, Malik.

Differentiable Stereopsis: Meshes from multiple views using differentiable rendering

[pdf] [bibtex]

Acknowledgements

We thank members of the BAIR and FAIR community for helpful discussions. This webpage template was borrowed from some colorful folks.