ECCV 2022 Tutorial on

Implicit Rendering for Novel View Synthesis using Implicitron and PyTorch3D

Location: Israel D (120)

Monday, October 24 (morning), 2022


In this tutorial, we propose a quick survey of the latest Novel View Synthesis Techniques for single-scene and multiple-scene use cases, and then introduce a new framework, Implicitron, directly available within the widely used open-source PyTorch3D library. Implicitron provides abstractions and implementations of common implicit representations and rendering components to allow for easy experimentation. Implicitron makes it easy to evaluate variations and combinations of novel view synthesis methods within a common codebase, without requiring significant expertise in either 3D computer vision or graphics, and often even without the need to change actual code!


09:15 - 09:30 Welcome and Tutorial Kick off - Nikhila Ravi

09:30 - 10:30 Introduction to novel view synthesis - - David Novotny

Review of existing methods for novel view synthesis that overfit to one scene (such as NeRF, SRN, IDR), and those that learn category priors and generate new views conditioned on one or few images (such as PixelNeRF, PiFu, NerFormer). A shared high-level terminology is introduced to describe the components of all these approaches. We will include a brief description of recent research projects at FAIR (e.g. Common Objects in 3D) which have been enabled by Implicitron.

10:30 - 10:45 ☕ Coffee Break

10:45 - 10:55 Overview of the PyTorch3D framework - Nikhila Ravi

Brief intro to the PyTorch3D framework.

11:00 - 11:55 Overview of the Implicitron framework and Demo - Jeremy Reizenstein

We go through the different components that make up a dataset and a model in Implicitron, matching these to the introduced terminology, and how to combine them together and train a new view synthesis model.

Links to Colab notebooks:
1. Data Loading
2. Rendering
3. Intro to Config System
4. Voxel Grid Training

12:00 - 12:30 Implicitron Demo: Render a radiance field - - Roman Shapovalov

In this practical session, we will show how to use the Implicitron renderer API and related abstractions. The audience will be encouraged to run the code live during the session.

12:30 - 12:40 Closing - Nikhila Ravi


Contact: Jeremy Reizenstein