Inverse graphics network for 3D building reconstruction using a GAN-based approach

The ability to extract 3D models from 2D images, a process called Inverse Graphics, finds applications in the many applications of Virtual Reality. For the development of urban digital twins, we focus our research on Inverse Graphics of buildings, which would greatly simplify the reconstruction of entire city blocks in a virtual environment. We plan to investigate a Generative Adversarial Network (GAN) and Differentiable Renderer based approach. We first plan to train a building image GAN which would allow for the creation of a synthetic self-supervised dataset. This dataset would then be used to train the Differentiable Renderer, which is responsible to 3D reconstruction. We expect the product to generate 3D mesh and texture from 2D images taken from phones or other digital cameras.

Faculty Supervisor:

Jonathan Li

Student:

Partner:

Presagis Canada Inc

Discipline:

Computer science

Sector:

Information and cultural industries; Professional, scientific and technical services

University:

University of Waterloo

Program: