Enhanced & Editable Deep Fakes

Traditional Computer Generated Imagery (CGI) splits the process into separate stages: we model geometry; apply textures and define surface properties; animate motion; and apply lighting; before combining the elements when rendering imagery. Deep-learning methods such as Deep Fakes, or NeRF train a network to combine many of these steps into a single network, and are able to produce convincing results in this way. By combining these different aspects of image generation into a single unit, these networks also make it challenging for an artist to edit the properties of the network in an intuitive way. This is because the underlying representation, which is typically a latent vector, doesn’t have a simple connection with the high-level controls an artist expects. We wish to explore ways of giving our artists intuitive ways to control neural rendering. We also want to explore ways of enhancing our results by adding additional inputs to the process.

Faculty Supervisor:

David Lindell

Student:

Partner:

DNEG

Discipline:

Computer science

Sector:

Information and cultural industries

University:

University of Toronto

Program: