Multi-view Neural Scene Inpainting

Capturing and manipulating 3D scenes is a topic of great interest with applications in photography, augmented reality, and robotics. Neural Radiance Fields, or NeRFs, is an emerging technology that enables high fidelity scene modeling and manipulation. This is of great interest to Samsung, as it can help create new photographic experiences for users. Our project focuses on a particular type of manipulation – object inpainting. This involves two components: segmenting the object from the scene with simple user interaction and inpainting the scene after object removal. Unlike existing NeRF manipulation work, we propose to train both components on large offline datasets of scenes and objects, enabling efficient and robust processing at test time without having to perform such learning on the fly. The intern will work in collaboration with other SAIC-Toronto staff and focus on the segmentation component of the project.

Faculty Supervisor:

Igor Gilitschenski

Student:

Partner:

Samsung Electronics Canada

Discipline:

Computer science

Sector:

Technology; Information and Communications Technology; New and Digital Media

University:

University of Toronto

Program: