Scene Graph Image Interpretation Tools

In this project we will look at tools to represent the content of an image and the relationships between its salient objects. The purpose of these tools is not only to enumerate the object represented in an image and identify their surroundings but also to describe how these entities are interacting with each other. We will do so in too ways; first we will look at methods to detect these entities in the image and then parse them into triplets. A triplet is formed of two objects (nodes in the space) and their relationship (a labeled connection in the space). We will make sure that the method that we propose permits to establish the explainability of the results given the inputs. Second, we will look into a method that parse these triplets into sentences that represent the corresponding images captions.

Faculty Supervisor:

Philippe Langlais

Student:

Partner:

Thales Recherche et Technologie

Discipline:

Computer science

Sector:

Management of companies and enterprises; Manufacturing; Professional, scientific and technical services

University:

Université de Montréal

Program: