Extracting 3D pose from video potentially using Neural ODEs

The project is self-contained. The goal of the project is to develop advanced AI assisted tools for artistic game
development. It is anticipated that the advanced modeling of pose based on a mix of 3D motion capture (MOCAP)
data and videos capturing human motion will help create advanced AI assisted game design tools that will reduce
the time and skill required to produce high quality game content. The goal of the project will be achieved by learning
a good representation of human pose in the joint space of monocular video and 3D MOCAP data. Developed tools will serve to create an AI assisted human body
posing system that can use monocular video as an input to assist in pose design. The results will also be applicable
in improving the quality of video mocap data captured via professional cinematic equipment.
We anticipate to use dynamical models on monocular videos and motion
capture data jointly and use implicit dynamical models that
circumvent many of the problems of recurrent methods in modeling temporal dynamics. Challenges and risks
include: the pose estimation quality of state-of-the-art is not sufficient to be used in production systems such as
artistry or animation.

Faculty Supervisor:

Christopher Pal

Student:

Partner:

Unity Technologies

Discipline:

Computer science

Sector:

Professional, scientific and technical services

University:

Université de Montréal

Program: