Evaluating Latency in Virtual Production Pipelines with Integrated Prediction Model for Motion Capture Data

Virtual Production (VP) is seeing a dramatic spike in interest and adaptation as the global film industry, particularly Hollywood, has been shutdown due to Covid-19. Virtual production is a broad term referring to a spectrum of computer-aided production and visualization filmmaking methods, and is also being used for broader applications from animation to industrial visualization. Of particular interest is the novel application of virtual production for live theatrical performance where machine learning is driving digital puppeteering from real-time motion capture for innovative and compelling storytelling. Machine learning for real-time motion mapping predictions is a key component of the live performance virtual production pipeline. However, serving predictions from trained machine learning models is emerging as a dominant challenge in production machine learning. These computationally intensive prediction pipelines must run continuously with a tight latency budget and in response to stochastic and often bursty query arrival processes. In this proposal research project, the intern(s) will review the latest applications and research related to latency issues in virtual production pipelines, with a focus on machine learning prediction.

Faculty Supervisor:

Jiannan Wang

Student:

Zhou Jessie Cen;Sumukha Bharadwaj Balasubramanya

Partner:

AMPD

Discipline:

Computer science

Sector:

Professional, scientific and technical services

University:

Simon Fraser University