Leveraging Stacks of Predictors for Efficient Inference and Uncertainty Estimation

Given the ever growing neural networks being developed and the abundant empirical evidence that model/data scale play an important role in enabling high-quality models of data, inference cost becomes a bottleneck to the deployment of state-of-the-art automated predictors. To address that, this research project aims to develop algorithms that can predict outcomes by combining predictions from different layers of a large-scale model. By using layerwise confidence scores, the algorithm can determine if a prediction is accurate enough to output a prediction early. The project will rely on self-ensembling approaches to improve accuracy and confidence scores by voting at different levels of a model stack. This approach can significantly enhance the efficiency of the inference process, particularly for easy examples whose labels can be determined in the initial layers, yielding faster and more accurate predictions, and improved uncertainty estimates.

Faculty Supervisor:

Ioannis Mitliagkas

Student:

Partner:

ServiceNow Canada

Discipline:

Computer science

Sector:

Artificial Intelligence; Information and Communications Technology; Cyber Security

University:

Université de Montréal

Program: