Diffusion Language Models for Conditional Code Generation

Have you heard about DALL·E 2, Midjourney or Stable Diffusion? Or perhaps you were fooled by the picture of the pope in a puffy white coat, or by some other deepfake memes involving stars and/or politicians? Well, good or bad, these feats are made possible by a technology called generative diffusion models. Although they are mostly known for generating visually stunning images, these models can basically learn to generate any kind of document, as long as we have enough such documents to teach them with. It turns out that there is a great deal of computer code on the internet, and that the BigCode project has already done the hard work of curating it in a responsible manner. We will thus train diffusion models that can write computer code, taking advantages of special properties of diffusion models to coerce them in doing what we want them to do.

Faculty Supervisor:

Tal Arbel

Student:

Partner:

ServiceNow Canada

Discipline:

Computer science

Sector:

Professional, scientific and technical services

University:

McGill University; Université de Montréal

Program: