Bridging Symbolic and Audio Spaces for Interactive Music Generation

This project explores a new workflow for creating adaptive music for video games and other interactive media. Today, most interactive music relies on pre-recorded audio segments, which limits flexibility and requires composers to manually prepare a large number of variations and transitions. Our research takes a different approach by working with MIDI, a digital data format that represents musical notes and instruments rather than audio recordings. This makes it easier to control, transform, and generate music using artificial intelligence and facilitates musical interpretation.. The project will develop a complete pipeline: first converting audio into MIDI using transcription models, then using a transformer-based AI model to generate smooth transitions or variations between musical segments, and finally synthesizing the result back into expressive, high-quality audio. By doing so, we aim to give developers and composers more tools to create rich, seamless, and responsive soundtracks without requiring advanced musical knowledge. For the partner organization, this research will strengthen their software by adding cutting-edge generative capabilities, making it more attractive to game studios, interactive artists, and educators who want affordable and flexible solutions for adaptive music.

Faculty Supervisor:

Philippe Pasquier

Student:

Partner:

Audiokinetic

Discipline:

Computer science

Sector:

Information and cultural industries

University:

Simon Fraser University

Program: