Training Generative Tabular Foundation Models

Layer 6 is a machine learning research and engineering company owned by The Toronto-Dominion Bank. Most of TDs transactional data is stored in relational database management systems in the form of tabular datasets, structured as rows and columns. While Large Language Models (LLMs) have revolutionized the handling of unstructured data (such as text, audio, and images) they still lag behind in processing tabular data. Recently, Layer 6 developed a novel tabular foundation model called TabDPT. To advance research in this emerging area, Layer 6 is collaborating with the team at Polytechnique Montréal to explore some of their ideas aimed at enhancing both the training processes and understanding of performance characteristics of these models. Models like TabDPT are designed to enable the reuse of pre-trained models across projects and use in-context learning, offering both potential economic advantages and gains in accuracy over traditional methods. These improvements can translate into performance boosts in critical downstream tasks containing regression and prediction for applications such as fraud detection.

Faculty Supervisor:

Amine Mhedhbi

Student:

Partner:

Layer 6 AI

Discipline:

Computer science

Sector:

Finance and Insurance; Professional, scientific and technical services

University:

Polytechnique Montréal

Program: