Investigation on the Enhancement of Design and Manufacturing Processes through the Integration of Large Language Models and Vision Models

In the world of design and manufacturing, there is always a need to supply certain goods (parts, assemblies, materials, etc.) from a manufacturer to a buyer. In most cases, there is a wide variety of procurement options, and choosing the right supply is a complicated process that requires taking multiple factors into account. The optimal supply is difficult to achieve.
Axya makes collaboration between buyers and suppliers easier, faster, and more transparent. Its software optimizes the procurement process by offering simple technological solutions to facilitate and organize the sourcing process.
However, Axya’s optimization process involves the manual processing of engineering documentation, such as engineering drawings and manufacturing procedures. Machine learning techniques, namely Vision Transformers (ViTs) and Large Language Models (LLMs), could facilitate this process by automatically interpreting and extracting knowledge from this documentation.
Thus, the main goal of this project is to develop an adaptation technique for ViTs and LLMs that will equip them with the ability to interpret and extract domain-specific knowledge. The adapted models will speed up Axya’s engineering documentation processing workflow.

Faculty Supervisor:

Yaoyao Fiona Zhao

Student:

Partner:

Axya Inc.

Discipline:

Computer science

Sector:

Information and cultural industries; Professional, scientific and technical services

University:

McGill University

Program: