Design and Development of a Chatbot-based Personalized Healthcare Tool to Support People with HIV

Large language models (LLMs), AI systems trained on vast amounts of online textual data with advanced language processing capabilities, have garnered significant interest in healthcare applications. Health chatbots integrate language models and various algorithmic tools to assist in decision-making, potentially revolutionizing patient self-management and reducing disparities in healthcare access.
However, previous work has shown notable threats to the equitable global deployment of LLMs. These include their tendency to perpetuate racial and gender bias through stereotype-laden responses and their variable performance across groups. For example, Hispanic and Black descriptors are most likely to alter judgments about relevant LLM outputs. Thus, it is critical to assess how LLM-based chatbots propagate existing biases that may amplify health disparities.
Using the retrieval augmented generation technique and an agentic workflow approach, we developed the LLM-based MARVIN chatbot for multiple health conditions, including HIV. MARVIN also integrated a triage algorithm to detect medication adherence barriers. This project will conduct bias/equity-related validation tests on MARVIN implementing the triage algorithm, including assessing gender, cultural, racial, and socioeconomic biases, as well as the chatbot’s performance in different languages. This will help ensure equitable implementation of LLM-chatbots based on LLMs, thereby advancing the development and implementation of human-centered health AI products.

Faculty Supervisor:

Sofiane Achiche

Student:

Partner:

Massachusetts Institute of Technology

Discipline:

Engineering

Sector:

Education

University:

Polytechnique Montréal

Program: