Dr. Vered Shwartz collaborates with Vancouver-based tech startup to study legal research engine

Original article on cs.ubc.ca by Sonja Soo reposted by Mitacs.

UBC Computer Science Assistant Professor Vered Shwartz and her research group will be collaborating with tech startup company Caseway on a research project focused on a legal research engine based on large language models. The project is supported by a two-year funded grant from the NSERC Alliance-Mitacs Accelerate program and aims to provide more reliable answers to legal queries.  

UBC Computer Science postdoctoral researcher Maksym Taranukhin and Master’s student Ethan Zhao, part of Dr. Shwartz’s research group, will be working on this collaborative project.   

Large language models can often give inaccurate or misleading information. A study published last year showed that when models were prompted with legal queries, they generated content that were inconsistent with legal facts at least 58% of the time.   

“Large language models are already deployed and people are using them to get answers for legal questions,” says Dr. Shwartz. “Our goal is to ensure that these AI tools are as reliable as possible so that professionals and the public can trust it to provide accurate legal information.”  

Many large language models use a technique called retrieval-augmented generation, which allows the large language model to search for information and retrieve it. While this technique helps improve the model by generating more specific and factual answers, inaccuracies are still pervasive.  

“Rushing AI into legal use without fixes could be harmful, especially in high-stakes contexts where people need to rely on accurate information to make important decisions,” says Dr. Taranukhin.   

To address this problem, the researchers will be integrating different methods to enhance the reasoning ability of the program, including the use of Bayesian networks, which are graphical models that describe how different factors are connected and affect each other based on probability. The researchers will also attempt to improve the model’s ability to avoid generating an answer under uncertainty.  

According to Dr. Shwartz, one of the challenges of this project is making sure that the tool is both user-friendly and accurate.  

“There’s a tradeoff between how helpful it is and how accurate it is,” she says. “It’s tricky to strike the right balance.”  

This project builds on Dr. Shwartz’s previous research on building AI tools for legal help, such as a study led by Dr. Taranukhin in which the researchers presented a chatbot that assists air passengers and educates them about their rights.   

“Getting the right information quickly can make all the difference,” says Caseway’s CEO, Alistair Vigier. “We’re not building this to replace lawyers, but to make legal knowledge far more accessible to those who can’t easily get professional help.”

Mitacs’ programs receive funding from multiple partners across Canada. We thank the Government of Canada, the Government of Alberta, the Government of British Columbia, Research Manitoba, the Government of New Brunswick, the Government of Newfoundland and Labrador, the Government of Nova Scotia, the Government of Ontario, Innovation PEI, the Government of Quebec, the Government of Saskatchewan, and the Government of Yukon for supporting us to empower Canadian innovation. 

Do you have a business challenge that could benefit from a research solution? If so, contact Mitacs today to discuss partnership opportunities: [email protected].