Using Machine Learning to Predict 30-Day Risk of Hospitalization, Emergency Visit or Death Among Albertans Who Received Opioid Prescriptions

When utilizing and implementing ML for prediction using administrative health data, two key issues are ML algorithm evaluation and generalizability21. Current approaches evaluate model performance by quantifying how closely the prediction made by the model matches known health outcomes. Evaluation metrics include sensitivity, specificity, and positive predictive value, as well as measures such as the area under the receiver operating characteristic (ROC) curve, the area under the precision-recall curve, and calibration. Because no single measurement reflects all of the desirable properties of a model, several measurements typically are reported to summarize the performance of the model16. Furthermore, model performance ultimately comes down to discrimination and calibration22. Discrimination is usually quantified using a concordance statistic (area under ROC) while calibration is graphically represented as observed to expected ratios.
Generalizability is also an issue that must be acknowledged in ML prediction settings21. ML models trained in one setting may not be valid in another. The same is true for populations. Furthermore, even ML algorithms that are considered generalizable may quickly become outdated as treatment guidelines or the population changes thus requiring model updating and re-evaluation 21.

Faculty Supervisor:

Irene Cheng

Student:

Tanya Joon;Navya Gururaj Rao

Partner:

OKAKI

Discipline:

Computer science

Sector:

Professional, scientific and technical services

University:

University of Alberta