Improving the Detection of Performance Regressions with Change Point Detection Methods

As software systems evolve, the software performance has to be constantly monitored to ensure that systems are responsive and economically sustainable. Companies like Mozilla have dedicated teams and an automated workflow to detect performance regression. However, due to the nature of performance measurements, the automatic detection of regressions is bound to flag issues falsely, which makes the performance team waste investigation time, and miss actual regressions, which could impact end users.

This research project aims to explore the state-of-the-art methods in change point detection (CPD) to improve the accuracy of Mozilla’s performance detection system. We plan to create a real-world dataset of validated performance alerts, explore the efficacy of multiple CPD methods on Mozilla’s data, and integrate the best techniques in Mozilla’s detection system. A more effective performance regression detection system would improve team productivity, enabling them to focus their efforts on fixing real performance anomalies, ultimately making the company’s products better and more responsive.

Faculty Supervisor:

Diego Elias Damasceno Costa

Student:

Partner:

Mozilla Corporation

Discipline:

Computer science

Sector:

Information and cultural industries

University:

Concordia University

Program: