One-class support vector machines for detecting population drift in deployed machine learning medical diagnostics

Jones, William S.; Farrow, Daniel J.

Abstract

Machine learning (ML) models are increasingly being applied to diagnose and predict disease, but face technical challenges such as population drift, where the training and real-world deployed data distributions differ. This phenomenon can degrade model performance, risking incorrect diagnoses. Current detection methods are limited: not directly measuring population drift and often requiring ground truth labels for new patient data. Here, we propose using a one-class support vector machine (OCSVM) to detect population drift. We trained a OCSVM on the Wisconsin Breast Cancer dataset and tested its ability to detect population drift on simulated data. Simulated data was offset at 0.4 standard deviations of the minimum and maximum values of the radius_mean variable, at three noise levels: 5%, 10% and 30% of the standard deviation; 10,000 records per noise level. We hypothesised that increased noise would correlate with more OCSVM-detected inliers, indicating a sensitivity to population drift. As noise increased, more inliers were detected: 5% (27 inliers), 10% (486), and 30% (851). Therefore, this approach could effectively alert to population drift, supporting safe ML diagnostics adoption. Future research should explore OCSVM monitoring on real-world data, enhance model transparency, investigate complementary statistical and ML methods, and extend applications to other data types.

Citation

Jones, W. S., & Farrow, D. J. (2025). One-class support vector machines for detecting population drift in deployed machine learning medical diagnostics. Scientific reports, 15(1), Article 12157. https://doi.org/10.1038/s41598-025-94427-x

• Journal Article Type: Article
• Acceptance Date: Mar 13, 2025
• Online Publication Date: Apr 9, 2025
• Publication Date: Apr 9, 2025
• Deposit Date: Apr 15, 2025
• Publicly Available Date: Apr 15, 2025
• Journal: Scientific Reports
• Print ISSN: 2045-2322
• Publisher: Nature Publishing Group
• Peer Reviewed: Peer Reviewed
• Volume: 15
• Issue: 1
• Article Number: 12157
• DOI: https://doi.org/10.1038/s41598-025-94427-x
• Keywords: Population drift; Covariate shift; Machine learning; Medical diagnostics; Monitoring; Deployment
• Public URL: https://hull-repository.worktribe.com/output/5130140