机器学习智能分诊优化药物警戒效率:MLIT统计安全预警工具
本研究设计、开发并前瞻性评估可解释的机器学习智能分诊(MLIT)工具,辅助药物警戒团队审核疫苗和药物组合的统计预警。疫苗模型加权平均F1为0.81、准确率0.79;前瞻性验证中92%预警按模型首位预测关闭,98%在前3位内关闭;审核员分诊时间减少24%。
要点
- MLIT对疫苗预警的前瞻性验证显示92%按首位预测关闭、98%在前3位内关闭。
- 安全审核员报告分诊活动时间减少24%,且模型决策逻辑与审核员概念一致。
- MLIT可识别人工分诊中的不一致和人为错误,具质量控制潜力。
核心结论
- 基于机器学习的MLIT工具可提升信号检测性能、降低漏检可能,同时提高操作效率并增强可重复性与透明度。
研究设计与样本
- 研究设计、开发并前瞻性评估可解释的MLIT工具,辅助药物警戒团队审核疫苗和药物组合的统计预警。
- 采用XGBoost算法,模型按75/25拆分数据集训练与测试,并用Shapley Additive Explanations迭代优化以确保可解释性。
- 在两项各为期四个月的前瞻性验证研究中验证,覆盖疫苗和药物组合中超过20个产品。
主要发现
- 疫苗模型加权平均F1分数为0.81,准确率为0.79。
- 前瞻性验证中,92%的疫苗预警按模型首位预测关闭,98%在前3位预测内关闭。
- 安全审核员报告分诊活动时间减少24%,可解释性分析确认模型决策与审核员逻辑概念一致。
- 药物组合观察到可比结果,MLIT还识别了人工分诊中的不一致和人为错误。
临床怎么用
- 可将MLIT用于统计预警的智能分诊,辅助审核员优先处理高置信度预警以节省时间。
- 利用模型首位及前3位预测与审核员决策的高一致性,优化预警关闭流程。
- 将MLIT作为质量控制机制,识别人工分诊中的不一致和人为错误。
- 对低置信度预测保留人工监督,确保决策可追溯与监管合规。
局限性
- 低置信度预测仍需人工监督,人类审核不可或缺。
- 更广泛实施与进一步自动化需持续模型优化和用户参与。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
BACKGROUND AND AIM: Pharmacovigilance is essential to ensuring patient safety by enabling timely identification of adverse reactions in increasingly complex and voluminous data. Routine quantitative signal detection methods generate statistical alerts for product-event pairs based on predefined criteria; however, most alerts do not warrant further investigation, creating inefficiencies and significant time demands for pharmacovigilance teams. Manual triage of these alerts is often resource-intensive, prone to variability, and challenging to audit, highlighting the need for more reliable, transparent and efficient triage strategies. This study aimed to design, develop and prospectively evaluate an explainable Machine Learning for Intelligent Triage (MLIT) tool to assist pharmacovigilance teams in reviewing statistical alerts for vaccine and drug portfolios. The objective was to enhance signal detection performance without increasing the risk of missing signals, improving operational efficiency and maintaining decision traceability and regulatory compliance. METHODS: Alert and individual case safety report data were retrieved from the company's safety and signal management databases. Feature selection was guided by prior experience with a published case completeness tool, called Clinical Utility Score for Prioritisation (CUSP), and expert input. Of several ML methods explored, eXtreme Gradient Boosting (XGBoost) emerged as the optimal algorithm, with models trained and tested using a 75/25 split dataset. Iterative model refinement was conducted using Shapley Additive Explanations analyses to ensure explainability and alignment with safety reviewers' decision-making processes. Refined models underwent prospective validation in two four-month prospective validation studies, covering over 20 products across vaccine and drug portfolios. The prospective validations assessed concordance between model predictions and reviewers' decision under real-world conditions, as well as estimated time savings. RESULTS: The vaccine model demonstrated robust predictive performance, achieving a weighted-average F1 score of 0.81 and an accuracy of 0.79. In the prospective validation phase, 92% of vaccine alerts were closed in alignment with the model's top-ranked prediction, while 98% were closed within the top 3 predictions. The MLIT tool also identified inconsistencies and human errors in manual triage, highlighting its potential role as a quality-control mechanism. Safety reviewers reported a 24% reduction in time spent on triage activities, and explainability analyses confirmed that the model's decision-making was conceptually aligned with safety reviewers' logic. Comparable results were observed for the drug portfolio. CONCLUSION: This study highlights the potential of ML-based tools to improve pharmacovigilance by enhancing signal detection performance, reducing the likelihood of missed signals, while increasing operational efficiency, and strengthening reproducibility and transparency. While MLIT demonstrated high concordance with expert decisions and provided meaningful time savings, human oversight remains essential, especially for low-confidence predictions. Ongoing refinement and user engagement will be critical for broader implementation and further automation, marking a significant step forward in ensuring safer and more efficient drug safety surveillance.
引用
- Ciccarelli L, Mahaux O, Roshan C, et al. Optimising Pharmacovigilance Efficiency with MLIT (Machine Learning for Intelligent Triage): A Tool for Statistical Safety Alerts. Drug Saf. 2026. PMID: 42573921. doi:10.1007/s40264-026-01696-0
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。