文献速递药物不良反应与药物警戒

基于监督机器学习的信号验证分类决策支持

Supervised Machine Learning-Based Decision Support for Signal Validation Classification.
PubMed 文献 · Drug Saf · 2022-05-17

本研究从公司安全数据库提取六种药品的累积数据(历史SDR验证和个例安全性报告),训练基于决策树的梯度提升多分类模型,并用SHAP分析识别关键预测特征;回顾性分析后开展前瞻性实验测试模型准确性和用户接受度。结果显示模型在3个月内对六种药品的预测准确率为83%至86%,安全专家确认了模型的适用性,且SHAP分析提供的透明预测获得高接受度。

要点

  • 梯度提升树模型对六种药品SDR验证分类的预测准确率为83%至86%。
  • SHAP分析识别关键预测特征,缓解集成模型黑箱效应,提高专家接受度。
  • 前瞻性实验中模型预测辅助安全专家高效、一致地审查SDR。

核心结论

  • 多分类机器学习模型可准确预测SDR的信号验证类别,且SHAP分析带来的透明度使安全专家高度接受。

研究设计与样本

  • 从公司安全数据库提取六种药品的累积数据(历史SDR验证和个例安全性报告),用于训练和测试梯度提升树多分类模型,并进行SHAP分析。
  • 回顾性分析后开展前瞻性实验,测试模型在真实场景中的准确性和用户接受度。

主要发现

  • 模型在3个月内对六种药品的预测准确率范围为83%至86%。
  • 公司安全专家确认了模型的适用性。
  • 系统预测为安全专家提供了有价值的信息,帮助他们高效且一致地审查SDR。

关键数值与推荐

  • 预测准确率在3个月内对六种药品为83%至86%。

临床怎么用

  • 机器学习模型可作为信号验证分类的决策支持工具,辅助安全专家审查SDR。
  • SHAP分析可提高模型透明度,促进专家接受和临床应用。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

INTRODUCTION: Signal validation in pharmacovigilance is the process of evaluating data to decide whether evidence is sufficient to justify further assessment of a detected signal. During the signal validation process, safety experts in our organization are required to review signals of disproportionate reporting (SDRs) and classify them into one of six predefined categories. OBJECTIVE: This experiment explored the extent to which predictive machine learning (ML) models can support the decision making of safety experts by accurately identifying the most appropriate predefined signal validation category. METHODS: We extracted cumulative data for six medicinal products, consisting of historic SDR validations and Individual Case Safety Reports, from the company's safety database for training and testing of the ML model. We implemented a decision tree-based supervised multiclass classifier model termed Gradient Boosted Trees followed by a SHapley Additive exPlanations (SHAP) analysis to mitigate the "black box" effect of the ensemble model by identifying the key predicting features in the model. Following a retrospective analysis, a prospective experiment was conducted to test the model accuracy and user acceptance in a real-life setting. RESULTS: The prediction accuracy of our ML model ranged from 83 to 86% over 3 months for the six medicinal products. The applicability of the model was confirmed by the company's safety experts. Additionally, the systematic predictions provided valuable information to the safety experts and assisted them in reviewing the SDRs efficiently and consistently. CONCLUSIONS: This experiment demonstrated that it is possible to train a multiclass classification model to accurately predict signal validation categories for SDRs. More importantly, the transparency of the predictions provided by the SHAP analysis led to high acceptance by the safety experts.

来源:Drug Saf,查看原文。版权归原作者所有。

引用

  1. Imran M, Bhatti A, King DM, et al. Supervised Machine Learning-Based Decision Support for Signal Validation Classification. Drug Saf. 2022. PMID: 35579820. doi:10.1007/s40264-022-01159-2

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。