增强智能用于药物警戒病例严重性判定
本研究基于Celgene收到的安全性报告分层随机样本,开发三种神经网络用于识别不良事件严重性:二分类严重性分类器、严重性分类分类器和严重性标准标注器。分类器在上市后报告、 solicited报告和医学文献报告中的准确率分别为83.0%、92.9%和86.3%;严重性分类F1分数为死亡77.7、住院78.9、重要医学事件75.5;标注器在 solicited报告和医学文献报告中F1分数分别为89.9和75.2。结论认为神经网络方法可为药物警戒实践者提供准确且可扩展的辅助解决方案。
要点
- 神经网络分类器在 solicited报告中严重性判定准确率达92.9%。
- 严重性分类F1分数:死亡77.7、住院78.9、重要医学事件75.5。
- 标注器在 solicited报告F1为89.9,医学文献报告为75.2。
核心结论
- 神经网络方法可为药物警戒实践者提供准确且可扩展的辅助解决方案,用于判定自发、 solicited和医学文献报告中的不良事件严重性。
研究设计与样本
- 使用Celgene收到的安全性报告分层随机样本,开发并评估三种神经网络模型:二分类严重性分类器、严重性分类分类器和严重性标准标注器;以药物警戒专家标注的真值为参考评估准确率和/或F1分数。
主要发现
- 严重性分类器在上市后报告中准确率为83.0%,在 solicited报告中为92.9%,在医学文献报告中为86.3%。
- 严重性分类的F1分数为:死亡77.7,住院78.9,重要医学事件75.5。
- 严重性标注器在 solicited报告中F1分数为89.9,在医学文献报告中为75.2。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
INTRODUCTION: Identification of adverse events and determination of their seriousness ensures timely detection of potential patient safety concerns. Adverse event seriousness is a key factor in defining reporting timelines and is often performed manually by pharmacovigilance experts. The dramatic increase in the volume of safety reports necessitates exploration of scalable solutions that also meet reporting timeline requirements. OBJECTIVE: The aim of this study was to develop an augmented intelligence methodology for automatically identifying adverse event seriousness in spontaneous, solicited, and medical literature safety reports. Deep learning models were evaluated for accuracy and/or the F1 score against a ground truth labeled by pharmacovigilance experts. METHODS: Using a stratified random sample of safety reports received by Celgene, we developed three neural networks for addressing identification of adverse event seriousness: (1) a binary adverse-event level seriousness classifier; (2) a classifier for determining seriousness categorization at the adverse-event level; and (3) an annotator for identifying seriousness criteria terms to provide supporting evidence at the document level. RESULTS: The seriousness classifier achieved an accuracy of 83.0% in post-marketing reports, 92.9% in solicited reports, and 86.3% in medical literature reports. F1 scores for seriousness categorization were 77.7 for death, 78.9 for hospitalization, and 75.5 for important medical events. The seriousness annotator achieved an F1 score of 89.9 in solicited reports, and 75.2 in medical literature reports. CONCLUSIONS: The results of this study indicate that a neural network approach can provide an accurate and scalable solution for potentially augmenting pharmacovigilance practitioner determination of adverse event seriousness in spontaneous, solicited, and medical literature reports.
引用
- Routray R, Tetarenko N, Abu-Assal C, et al. Application of Augmented Intelligence for Pharmacovigilance Case Seriousness Determination. Drug Saf. 2020. PMID: 31605285. doi:10.1007/s40264-019-00869-4
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。