文献速递药物不良反应与药物警戒

人工智能自动编码患者药品不良反应报告的全法药物警戒数据验证

Validation of Artificial Intelligence to Support the Automatic Coding of Patient Adverse Drug Reaction Reports, Using Nationwide Pharmacovigilance Data.
PubMed 文献 · Drug Saf · 2022-05-17

内部验证中,不良反应识别AUC达0.97,F-measure 0.80;严重性评估AUC 0.85,F-measure 0.63,外部验证结果一致稳健。该系统已于2021年1月起被法国卫生当局用于COVID-19疫苗药物警戒。

要点

  • AI模型可自动预编码患者不良反应报告,识别不良反应AUC 0.97,评估严重性AUC 0.85。
  • 不同机器学习方法性能相似,TF-IDF+LGBM与XLM在不良反应识别上AUC均为0.97。
  • 系统已部署用于COVID-19疫苗药物警戒,但真实场景性能仍需进一步验证。

核心结论

  • AI模型在自动编码患者不良反应报告方面展现出有前景的性能,不同方法结果非常相似。
  • 该系统自2021年1月起被法国国家卫生当局部署用于COVID-19疫苗药物警戒。

研究设计与样本

  • 使用26个中心的数据进行训练和内部验证,剩余中心同期数据进行外部验证。
  • 参考标准为专家编码的不良事件症状(MedDRA首选术语)和严重性。

主要发现

  • 内部验证中,不良反应识别任务TF-IDF+LGBM的AUC为0.97、F-measure为0.80;XLM transformer的AUC为0.97、F-measure为0.78。
  • 内部验证中,严重性评估任务FastText+LGBM的AUC为0.85、F-measure为0.63;CamemBERT+LGBM的AUC为0.84、F-measure为0.63。
  • 外部验证中,不良反应识别和严重性评估任务均获得一致且稳健的结果。

临床怎么用

  • AI系统可自动预编码患者报告,帮助药物警戒专家在卫生危机期间优先或加速人工评估。
  • 该系统已用于COVID-19疫苗药物警戒,可支持上报激增时的信号检测工作。

局限性

  • 需进一步研究验证该工具在真实世界环境中的性能。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

INTRODUCTION: Adverse drug reaction reports are usually manually assessed by pharmacovigilance experts to detect safety signals associated with drugs. With the recent extension of reporting to patients and the emergence of mass media-related sanitary crises, adverse drug reaction reports currently frequently overwhelm pharmacovigilance networks. Artificial intelligence could help support the work of pharmacovigilance experts during such crises, by automatically coding reports, allowing them to prioritise or accelerate their manual assessment. After a previous study showing first results, we developed and compared state-of-the-art machine learning models using a larger nationwide dataset, aiming to automatically pre-code patients' adverse drug reaction reports. OBJECTIVES: We aimed to determine the best artificial intelligence model identifying adverse drug reactions and assessing seriousness in patients reports from the French national pharmacovigilance web portal. METHODS: Reports coded by 27 Pharmacovigilance Centres between March 2017 and December 2020 were selected (n = 11,633). For each report, the Portable Document Format form containing free-text information filled by the patient, and the corresponding encodings of adverse event symptoms (in Medical Dictionary for Regulatory Activities Preferred Terms) and seriousness were obtained. This encoding by experts was used as the reference to train and evaluate models, which contained input data processing and machine-learning natural language processing to learn and predict encodings. We developed and compared different approaches for data processing and classifiers. Performance was evaluated using receiver operating characteristic area under the curve (AUC), F-measure, sensitivity, specificity and positive predictive value. We used data from 26 Pharmacovigilance Centres for training and internal validation. External validation was performed using data from the remaining Pharmacovigilance Centres during the same period. RESULTS: Internal validation: for adverse drug reaction identification, Term Frequency-Inverse Document Frequency (TF-IDF) + Light Gradient Boosted Machine (LGBM) achieved an AUC of 0.97 and an F-measure of 0.80. The Cross-lingual Language Model (XLM) [transformer] obtained an AUC of 0.97 and an F-measure of 0.78. For seriousness assessment, FastText + LGBM achieved an AUC of 0.85 and an F-measure of 0.63. CamemBERT (transformer) + Light Gradient Boosted Machine obtained an AUC of 0.84 and an F-measure of 0.63. External validation for both adverse drug reaction identification and seriousness assessment tasks yielded consistent and robust results. CONCLUSIONS: Our artificial intelligence models showed promising performance to automatically code patient adverse drug reaction reports, with very similar results across approaches. Our system has been deployed by national health authorities in France since January 2021 to facilitate pharmacovigilance of COVID-19 vaccines. Further studies will be needed to validate the performance of the tool in real-life settings.

来源:Drug Saf,查看原文。版权归原作者所有。

引用

  1. Martin GL, Jouganous J, Savidan R, et al. Validation of Artificial Intelligence to Support the Automatic Coding of Patient Adverse Drug Reaction Reports, Using Nationwide Pharmacovigilance Data. Drug Saf. 2022. PMID: 35579816. doi:10.1007/s40264-022-01153-8

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。