文献速递药物不良反应与药物警戒

大型语言模型从电子病历临床记录中提取上市后不良药物事件

A Large Language Model for Extracting Post-marketing Adverse Drug Events from Clinical Notes in the Electronic Health Record.
PubMed 文献 · Drug Saf · 2026-06-12

本研究构建基于OpenAI o1的两步流程大语言模型,读取372份去标识化临床记录,识别191个不良药物事件。结果显示精确率94.2%、召回率84.1%、F1值88.9%,严重性判断100%正确,标签状态93.9%正确,可将自由文本转化为不良事件数据以发现新安全信号。

要点

  • 大语言模型提取不良药物事件精确率94.2%、召回率84.1%、F1值88.9%
  • 严重性判断100%正确,标签状态93.9%正确,MedDRA LLT编码92.5%正确
  • 有效不良事件中12.2%符合FDA严重标准,15.0%为未标注,8.9%为疗效失败

核心结论

  • 大语言模型可将自由文本电子病历记录转化为不良药物事件数据,精确率94.2%,聚合分析可发现潜在药物副作用新安全信号。

研究设计与样本

  • 采用OpenAI o1模型两步工作流:第一步筛选记录中的不良药物事件,第二步添加20个结构化字段。
  • 样本为加州大学旧金山分校医生和药师提供的372份去标识化记录,覆盖31个专科/场景单元,共产生191个不良药物事件。

主要发现

  • 191个不良药物事件中180个为真阳性,精确率94.2%,召回率84.1%,F1值88.9%。
  • 严重性判断100%正确,标签状态93.9%正确,MedDRA最低层级术语编码在92.5%的有效不良事件中正确,错误多为不存在的术语。
  • 所有有效不良事件中12.2%符合FDA严重标准,15.0%为未标注,8.9%为疗效失败。
  • 第一步中84.9%的记录不含不良药物事件,推理成本为每份记录0.18美元、每个验证不良事件0.35美元。
  • 模型推断出医生未提及的不良事件,如他克莫司相关低镁血症。

临床怎么用

  • 该大语言模型可集成至美国Sentinel或欧盟Darwin EU等平台,快速发现罕见、严重和未标注的不良药物事件供监管分析。

局限性

  • 研究评估的大语言模型并不完美,精确率、召回率和编码准确性均未达100%。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

BACKGROUND: Clinical notes contain a vast amount of potentially useful information about adverse drug event (ADE) signals that never reach pharmacovigilance databases. Traditional rule-based or sentence-level models often miss subtle causal cues and generate excess false positives. OBJECTIVE: To build a large language model (LLM) pipeline that reads entire electronic health record (EHR) notes, identifies drug-event pairs with a "reasonable possibility" of causation, and infers other important properties of each ADR, such whether the event is "serious" or "unlabeled". METHODS: We adopted a two-pass workflow using model "OpenAI o1": Pass 1 screens each note for ADEs; Pass 2 adds 20 structured fields. A diverse sample of 372 deidentified notes from physicians and pharmacists at the University of California, San Francisco (UCSF), from 31 specialty/setting cells, yielded 191 ADEs. One medical expert reviewed each ADE for validity, seriousness, and label status. Another expert created a gold standard manually curated ADE set on 100 of the 372 ADEs to give us a percent "recall" estimate. A third expert met with the first expert to arrive at a consensus on the validity of LLM ADEs validated by the first expert but not found in the gold standard ADEs, giving us a estimate of "accuracy". RESULTS: Of 191 ADEs, 180 were true positives (94.2% precision) with 84.1% recall (F1 = 88.9%). Seriousness was correct in 100% and label status in 93.9% of cases. Medical Dictionary for Regulatory Activities (MedDRA) lowest level term (LLT) coding was correct in 92.5% of valid ADEs; errors were mostly non-existent LLTs. Of all valid ADEs, 12.2% met FDA "serious" criteria, 15.0% were unlabeled, and 8.9% were "failure of efficacy." On the first pass, 84.9% of notes contained no ADEs, keeping inference costs to USD $0.18 per note and $0.35 per validated ADE. The model inferred some ADEs not mentioned by physicians, e.g., tacrolimus-associated hypomagnesemia. CONCLUSIONS: While the LLM evaluated in this study is not perfect, it can transform free-text EHR notes into ADEs with 94% accuracy, and such data, when statistically analyzed in aggregate, can lead to new safety signals of potential drug side effects. Integrated with platforms like Sentinel in the USA, or Darwin EU, in the European Union, this approach could rapidly surface rare, serious, and unlabeled ADEs for further regulatory analysis.

来源:Drug Saf,查看原文。版权归原作者所有。

引用

  1. Ludwig D, Wang M, Buchanan J, et al. A Large Language Model for Extracting Post-marketing Adverse Drug Events from Clinical Notes in the Electronic Health Record. Drug Saf. 2026. PMID: 42283794. doi:10.1007/s40264-026-01682-6

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。