文献速递药物相互作用

评估ChatGPT预测药物相互作用的能力:基于住院患者数据的真实世界证据

Evaluating the capability of ChatGPT in predicting drug-drug interactions: Real-world evidence using hospitalized patient data.
PubMed 文献 · Br J Clin Pharmacol · 2024-10-02

本研究以120例住院患者的人口学、诊断和处方数据,通过三种标准化提示输入ChatGPT 3.5,与药师DDI评估对比以估计诊断准确性,并计算ROC曲线下面积和评分者间信度。ChatGPT的应答随提示措辞风格而异,提及「药物相互作用」的提示敏感性更高。混淆矩阵显示真阳性率低、真阴性率高,ChatGPT与药师间一致性极低(Cohen's kappa 0.077-0.143)。低敏感性表明ChatGPT在识别DDI方面尚不成功,需进一步开发才能可靠评估真实世界中的潜在DDI。

要点

  • ChatGPT 3.5识别住院患者DDI的敏感性低,真阳性率低而真阴性率高。
  • ChatGPT与药师DDI评估的一致性极低(Cohen's kappa 0.077-0.143)。
  • 提示中提及「药物相互作用」时ChatGPT的敏感性更高,应答受提示措辞影响。

核心结论

  • ChatGPT 3.5在真实世界住院患者数据中识别DDI的敏感性低,与药师评估一致性极低,尚不能可靠评估潜在DDI。

研究设计与样本

  • 研究纳入120例住院患者,将其人口学、诊断和处方药物信息通过三种标准化提示输入ChatGPT 3.5,并与药师DDI评估对比以估计诊断准确性。

主要发现

  • ChatGPT的应答因提示措辞风格而异,提及「药物相互作用」的提示敏感性更高。
  • 混淆矩阵显示真阳性率低、真阴性率高。
  • ChatGPT与药师间的Cohen's kappa值为0.077-0.143,提示一致性极低。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

Drug-drug interactions (DDIs) present a significant health burden, compounded by clinician time constraints and poor patient health literacy. We assessed the ability of ChatGPT (generative artificial intelligence-based large language model) to predict DDIs in a real-world setting. Demographics, diagnoses and prescribed medicines for 120 hospitalized patients were input through three standardized prompts to ChatGPT version 3.5 and compared against pharmacist DDI evaluation to estimate diagnostic accuracy. Area under receiver operating characteristic and inter-rater reliability (Cohen's and Fleiss' kappa coefficients) were calculated. ChatGPT's responses differed based on prompt wording style, with higher sensitivity for prompts mentioning 'drug interaction'. Confusion matrices displayed low true positive and high true negative rates, and there was minimal agreement between ChatGPT and pharmacists (Cohen's kappa values 0.077-0.143). Low sensitivity values suggest a lack of success in identifying DDIs by ChatGPT, and further development is required before it can reliably assess potential DDIs in real-world scenarios.

来源:Br J Clin Pharmacol,查看原文。版权归原作者所有。

引用

  1. Radha Krishnan RP, Hung EH, Ashford M, et al. Evaluating the capability of ChatGPT in predicting drug-drug interactions: Real-world evidence using hospitalized patient data. Br J Clin Pharmacol. 2024. PMID: 39359001. doi:10.1111/bcp.16275

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。