ChatGPT4在确定结肠镜后复查与监测建议方面优于内镜医师
本研究将2家大型学术中心505例连续结肠镜患者的临床资料输入ChatGPT4,比较其推荐与真实消化科实践及USMSTF专家小组建议的一致性。ChatGPT4与USMSTF小组的一致率为85.7%,高于消化科实践的75.4%(P < .001),提示其可基于临床数据准确制定常规结肠镜筛查间期,但广泛应用仍需进一步训练。
要点
- ChatGPT4推荐与USMSTF小组一致率85.7%,高于消化科实践的75.4%(P < .001)。
- ChatGPT4与USMSTF小组的评分者间信度良好(Fleiss κ 0.786,95% CI 0.734-0.838)。
- ChatGPT4与USMSTF小组不一致的建议中,推荐更晚筛查26例(5.1%),推荐更早筛查44例(8.7%)。
核心结论
- 初步真实世界结果表明,ChatGPT4可基于逐字输入的临床数据准确制定常规结肠镜筛查间期。
- 大语言模型具有临床应用潜力,但广泛应用仍需进一步训练。
研究设计与样本
- 研究纳入2023年1月1日至4月30日在2家大型学术中心接受结肠镜检查的505例连续患者。
- 将患者现病史、结肠镜数据和病理报告去标识化后输入ChatGPT4,查询下一次结肠镜随访间期建议。
- 采用McNemar检验和评分者间信度,比较ChatGPT4建议与内镜医师操作报告中的实际监测间期及USMSTF指导。
- USMSTF指导由专家小组参考临床信息和指南文件为每例病例生成。
主要发现
- 505例查询中,ChatGPT4在99.2%的查询中成功给出随访建议。
- ChatGPT4与USMSTF小组的一致率为85.7%,消化科实践与USMSTF小组的一致率为75.4%(P < .001)。
- ChatGPT4与USMSTF小组不一致的建议占14.3%,其中推荐更晚筛查26例(5.1%),推荐更早筛查44例(8.7%)。
- ChatGPT4与USMSTF小组的评分者间信度良好(Fleiss κ 0.786;95% CI 0.734-0.838;P < .001)。
临床怎么用
- 作者认为,ChatGPT4可基于逐字输入的临床数据准确制定常规结肠镜筛查间期。
- 作者认为,大语言模型具有临床应用潜力,但广泛应用仍需进一步训练。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
BACKGROUND & AIMS: Large language models including Chat Generative Pretrained Transformers version 4 (ChatGPT4) improve access to artificial intelligence, but their impact on the clinical practice of gastroenterology is undefined. This study compared the accuracy, concordance, and reliability of ChatGPT4 colonoscopy recommendations for colorectal cancer rescreening and surveillance with contemporary guidelines and real-world gastroenterology practice. METHODS: History of present illness, colonoscopy data, and pathology reports from patients undergoing procedures at 2 large academic centers were entered into ChatGPT4 and it was queried for the next recommended colonoscopy follow-up interval. Using the McNemar test and inter-rater reliability, we compared the recommendations made by ChatGPT4 with the actual surveillance interval provided in the endoscopist's procedure report (gastroenterology practice) and the appropriate US Multisociety Task Force (USMSTF) guidance. The latter was generated for each case by an expert panel using the clinical information and guideline documents as reference. RESULTS: Text input of de-identified data into ChatGPT4 from 505 consecutive patients undergoing colonoscopy between January 1 and April 30, 2023, elicited a successful follow-up recommendation in 99.2% of the queries. ChatGPT4 recommendations were in closer agreement with the USMSTF Panel (85.7%) than gastroenterology practice recommendations with the USMSTF Panel (75.4%) (P < .001). Of the 14.3% discordant recommendations between ChatGPT4 and the USMSTF Panel, recommendations were for later screening in 26 (5.1%) and for earlier screening in 44 (8.7%) cases. The inter-rater reliability was good for ChatGPT4 vs USMSTF Panel (Fleiss κ, 0.786; 95% CI, 0.734-0.838; P < .001). CONCLUSIONS: Initial real-world results suggest that ChatGPT4 can define routine colonoscopy screening intervals accurately based on verbatim input of clinical data. Large language models have potential for clinical applications, but further training is needed for broad use.
引用
- Chang PW, Amini MM, Davis RO, et al. ChatGPT4 Outperforms Endoscopists for Determination of Postcolonoscopy Rescreening and Surveillance Recommendations. Clin Gastroenterol Hepatol. 2024. PMID: 38729387. doi:10.1016/j.cgh.2024.04.022
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。