文献速递消化内镜消化道肿瘤与早癌

基于大语言模型的结肠镜监测间隔临床决策支持系统的开发与外部验证

Development and External Validation of a Large Language Model-Based Clinical Decision-Support System for Colonoscopy Surveillance Intervals.
PubMed 文献 · Clin Gastroenterol Hepatol · 2026-05-30

本研究开发并外部验证了一种混合大语言模型与规则引擎的临床决策支持系统,用于根据2020年美国多学会工作组指南分配结肠镜监测间隔。系统在内部验证中达到94.0%的指南一致性,外部验证中达到92.8%,幻觉相关错误低于1%。

要点

  • 基于开源大语言模型和规则引擎的决策支持系统可实现高指南一致性的结肠镜监测间隔分配。
  • 系统在外部验证中保持>90%的一致性,且幻觉相关错误率低于1%。
  • 误分类主要源于腺瘤计数错误和诊断性活检被误判为切除。

核心结论

  • 开发并外部验证的混合大语言模型-规则引擎临床决策支持系统在结肠镜监测间隔分配中达到>90%的指南一致性,同时保持透明性、可审计性和指南可追溯性。

研究设计与样本

  • 研究组装了开发队列(n=256)、内部验证队列(n=450)和外部验证队列(n=415)的结肠镜-病理配对数据。
  • 开发了三阶段临床决策支持系统:基于大语言模型提取病理报告中含诊断的章节、提取监测相关变量,以及使用美国多学会工作组逻辑确定性分配监测间隔。
  • 对七种开源大语言模型(3.8-32B参数)进行基准测试,按宏平均F1分数锁定表现最佳的模型后再进行验证。

主要发现

  • 表现最佳的开源模型为Gemma-27B-Instruct(宏平均F1分数0.992)。
  • 临床决策支持系统在内部验证中达到94.0%的指南一致性监测间隔分配(95%CI 91.8%-96.0%;κ=0.92;95%CI 0.89-0.95)。
  • 外部验证中达到92.8%的一致性(95%CI 90.1%-95.2%;κ=0.88;95%CI 0.84-0.91)。
  • 误分类主要由于腺瘤计数错误和诊断性活检被误判为切除。
  • 幻觉相关错误发生在不到1%的病例中。

关键数值与推荐

  • 主要结局为指南一致性监测间隔分配,目标为90%。
  • 内部验证指南一致性为94.0%(95%CI 91.8%-96.0%),外部验证为92.8%(95%CI 90.1%-95.2%)。

临床怎么用

  • 该系统可本地部署,使用开源大语言模型,有助于在临床实践中自动化结肠镜监测间隔分配并提高指南依从性。
  • 系统的透明性和可审计性支持临床医生审查和追溯指南依据。

局限性

  • 误分类仍存在,主要源于腺瘤计数错误和诊断性活检误判。
  • 研究未报告系统在其他人群或医疗环境中的广泛适用性数据。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

BACKGROUND & AIMS: Adherence to guideline-based colonoscopy surveillance intervals remains suboptimal. Large language models show promise for automating interval assignment, but prior studies relied on proprietary models and have variably assessed generalizability. We developed and externally validated a clinical decision-support system that integrates locally deployable, open-source large language model-based variable extraction with a rules engine that encodes the 2020 United States Multi-Society Task Force guidelines. METHODS: We assembled development (n = 256), internal (n = 450), and external validation (n = 415) cohorts of colonoscopy-pathology dyads. We developed a 3-stage clinical decision-support system including: (1) large language model-based extraction of diagnosis-bearing sections from pathology reports; (2) large language model-based extraction of surveillance-relevant variables; and (3) deterministic assignment of surveillance intervals using United States Multi-Society Task Force logic. Seven open-source large language models (3.8-32B parameters) were benchmarked; the highest-performing model by macro-averaged F1 score was locked prior to validation. The primary outcome was guideline-concordant interval assignment (target, 90%); secondary outcomes included extraction performance and hallucination rate. RESULTS: The best-performing open-source model, Gemma-27B-Instruct (macro-averaged F1 score, 0.992), was locked. The clinical decision-support system achieved 94.0% guideline-concordant surveillance interval assignment in internal validation (95% confidence interval, 91.8%-96.0%; κ = 0.92; 95% confidence interval, 0.89-0.95) and 92.8% concordance in external validation (95% confidence interval, 90.1%-95.2%; κ = 0.88; 95% confidence interval, 0.84-0.91). Misclassifications were primarily due to adenoma miscounting and diagnostic biopsies misinterpreted as resections. Hallucination-related errors occurred in fewer than 1% of cases. CONCLUSIONS: We report an externally validated, hybrid large language model rules-based clinical decision-support system for colonoscopy surveillance that achieves >90% concordance while preserving transparency, auditability, and guideline traceability.

来源:Clin Gastroenterol Hepatol,查看原文。版权归原作者所有。

引用

  1. Rao A, Bali A, Tran V, et al. Development and External Validation of a Large Language Model-Based Clinical Decision-Support System for Colonoscopy Surveillance Intervals. Clin Gastroenterol Hepatol. 2026. PMID: 42218968. doi:10.1016/j.cgh.2026.05.017

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。