指南与共识消化、代谢与肾脏急症重症综合管理与镇静镇痛

OpenEvidence在结构化分诊推荐测试中偏向安全侧

OpenEvidence errs on the safe side in a structured test of triage recommendations.
指南 / 共识 · Int J Med Inform · 2026-08-29
一句话结论在相同基准测试下,OpenEvidence漏诊急诊比例12.5%,低于ChatGPT Health的51.6%,但过度分诊非紧急病例达68.0%,且6.8%拒绝分诊。

该研究用60个临床情景(含客观数据与否×人口学/情境修饰的2×2×2×2析因设计,共960条提示)评估医生端检索增强平台OpenEvidence的分诊安全性,并与既往ChatGPT Health基准比较。在449条返回推荐的明确病例中,准确率71.3%;OpenEvidence对急诊表现漏诊12.5%(ChatGPT Health为51.6%),对非紧急家庭表现过度分诊68.0%(ChatGPT Health为64.8%)。锚定语句未改变推荐(OR=1.08,95%CI 0.62-1.88;Holm校正P=1.0)。提供客观临床数据可消除急诊漏诊(25%降至0%;P=0.005)并减少非紧急过度分诊(78.7%降至57.8%;P=0.014)。960条回复中65条(6.8%)拒绝分诊,仅见于仅有症状的家庭或常规提示。

要点

  • 在449条返回推荐的明确病例中,OpenEvidence分诊准确率为71.3%。
  • OpenEvidence对急诊表现漏诊12.5%,低于既往ChatGPT Health基准的51.6%。
  • OpenEvidence对非紧急家庭表现过度分诊68.0%(ChatGPT Health为64.8%)。
  • 提供客观临床数据使急诊漏诊从25%降至0%(P=0.005),非紧急过度分诊从78.7%降至57.8%(P=0.014)。
  • 锚定语句未改变推荐(OR=1.08,95%CI 0.62-1.88;Holm校正P=1.0)。

适用人群与场景

  • 评估对象为医生端检索增强平台OpenEvidence,使用60个临床医生撰写的临床情景,覆盖21个领域、30个临床场景,每个场景含客观临床数据与否,并与人口学和情境修饰因素交叉。
  • 提示分为480条明确病例和480条边缘病例,回复按临床医生金标准分类为正确分诊、漏诊、过度分诊或寻求证据的拒绝。

研究设计与关键结果

  • 研究类型为结构化基准测试,使用60个临床情景生成960条提示(480条明确病例、480条边缘病例),采用聚类自助重采样、混合效应逻辑回归和Holm-Bonferroni校正分析。
  • 在449条返回推荐的明确病例中,OpenEvidence分诊准确率为71.3%。
  • OpenEvidence对急诊表现漏诊12.5%,既往ChatGPT Health基准为51.6%;对非紧急家庭表现过度分诊68.0%,ChatGPT Health为64.8%。
  • 锚定语句未改变推荐(OR=1.08,95%CI 0.62-1.88;Holm校正P=1.0)。
  • 提供客观临床数据使急诊漏诊从25%降至0%(P=0.005),非紧急过度分诊从78.7%降至57.8%(P=0.014)。
  • 960条回复中65条(6.8%)拒绝分诊,仅见于仅有症状的家庭或常规提示。

证据类型、等级与局限

  • 证据类型为结构化基准测试,比较OpenEvidence与既往报告的ChatGPT Health基准。
  • 局限:原文未报告局限性。

对床旁实践的改变

  • 作者认为,这些发现支持在部署情境中评估健康人工智能,并将拒绝分诊视为一种独立的输出类别,其临床意义需要单独评估。

提炼说明

  • 本条依据论文摘要整理;摘要未涉及「诊断标准与关键阈值」「处置流程与药物剂量」「红旗征与不能漏的情况」「指南或共识出处」,故未列出这些栏目;全文请点「阅读原文」。
英文原文(摘要)· 点击展开

BACKGROUND: Large language model (LLM) chatbots are increasingly consulted for triage decisions. A structured benchmark reported that ChatGPT Health, a consumer-facing health assistant, under-triaged 51.6 % of true emergencies and was susceptible to social anchoring. Whether a physician-facing clinical decision support platform fails similarly is unknown.

OBJECTIVE: To characterize the triage safety profile of OpenEvidence, a retrieval-augmented, physician-facing platform, using the identical benchmark previously applied to ChatGPT Health.

METHODS: We evaluated 60 clinician-authored vignettes from 30 clinical scenarios across 21 domains. Each scenario was written with and without objective clinical data and crossed with demographic and contextual modifiers in a 2 × 2 × 2 × 2 factorial design, yielding 960 prompts (480 clear-case, 480 edge-case). Responses were classified against a clinician gold standard as correct triage, under-triage, over-triage, or evidence-seeking refusal. Analyses used cluster bootstrap resampling, mixed-effects logistic regression, and Holm-Bonferroni correction.

RESULTS: Among 449 clear-case responses that returned a recommendation, accuracy was 71.3 %. OpenEvidence under-triaged 12.5 % of emergency presentations versus 51.6 % in the previously reported ChatGPT Health benchmark, and over-triaged 68.0 % of nonurgent Home presentations (ChatGPT Health, 64.8 %). Anchoring statements did not alter recommendations (OR = 1.08, 95 % CI 0.62-1.88; Holm-adjusted p = 1.0). Objective clinical data eliminated emergency under-triage (25 % to 0 %; p = 0.005) and reduced nonurgent over-triage (78.7 % to 57.8 %; p = 0.014). In 65 of 960 responses (6.8 %), the platform declined to assign a triage level, exclusively in symptom-only Home or Routine prompts.

CONCLUSIONS: Under this benchmark, OpenEvidence produced fewer missed emergencies than the historical ChatGPT Health comparison, while errors concentrated in over-triage and evidence-seeking refusal. These findings support evaluating health AI within its deployment context and treating refusal as a distinct output category whose clinical implications require separate assessment.

来源:Int J Med Inform,查看原文。版权归原作者所有。

参考来源

  1. Jia E, Omar M, Barash Y, et al. OpenEvidence errs on the safe side in a structured test of triage recommendations. Int J Med Inform. 2026. PMID: 42673790. doi:10.1016/j.ijmedinf.2026.106687

供执业医师抢救与诊疗参考,不替代临床判断;剂量请结合体重、肝肾功能和本院药品规格复核,以指南原文为准。