五种大语言模型在髋部骨折手术麻醉决策中的大规模评估
该研究用216个标准化髋部骨折手术案例评估五种通用大语言模型,共生成54000条回复,发现模型普遍偏好椎管内麻醉(76.1%-88.6%),但自由文本理由常引用近期随机试验不支持的椎管内麻醉优势,提示不加批判使用可能改变实践模式而不改善患者结局。
要点
- 五种通用LLM均偏好椎管内麻醉(76.1%-88.6%),但理由常缺乏近期随机试验支持。
- 除GPT-5 nano外,模型均推荐术前外周神经阻滞(92.6%-99.3%),对动脉置管态度保守。
- 仅发现有限社会人口学偏差,150次比较中仅1个显著且有临床意义的效应。
核心结论
- LLM提供的建议总体合理,但系统性偏好与当代证据不符,不加批判使用可能改变实践模式而不改善患者结局。
研究设计与样本
- 评估五种通用LLM(DeepSeek 3.2、Gemini 2.5 Flash、GPT-5、GPT-5 mini、GPT-5 nano),使用216个标准化髋部骨折手术案例,覆盖6种手术类型、2种性别和18个患者变量,每个组合生成50个样本,共54000条回复。
- 另对两种临床LLM(OpenEvidence、Doximity GPT)进行有限敏感性分析,各评估36条回复。
主要发现
- 所有模型均偏好椎管内麻醉而非全身麻醉(76.1%-88.6%的回复)。
- 除DeepSeek 3.2外,所有模型均能根据相关医学禁忌症适当调整建议。
- 除GPT-5 nano外,所有模型均推荐术前外周神经阻滞(92.6%-99.3%),并对动脉置管持适当保守态度。
- 自由文本理由频繁引用近期随机试验不支持的椎管内麻醉优势,且多数模型在缺乏临床依据的情况下给出强烈椎管内麻醉推荐。
- 在150次比较中仅发现1个显著且有临床意义的社会人口学偏差效应。
临床怎么用
- 临床使用LLM辅助麻醉决策时需警惕其系统性偏好可能偏离当代证据,应结合临床判断和最新证据批判性采纳建议。
局限性
- 敏感性分析仅评估两种临床LLM且样本量有限(各36条回复)。
- 研究基于标准化案例而非真实临床场景,可能无法完全反映实际决策复杂性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
BACKGROUND: Large language models (LLMs) show promise for perioperative decision support, but persistent issues, including hallucinations, miscalibration, and biases, indicate they require rigorous evaluation before clinical use. As LLM adoption increases in perioperative settings, systematic evaluation is needed to determine how patient and surgical factors affect performance. METHODS: We evaluated five general-purpose LLMs (DeepSeek 3.2, Gemini 2.5 Flash, GPT-5, GPT-5 mini, GPT-5 nano) using 216 standardized hip fracture surgery vignettes crossing six surgery types, two sexes, and 18 patient variables. We generated 50 samples per combination for 54,000 total responses and collected both structured recommendations and free-text justifications. We used logistic regression to estimate effects on three primary outcomes: anesthesia type, peripheral nerve block placement, and arterial line placement. In a limited sensitivity analysis, we evaluated two clinical LLMs (OpenEvidence, Doximity GPT) with 36 responses each. RESULTS: All models favored neuraxial over general anesthesia (76.1%-88.6% of responses), and all but DeepSeek 3.2 appropriately adjusted recommendations for relevant medical contraindications. All models except GPT-5 nano recommended preoperative peripheral nerve blocks (92.6%-99.3%) and were appropriately conservative regarding arterial line placement. However, free-text justifications frequently cited neuraxial benefits unsupported by recent randomized trials, and most models issued strong neuraxial recommendations despite a lack of clinical justification. We identified limited sociodemographic biases, with only one significant and clinically meaningful effect across 150 comparisons. Clinical LLMs provided similar recommendations to general-purpose models. CONCLUSIONS: While LLMs provided generally reasonable recommendations, systematic preferences diverging from contemporary evidence suggest uncritical use could shift practice patterns without improving patient outcomes.
引用
- Chen R, Warburton A, Do R, et al. Large-Scale Evaluation of Five Large Language Models in Anesthesia Decision-Making for Hip Fracture Surgery. Anesth Analg. 2026. PMID: 42268736. doi:10.1213/ANE.0000000000008148
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。