大语言模型与血栓形成专家对比:静脉血栓栓塞症患者教育与临床决策的比较研究
本研究比较了3个大语言模型(Le Chat Pixtral Large、DeepSeek-R1、ChatGPT-4.5)与3位血栓与止血专家对3个患者教育问题和3个临床决策问题的回答质量,并由37名医师评分。结果显示大语言模型在患者教育方面显著优于专家,在临床决策方面DeepSeek-R1优于专家,其余两个模型与专家相当;评估者无法区分专家与大语言模型生成的回答。
要点
- 大语言模型在静脉血栓栓塞症患者教育回答的 adequacy 评分显著优于专家。
- 临床决策方面DeepSeek-R1优于专家,Le Chat Pixtral Large与ChatGPT-4.5与专家相当。
- 评估者无法区分专家与大语言模型生成的回答(中位数均为6.0)。
核心结论
- 大语言模型在静脉血栓栓塞症相关患者教育中优于专家,在临床决策中与专家相当或超过专家,且其回答与专家回答无法区分。
- 尽管存在需要解决的主要障碍,大语言模型在血栓与止血领域的临床管理和患者教育方面具有强大潜力。
研究设计与样本
- 比较研究:3位血栓与止血专家和3个大语言模型(Le Chat Pixtral Large、DeepSeek-R1、ChatGPT-4.5)回答3个患者教育问题和3个临床决策问题。
- 37名医师对回答的 adequacy 进行评分(1=极差,10=优秀),并估计来源(1=肯定是LLM,10=肯定是人类)。
- 均值差异用t检验,中位数用Wilcoxon检验,相关性用Spearman检验,所有P值经Bonferroni校正。
主要发现
- 患者教育回答的 adequacy 均值差异:Le Chat Pixtral Large +1.6(95% CI,1.3-2.0;P < .01),DeepSeek-R1 +1.7(95% CI,1.3-2.1;P < .001),ChatGPT-4.5 +1.9(95% CI,1.6-2.3;P < .001),均显著优于专家。
- 临床决策回答的 adequacy 均值差异:DeepSeek-R1 +1.4(95% CI,1.1-1.8;P < .001)优于专家;Le Chat Pixtral Large -0.3(95% CI,-0.8-0.1;P = .96)和ChatGPT-4.5 +0.5(95% CI,0.0-0.9;P = .18)与专家表现相当。
- 评估者无法区分专家(中位数6.0;IQR,3.0-8.0)和大语言模型生成的回答(中位数6.0;IQR,4.0-8.0)。
临床怎么用
- 原文未据此给出推荐。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
BACKGROUND: Large language models (LLMs) have demonstrated remarkable capabilities in various medical fields, yet their performance in thrombosis and hemostasis, particularly in patient education and complex clinical decision making, is unexplored. OBJECTIVES: We aimed to compare the quality of responses from LLMs vs thrombosis experts and assess clinician's ability to distinguish between them. METHODS: Three experts on thrombosis and hemostasis and 3 LLMs (Le Chat Pixtral Large, DeepSeek-R1, and ChatGPT-4.5) answered 3 patient education and 3 clinical decision-making queries. Thirty-seven physicians rated responses for adequacy (1 = very poor; 10 = excellent) and estimated their origin (1 = certainly LLM; 10 = certainly human). Mean differences were assessed via t-tests, medians via Wilcoxon tests, and correlations via Spearman test. All P values were adjusted with Bonferroni correction. RESULTS: LLMs provided significantly better patient education responses than experts. Mean adequacy score differences were as follows: Le Chat Pixtral Large +1.6 (95% CI, 1.3-2.0; P < .01), DeepSeek-R1 +1.7 (95% CI, 1.3-2.1; P < .001), and ChatGPT-4.5 +1.9 (95% CI, 1.6-2.3; P < .001). In clinical decision-making, DeepSeek-R1 outperformed experts (+1.4; 95% CI, 1.1-1.8; P < .001), whereas Le Chat Pixtral Large (-0.3; 95% CI, -0.8-0.1; P = .96) and ChatGPT-4.5 (+0.5; 95% CI, 0.0-0.9; P = .18), performed comparably with experts. Evaluators could not distinguish between expert (median, 6.0; IQR, 3.0-8.0) and LLM-generated responses (median, 6.0; IQR, 4.0-8.0). CONCLUSION: LLMs outperform experts in venous thromboembolism-related patient education and match or exceed them in clinical decision making, providing responses indistinguishable from experts. Although major barriers need to be addressed, LLMs have strong potential to support clinical management and patient education in the field of thrombosis and hemostasis.
引用
- Vladic N, Nopp S, Pabinger I, et al. Large language models vs thrombosis experts: a comparative study on patient education and clinical decision-making in venous thromboembolism. J Thromb Haemost. 2025. PMID: 41015158. doi:10.1016/j.jtha.2025.09.004
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。