利用基于上下文的方法改进GPT-4o对美国放射学会RADS指南的查询
该研究针对美国放射学会(ACR)五个主要RADS指南(CAD-RADS、BI-RADS、LI-RADS、PI-RADS、Lung-RADS)各起草40道代表性问题共200题,分别测试三种GPT-4o模型:无上下文、附PDF指南、基于网络的检索增强生成(RAG)。结果显示附PDF的GPT-4o正确回答90%(180/200),优于RAG的83%(165/200)和无上下文的70%(140/200),均P<0.001;三者平均响应时间相似(P=0.97)。结论认为整合指南文档等上下文信息可显著提高GPT-4o回答ACR RADS相关问题的准确性。
要点
- 附PDF指南的GPT-4o正确率90%(180/200),高于RAG的83%(165/200)和无上下文70%(140/200),均P<0.001。
- 三种模型平均响应时间相似:附PDF 13.6±6.8秒、RAG 13.5±3.5秒、无上下文13.6±6.3秒,P=0.97。
- 整合指南文档等上下文信息可显著提高GPT-4o回答ACR RADS问题的准确性。
核心结论
- 整合指南文档等上下文信息可显著提高GPT-4o回答ACR RADS相关问题的准确性。
研究设计与样本
- 针对美国放射学会(ACR)五个主要RADS指南(CAD-RADS、BI-RADS、LI-RADS、PI-RADS、Lung-RADS)各起草40道代表性问题共200题,分别对三种GPT-4o模型各呈现一次提示:无额外上下文、附指南PDF、基于网络的检索增强生成(RAG)。
诊断标准与关键数值
- 附PDF的GPT-4o正确回答90%(180/200),优于RAG的83%(165/200),P<0.001。
- 附PDF的GPT-4o正确回答90%(180/200),优于无上下文的70%(140/200),P<0.001。
- 平均响应时间:附PDF 13.6±6.8秒、RAG 13.5±3.5秒、无上下文13.6±6.3秒,P=0.97。
临床应用与阅片要点
- 作者认为整合指南文档等上下文信息可显著提高GPT-4o回答ACR RADS相关问题的准确性。
局限性
- 原文未报告局限性
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
RATIONALE AND OBJECTIVES: Context-based large language models (LLM) with features like PDF integration and retrieval-augmented generation (RAG) may enhance accuracy on medical guideline queries.
MATERIALS AND METHODS: 200 representative questions were drafted across five major guidelines of the American College of Radiology (ACR) Reporting and Data Systems (RADS), with 40 questions each for CAD-RADS, BI-RADS, LI-RADS, PI-RADS, and Lung-RADS. Prompts were presented once to three GPT-4o-based models: without additional context, with PDF attachment of the guideline, and using web-based retrieval-augmented generation (RAG).
RESULTS: GPT-4o with PDF support answered 90% of questions correctly (180/200), outperforming both GPT-4o with RAG (83%, 165/200; P < 0.001) and GPT-4o without additional context (70%, 140/200; P < 0.001). The average response time was similar across all models: GPT-4o with PDF (13.6 ± 6.8 sec), GPT-4o with RAG (13.5 ± 3.5 sec), and GPT-4o without context (13.6 ± 6.3 sec; P = 0.97).
CONCLUSION: Incorporating contextual information such as guideline documents substantially improves the accuracy of GPT-4o in answering ACR RADS-related questions.
引用
- Sieber J, Salam B, Nowak S, et al. Utilizing GPT-4o with context-based approaches for improved query of ACR RADS guidelines. Eur J Radiol. 2026. PMID: 41849869. doi:10.1016/j.ejrad.2026.112807
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。