使用ChatGPT大语言模型从放射学报告自由文本印象中提取额外影像检查建议的细节
这项回顾性单中心研究评估大语言模型从放射学报告印象中提取额外影像检查建议(RAI)可操作细节的性能,使用206份报告测试GPT-3.5和GPT-4。GPT-4在提取RAI检查方式(94.2% vs 85.4%,p < .001)、身体部位(86.9% vs 77.2%,p = .004)和时间范围(99.0% vs 95.6%,p = .02)方面比GPT-3.5更准确,两者提取RAI理由的准确率均为91.7%(189/206)。
要点
- GPT-4提取RAI检查方式准确率94.2%(194/206),高于GPT-3.5的85.4%(176/206),p < .001。
- GPT-4提取RAI时间范围准确率99.0%(204/206),高于GPT-3.5的95.6%(197/206),p = .02。
- 两种模型提取RAI理由的准确率均为91.7%(189/206)。
核心结论
- 大语言模型可从放射学报告自由文本印象部分提取RAI的可操作细节,GPT-4表现优于GPT-3.5。
研究设计与样本
- 回顾性单中心研究,评估2023年8月五个亚专科(腹部影像、肌肉骨骼影像、神经放射学、核医学、胸部影像)跨模态和照护环境的诊断放射学检查报告;250份经自然语言处理算法识别含RAI的报告随机选取,231份经人工复核确认含RAI,其中25份用于设计提示词,其余206份用于测试GPT-3.5和GPT-4。
诊断标准与关键数值
- GPT-4提取RAI检查方式准确率94.2%(194/206)vs GPT-3.5 85.4%(176/206),p < .001;GPT-4提取RAI身体部位准确率86.9%(179/206)vs GPT-3.5 77.2%(159/206),p = .004;GPT-4提取RAI时间范围准确率99.0%(204/206)vs GPT-3.5 95.6%(197/206),p = .02;两种模型提取RAI理由准确率均为91.7%(189/206)。
临床应用与阅片要点
- 该技术可能代表一种创新方法,有助于及时完成临床必要的放射科医师建议。
局限性
- 原文未报告局限性
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
BACKGROUND. Automated extraction of actionable details of recommendations for additional imaging (RAIs) from radiology reports could facilitate tracking and timely completion of clinically necessary RAIs and thereby potentially reduce diagnostic delays. OBJECTIVE. The purpose of the study was to assess the performance of large language models (LLMs) in extracting actionable details of RAIs from radiology reports. METHODS. This retrospective single-center study evaluated reports of diagnostic radiology examinations performed across modalities and care settings within five subspecialties (abdominal imaging, musculoskeletal imaging, neuroradiology, nuclear medicine, thoracic imaging) in August 2023. Of reports identified by a previously validated natural language processing algorithm to contain an RAI, 250 were randomly selected; 231 of these reports were confirmed to contain an RAI on manual review and formed the study sample. Twenty-five reports were used to engineer a prompt instructing an LLM, when inputted in a report impression containing an RAI, to extract details about the modality, body part, time frame, and rationale of the RAI; the remaining 206 reports were used for testing the prompt in combination with GPT-3.5 and GPT-4. A 4th-year medical student and radiologist from the relevant subspecialty independently classified the LLM outputs as correct versus incorrect for extracting the four actionable details of RAIs in comparison with the report impressions; a third reviewer assisted in resolving discrepancies. Extraction accuracy was summarized and compared between LLMs using consensus assessments. RESULTS. For GPT-3.5 and GPT-4, the two reviewers agreed about classification of LLM output as correct versus incorrect with respect to report impressions for 95.6% and 94.2% for RAI modality, 89.3% and 88.3% for RAI body part, 96.1% and 95.1% for RAI time frame, and 89.8% and 88.8% for RAI rationale, respectively. GPT-4 was more accurate than GPT-3.5 in extracting RAI modality (94.2% [194/206] vs 85.4% [176/206], p < .001), RAI body part (86.9% [179/206] vs 77.2% [159/206], p = .004), and RAI time frame (99.0% [204/206] vs 95.6% [197/206], p = .02). Both LLMs had accuracy of 91.7% (189/206) for extracting RAI rationale. CONCLUSION. LLMs were used to extract actionable details of RAIs from free-text impression sections of radiology reports; GPT-4 outperformed GPT-3.5. CLINICAL IMPACT. The technique could represent an innovative method to facilitate timely completion of clinically necessary radiologist recommendations.
引用
- Li KW, Lacson R, Guenette JP, et al. Use of ChatGPT Large Language Models to Extract Details of Recommendations for Additional Imaging From Free-Text Impressions of Radiology Reports. AJR Am J Roentgenol. 2025. PMID: 39878409. doi:10.2214/AJR.24.32341
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。