零样本大语言模型自动生成胸部肿瘤病史
本研究回顾性纳入2018年1月至2024年4月胸部肿瘤患者电子健康档案数据,开发基于检索增强生成的零样本大语言模型流程。由GPT-4o mini、o3-mini与GPT-5-mini三种模型自动生成结构化肿瘤病史,并以GPT-5-mini输出作为完整性参考标准进行人工评估。结果显示流程无需人工微调即可较准确总结肿瘤病史,不同模型间完整性与速度存在差异。
要点
- 零样本流程无需人工检索与微调即可生成肿瘤病史摘要。
- GPT-5-mini完整性平均95.5%,准确性最高达97.9%。
- 自动生成每份摘要最快为12.7秒,远低于人工240秒基准。
核心结论
- 零样本检索增强生成的大语言模型流程可准确总结肿瘤病史,无需人工微调或信息检索。
- 不同大语言模型生成的摘要完整性与处理速度存在差异。
研究设计与样本
- 回顾性研究,收集2018年1月至2024年4月癌症中心胸部肿瘤患者电子健康档案数据,共2433例,随机选取50例进行大语言模型总结。
- 20名接受调查的放射科医生选定10项需总结的临床参数。
- 三种符合HIPAA的大语言模型OpenAI GPT-4o mini、o3-mini与GPT-5-mini采用定义的数据模式及零样本提示方法生成结构化摘要。
- 以GPT-5-mini输出作为完整性参考标准,对摘要进行人工评估。
诊断标准与关键数值
- GPT-5-mini流程平均完整性评分最高,为95.5%(95% CI: 94.2, 96.7)。
- GPT-5-mini与o3-mini平均准确性相似,为97.9%(95% CI: 96.8, 99.1)。
- GPT-4o mini每份摘要平均处理时间最快,为12.7秒,与240秒人工基准相比差异有统计学意义(P < .001)。
- 按时间节约推算的年增量收入为每名放射科医生19 525至70 942美元,成本为30.26至32.46美元。
临床应用与阅片要点
- 阅片前可通过零样本流程快速生成肿瘤病史摘要,减少手动病史查阅时间。
- 选用处理速度快的大语言模型可提升报告前准备效率,但应关注模型间完整性差异。
局限性
- 回顾性设计,仅随机纳入50例患者进行大语言模型总结。
- 完整性参考标准基于GPT-5-mini输出,而非独立外部标准。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
Background Gathering oncologic history is necessary but inefficient. Recent efforts to generate clinical summaries using large language models (LLMs) have relied on manual information retrieval, prompt tuning, and subjective evaluation, limiting clinical translation. Purpose To develop and evaluate a zero-shot LLM pipeline that programmatically generates structured oncologic histories using retrieval-augmented generation to ground LLM outputs. Materials and Methods Retrospective electronic health record data were collected for thoracic oncology patients at a cancer center from January 2018 to April 2024. Twenty surveyed radiologists selected 10 clinical parameters for summarization. A retrieval-augmented generation pipeline was developed to filter clinical data. Three Health Insurance Portability and Accountability Act-compliant LLMs-OpenAI's GPT-4o mini, o3-mini, and GPT-5-mini-produced structured summaries using a defined data schema and a zero-shot prompt approach. Summaries were manually evaluated against retrieved clinical text, with completeness reference standard based on the output of GPT-5-mini. Two radiology residents were timed summarizing clinical parameters, establishing a 240-second benchmark. Comparisons between models were conducted using paired t tests for accuracy, completeness, cost, and processing time. Cost-benefit analyses were conducted using manual extraction time, automated summary generation time, and national survey data on radiologist productivity. Results Of 2433 patients, 50 were randomly selected for LLM summarization (mean age, 67.0 years ± 12.5 [SD]; 32 female patients). The GPT-5-mini pipeline achieved the highest mean completeness score of 95.5% (95% CI: 94.2, 96.7) and was similar to o3-mini for a mean accuracy of 97.9% (95% CI: 96.8, 99.1). GPT-4o mini had the fastest processing time per summary, averaging 12.7 seconds compared with the 240-second benchmark (P < .001). Ideally, these time savings project to annual incremental revenue of $19 525 to $70 942 per radiologist, with costs of $30.26 to $32.46. Conclusion The LLM pipeline, which does not require manual fine-tuning or information retrieval, accurately summarizes oncologic history. The completeness and speed of these summaries vary depending on the LLM used. © RSNA, 2026 Supplemental material is available for this article. See also the editorial by Mohajer and Terwiesch in this issue.
引用
- Jani, Mattay, Narra, et al. Zero-shot Thoracic Oncologic History Generation for Radiologists Using Retrieval-augmented Large Language Model Pipeline. Radiology. 2026. PMID: 42301010. doi:10.1148/radiol.251581
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。