基于大语言模型从盆腔MRI报告自动计算卵巢-附件报告和数据系统MRI评分的准确性
本回顾性单中心研究纳入284份盆腔MRI报告(372个附件病变),评估两种大语言模型策略自动计算O-RADS MRI评分的准确性。混合策略(GPT-4分类特征+确定性公式)准确率97%(168/173),优于仅用大语言模型策略的90%(155/173;P=.006),也超过原始报告放射科医师的88%(139/158;P=.004)。
要点
- 混合策略自动计算O-RADS MRI评分准确率97%,优于仅用大语言模型策略的90%(P=.006)。
- 混合策略准确率超过原始报告放射科医师(97% vs 88%;P=.004)。
- 在O-RADS实施前的183个病变中,混合策略准确率95%,优于仅用大语言模型策略的87%(P=.01)。
核心结论
- 结合大语言模型特征分类与确定性元素的混合策略能准确从报告描述中自动分配O-RADS MRI评分,优于仅用大语言模型策略和原始报告放射科医师。
研究设计与样本
- 回顾性单中心研究,来自大型四级癌症中心,纳入2021年7月至2023年10月连续钆螯合物增强盆腔MRI报告284份(284例女性患者,平均年龄53.2岁±16.3[SD],372个附件病变),另随机选取2018年1月至2019年10月(O-RADS MRI实施前)报告140份(183个病变)用于额外测试;参考标准由放射科医师解读报告确定。
诊断标准与关键数值
- 混合模型分配O-RADS MRI评分准确率97%(168/173),仅用大语言模型策略90%(155/173),P=.006;对有原始O-RADS评分的病变,混合模型准确率97%(153/158)vs 原始报告放射科医师88%(139/158),P=.004;O-RADS实施前183个病变中,混合模型准确率95%(173/183)vs 仅用大语言模型策略87%(159/183),P=.01。
临床应用与阅片要点
- 作者结论认为混合策略能准确从报告描述中自动分配O-RADS MRI评分,准确率超过仅用大语言模型策略和原始报告放射科医师。
局限性
- 原文未报告局限性
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
Background Ovarian-Adnexal Reporting and Data System (O-RADS) for MRI helps assign malignancy risk, but radiologist adoption is inconsistent. Automatic assignment of O-RADS scores from reports could increase adoption and accuracy. Purpose To evaluate the accuracy of large language models (LLMs), after strategic optimization, for automatically calculating O-RADS scores from reports. Materials and Methods This retrospective single-center study from a large quaternary care cancer center included consecutive gadolinium chelate-enhanced pelvic MRI reports with at least one assigned O-RADS score from July 2021 to October 2023. Reports from January 2018 to October 2019 (before O-RADS MRI implementation) were randomly selected for additional testing. Reference standard O-RADS scores were determined by radiologists interpreting reports. After prompt optimization using a subset of reports, two LLM-based strategies were evaluated: few-shot learning with GPT-4 (version 0613; OpenAI) prompted with O-RADS rules ("LLM only") and a hybrid strategy leveraging GPT-4 to classify features fed into a deterministic formula ("hybrid"). Accuracy of each model and originally reported scores were calculated and compared using the McNemar test. Results A total of 284 reports from 284 female patients (mean age, 53.2 years ± 16.3 [SD]) with 372 adnexal lesions were included: 10 reports in the training set (16 lesions), 134 reports in the internal test set 1 (173 lesions; 158 O-RADS assigned), and 140 reports in internal test set 2 (183 lesions). For assigning O-RADS MRI scores, the hybrid model accuracy (97%; 168 of 173) outperformed LLM-only model (90%; 155 of 173; P = .006). For lesions with an originally reported O-RADS score, hybrid model accuracy exceeded that of reporting radiologists (97% [153 of 158] vs 88% [139 of 158]; P = .004). Hybrid model also outperformed LLM-only model for 183 lesions from before O-RADS implementation (95% [173 of 183] vs 87% [159 of 183], respectively; P = .01). Conclusion A hybrid LLM-based application, combining LLM feature classification with deterministic elements, accurately assigned O-RADS MRI scores from report descriptions, exceeding both an LLM-only strategy and the original reporting radiologist. © RSNA, 2025 Supplemental material is available for this article.
引用
- Bhayana R, Jajodia A, Chawla T, et al. Accuracy of Large Language Model-based Automatic Calculation of Ovarian-Adnexal Reporting and Data System MRI Scores from Pelvic MRI Reports. Radiology. 2025. PMID: 40167432. doi:10.1148/radiol.241554
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。