基于mpMRI结构化文本报告推断前列腺癌病理分级的可行性:四种大语言模型的比较评估
本研究评估四种大语言模型(LLMs)从前列腺多参数MRI结构化文本报告推断ISUP分级组的可重复性和预测准确性,纳入150例患者。LLMs可重复性良好至优秀(Kappa 0.671-0.861),ChatGPT-4.1预测表现和响应时间最优,但准确性(32.7-50.0%)仍低于中高级放射科医生(66.0-76.0%),与初级医生相当。
要点
- 通用LLMs从mpMRI文本报告推断ISUP分级准确性有限,低于中高级放射科医生。
- LLMs可重复性良好至优秀(Kappa 0.671-0.861),ChatGPT-4.1响应最快。
- 临床应用前需针对该任务对通用LLMs进行专项微调。
核心结论
- 通用大语言模型从前列腺mpMRI结构化文本报告推断ISUP分级组具有优秀可重复性,但预测准确性低于中高级放射科医生,与初级医生相当,临床应用前需专项微调。
研究设计与样本
- 可行性研究,纳入150例患者(中位年龄69岁),比较四种大语言模型基于前列腺mpMRI结构化文本报告预测ISUP分级组,每轮预测重复三次评估一致性,并由三名放射科医生独立完成两轮预测及第三轮参考LLMs预测后评估。
诊断标准与关键数值
- 四种LLMs的可重复性Kappa为0.671-0.861,判定为良好至优秀。
- LLMs预测ISUP分级组的准确性为32.7-50.0%,高级放射科医生为72.7-76.0%,中级为66.0-68.7%,初级为59.3-65.3%。
- ChatGPT-4.1响应时间为0.95-17.19秒,在四种LLMs中最短。
临床应用与阅片要点
- LLMs可用于结构化文本报告的ISUP分级初步推断,但结果准确性不足以替代中高级放射科医生的判读。
- 不同ISUP分级组间年龄、PSA水平、前列腺体积、PSA密度和PI-RADS评分存在统计学显著差异,阅片和报告时可结合这些临床信息综合判断。
局限性
- 研究未报告多中心外部验证,样本量为150例,结论的泛化性需进一步验证。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
OBJECTIVES: This study conducted a natural language processing feasibility analysis aimed at comparing four large language models (LLMs) in terms of (a) reproducibility and (b) predictive accuracy for International Society of Urological Pathology Grade Groups (ISUP GGs) based on structured text reports from prostate multiparametric magnetic resonance imaging (mpMRI).
METHODS: The study first used LLMs to perform the initial round of ISUP GGs predictions based solely on the mpMRI text reports. This was followed by a second round of predictions that incorporated clinical information. Each prediction round was repeated three times to assess consistency. Three radiologists independently completed the first two rounds of ISUP GG predictions and then performed a third round of assessment after reviewing the LLMs' predictions. The study recorded the response times.
RESULTS: The study included 150 patients (median age, 69 years). Statistically significant differences were observed among different ISUP GGs in terms of age, PSA levels, prostate volume, PSA density, and PI-RADS scores. The four LLMs demonstrated good to excellent reproducibility (Kappa 0.671-0.861). ChatGPT-4.1 had the shortest response time (0.95-17.19 s). Furthermore, the study found that the accuracy of the LLMs (32.7-50.0%) was significantly lower than that of senior radiologist (72.7-76.0%) and intermediate-level radiologist (66.0-68.7%), but was comparable to that of junior radiologist (59.3-65.3%).
CONCLUSION: General-purpose LLMs demonstrate excellent reproducibility. While ChatGPT-4.1 outperforms other LLMs in ISUP GGs prediction and response time, its predictive accuracy remains inferior to that of intermediate and senior radiologists. Therefore, specific fine-tuning of this technology is necessary before general-purpose LLMs can be applied in clinical practice.
引用
- Niu Y, Shen L, Liu J, et al. Exploring the feasibility of inferring prostate cancer pathological grade from multiparametric MRI text reports using natural language processing: assessment of four large language models. Abdom Radiol (NY). 2026. PMID: 41945149. doi:10.1007/s00261-026-05492-3
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。