文献速递技术与安全

大语言模型转换放射学报告的可读性与准确性:不同阅读年级水平下利益相关者的偏好

Readability versus accuracy in LLM-transformed radiology reports: stakeholder preferences across reading grade levels.
PubMed 文献 · Radiol Med · 2025-09-29

该回顾性研究纳入五家影像模态的500份放射学报告,用Gemini转换为7–17共11个阅读年级水平,由放射科医生、内科医生和普通人按5分制评分准确性、可读性与偏好。结果显示偏好随年级、利益相关者群体、模态和临床场景变化,总体在16年级达峰、17年级下降;准确性是所有群体偏好的最强预测因子。

要点

  • 偏好总体在16年级达峰,17年级下降,普通人群体尤为明显。
  • 低年级提升可读性但增加错误,高年级提升准确性但降低可读性。
  • 多变量分析中准确性是各群体偏好的最强预测因子。

核心结论

  • 较高年级水平因准确性更好而总体更受偏好,范围为12–17年级;进一步升高年级会急剧降低可读性从而限制偏好,提示无监督大语言模型转换存在局限,作者建议采用保留原始报告并加入解释性内容的混合方式以平衡准确性与可读性。

研究设计与样本

  • 回顾性研究,纳入来自学术医院与社区医院、覆盖五种影像模态的500份放射学报告,使用Gemini将报告转换为7–17共11个阅读年级水平,由放射科医生、内科医生和普通人按5分制评分准确性、可读性与偏好。

诊断标准与关键数值

  • 偏好随阅读年级水平、利益相关者群体、影像模态和临床场景而变化,总体在16年级达峰、17年级下降,普通人群体尤为明显。
  • 多变量分析中准确性是所有群体偏好的最强预测因子,放射科医生OR 30.29、内科医生OR 33.05、普通人OR 2.16,均p<0.001。
  • 可读性为次要预测因子,放射科医生OR 2.73、内科医生OR 1.70、普通人OR 2.01,均p<0.001。

局限性

  • 作者在结论中指出这些发现凸显了无监督大语言模型转换的局限性。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

英文原文(摘要)· 点击展开

PURPOSE: To examine how reading grade levels affect stakeholder preferences based on a trade-off between accuracy and readability.

MATERIAL AND METHODS: A retrospective study of 500 radiology reports from academic and community hospitals across five imaging modalities was conducted. Reports were transformed into 11 reading grade levels (7-17) using Gemini. Accuracy, readability, and preference were rated on a 5-point scale by radiologists, physicians, and laypersons. Errors (generalizations, omissions, hallucinations) and potential changes in patient management (PCPM) were identified. Ordinal logistic regression analyzed preference predictors, and weighted kappa measured interobserver reliability.

RESULTS: Preferences varied across reading grade levels depending on stakeholder group, modality, and clinical setting. Overall, preferences peaked at grade 16, but declined at grade 17, particularly among laypersons. Lower reading grades improved readability but increased errors, while higher grades improved accuracy but reduced readability. In multivariable analysis, accuracy was the strongest predictor of preference for all groups (OR: 30.29, 33.05, and 2.16; p <0 .001), followed by readability (OR: 2.73, 1.70, 2.01; p <0.001).

CONCLUSION: Higher-grade levels were generally preferred due to better accuracy, with a range of 12-17. Further increasing grade levels reduced readability sharply, limiting preference. These findings highlight the limitations of unsupervised LLM transformations and suggest the need for hybrid approaches that maintain original reports while incorporating explanatory content to balance accuracy and readability.

来源:Radiol Med,查看原文。版权归原作者所有。

引用

  1. Lee HS, Kim S, Kim S, et al. Readability versus accuracy in LLM-transformed radiology reports: stakeholder preferences across reading grade levels. Radiol Med. 2025. PMID: 41023287. doi:10.1007/s11547-025-02098-5

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。