人在回路大语言模型增强胸部影像诊断推理:放射学专长的影响
该回顾性研究纳入93例胸部影像病例,由5名胸部放射科医生和5名住院医师独立解读并撰写自由文本描述,大语言模型基于描述生成鉴别诊断排序。结果显示,基于医生描述时LLM准确率63.9%,高于基于图像的52.7%;住院医师从LLM辅助中获益更大,但更易接受LLM错误建议。
要点
- 自由文本描述输入LLM的准确率(63.9%)高于直接输入图像(52.7%)。
- 住院医师接受LLM辅助后准确率提升16.1个百分点,高于胸部放射科医生的9.2个百分点。
- 住院医师更易被LLM误导而改错诊断(60.6% vs 32.4%),需警惕自动化偏倚。
核心结论
- 基于文本的LLM辅助工作流可提高阅片者诊断准确率,但受益程度高度依赖阅片者专长。
- 人在回路工作流的效用源于阅片者与LLM的动态交互,受输入表述和输出批判性评估能力影响。
研究设计与样本
- 第一轮阅片者选择最可能诊断并撰写关键发现自由文本描述;LLM(Gemini 3.0 Pro)仅接收描述文本(不含图像),输出5个鉴别诊断排序及前3位解释。
- 第二轮阅片者获得基于自身描述的LLM输出后重新选择诊断;另评估LLM仅输入图像(无描述)的准确率。
- LLM准确率以排名第一的诊断判定;阅片者组间比较采用广义估计方程。
诊断标准与关键数值
- LLM输入图像时准确率52.7%,输入阅片者描述时准确率63.9%。
- 基于胸部放射科医生描述时LLM准确率67.3%,高于住院医师描述的60.4%(p < .001)。
- 胸部放射科医生准确率从第一轮56.3%升至第二轮65.6%,住院医师从42.4%升至58.5%。
- 住院医师准确率提升幅度大于胸部放射科医生(16.1 vs 9.2个百分点;p = .02)。
- 住院医师接受LLM推荐诊断的比率高于胸部放射科医生(73.2% vs 48.9%;p < .001),被误导后改错诊断的比率也更高(60.6% vs 32.4%;p = .009)。
临床应用与阅片要点
- 阅片时可先撰写结构化关键发现文本,再交LLM生成鉴别排序,可能提升诊断表现。
- 低年资医师使用LLM辅助时需警惕过度接受模型建议,应独立审阅图像后再参考输出。
- 胸部放射科医生应关注LLM输出中的误导性推理,避免因模型错误排序而更改正确诊断。
局限性
- 回顾性研究,样本来自 quiz 平台,可能存在选择偏倚。
- 样本量有限(93例),仅涉及单一LLM版本,结果外推需谨慎。
- 未报告阅片者间一致性及LLM输出解释质量的系统评估。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
BACKGROUND. Technical and regulatory constraints limit application of large language models (LLMs) for augmenting diagnostic reasoning in radiology. Reader-mediated text-based workflows may provide a practical alternative. OBJECTIVE. The purpose of this study was to evaluate the impact on diagnostic performance of LLM assistance using reader-generated free-text image descriptions and to assess the effect of reader expertise on this LLM-augmented diagnostic workflow. METHODS. This retrospective study included 93 cases (encompassing radiographic, CT, MRI, and PET/CT images) from the Korean Society of Thoracic Radiology quiz platform from January 2014 to December 2017. Five differential diagnoses (the correct diagnosis and four distracters) were assembled for each case. Ten readers (five thoracic radiologists and five radiology residents) independently interpreted cases. In session 1, readers selected the most likely diagnosis and provided a free-text description of key findings. An LLM (Gemini 3.0 Pro) received as input the free-text description-without case images-and generated output that ranked the five differential diagnoses along with explanatory rationales for the top-three options. In session 2, readers were provided the LLM output from their own free-text description and reselected a most likely diagnosis. LLM performance using images-without free-text descriptions-was also assessed. LLM accuracy was determined using top-ranked diagnoses. Reader groups were compared using generalized estimating equations. RESULTS. LLM accuracy was 52.7% when images were inputted and 63.9% when reader-generated descriptions were inputted. LLM accuracy was greater using descriptions generated by thoracic radiologists than by residents (67.3% vs. 60.4%; p < .001). From session 1 to session 2, accuracy increased from 56.3% to 65.6% for thoracic radiologists and from 42.4% to 58.5% for residents, respectively. Improvement in accuracy between sessions was greater for residents than thoracic radiologists (16.1 vs 9.2 percentage points; p = .02). Residents, compared with thoracic radiologists, had a greater rate of accepting LLM-favored diagnoses (73.2% vs 48.9%; p < .001), including a greater rate of switching to an incorrect diagnosis after a misleading LLM output (60.6% vs 32.4%; p = .009). CONCLUSION. The text-based LLM-assisted workflow yielded improved reader accuracy, although this was heavily influenced by reader expertise. CLINICAL IMPACT. The utility of human-in-the-loop workflows arises from dynamic reader-LLM interactions shaped by the expertise of the operator formulating model inputs and critically evaluating model outputs.
引用
- Song J, Ko H, Han DH, et al. Human-in-the-Loop Large Language Model-Augmented Diagnostic Reasoning in Thoracic Imaging: Impact of Radiologic Expertise. AJR Am J Roentgenol. 2026. PMID: 42160120. doi:10.2214/AJR.26.34999
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。