多模态大模型肺炎检测的儿科与成人泛化差异
该研究在儿科和成人各1000例胸片(各500例肺炎/正常)上评估三种多模态大模型的零样本肺炎检测,并与领域训练的InceptionV3对照。结果显示大模型在儿科表现明显低于成人,泛化差距大于CNN,主要源于特异度下降。研究提示多模态大模型临床部署前需进行儿科等亚组评估。
要点
- 三种多模态大模型在儿科胸片肺炎检测中MCC均低于成人,存在明显年龄相关泛化差距。
- 领域训练的CNN在儿科(MCC 0.799)和成人(MCC 0.850)均优于全部大模型。
- 大模型儿科表现下降主要由特异度下降驱动,阅片时需警惕儿童胸片假阳性。
核心结论
- 零样本多模态大模型在儿科胸片肺炎检测中呈现较大的年龄相关泛化差距和临床相关错误不对称性。
- 领域训练的CNN在其训练领域内保持稳健,儿科和成人测试中MCC均优于全部大模型。
- 多模态大模型临床部署前必须进行包括儿科人群在内的严格亚组评估。
研究设计与样本
- 研究类型为诊断性能对比研究,在儿科和成人各1000例正位胸片平衡测试集(各500例肺炎/500例正常)上评估。
- 测试大模型为GPT-5.2(OpenAI)、Claude Opus 4.5(Anthropic)和Gemini 2.5 Pro(Google),均为零样本模式。
- 对照组为队列特异性InceptionV3 CNN,分别用剩余开发池图像训练(儿科4715例,成人13863例)并在相同测试集上评估。
- 性能评估指标为Matthews相关系数(MCC)及95%自助法置信区间,域偏移量化为Δ=成人-儿科。
诊断标准与关键数值
- 儿科测试中CNN的MCC为0.799(95% CI 0.766-0.832),高于GPT-5.2的0.484(0.436-0.532)、Claude Opus 4.5的0.470(0.418-0.521)和Gemini 2.5 Pro的0.272(0.224-0.316)。
- 成人测试中所有模型表现均有提升,CNN仍最优,MCC为0.850(95% CI 0.816-0.882)。
- 年龄相关增益中,大模型ΔMCC范围从+0.220(95% CI 0.160-0.281)到+0.466(0.407-0.525),大于CNN的ΔMCC +0.051(0.005-0.098)。
- 大模型从儿科到成人的性能提升主要由特异度增加驱动。
临床应用与阅片要点
- 放射科使用通用多模态大模型辅助肺炎检测时,对儿科胸片应保持高度警惕,其假阳性风险可能较高。
- 对儿童胸片诊断任务,专门训练的领域模型可能比零样本大模型更可靠。
- 不应将成人胸片上表现良好的大模型直接外推至儿科人群,报告系统引入前需验证亚组性能。
局限性
- 原文未报告大模型提示词及采样策略的具体细节,可能影响可重复性和结果解读。
- 原文未列出研究本身的局限性段落,儿科与成人测试集规模相等但开发池样本量差异较大。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
RATIONALE AND OBJECTIVES: Multimodal large language models (LLMs) are increasingly applied to image-based radiology tasks, but their diagnostic accuracy across clinically distinct populations remains poorly characterized. We quantified age-related differences in zero-shot LLM performance for pneumonia detection on pediatric vs. adult chest radiographs and compared generalization gaps with a domain-trained convolutional neural network (CNN) baseline.
MATERIALS AND METHODS: GPT-5.2 (OpenAI), Claude Opus 4.5 (Anthropic), and Gemini 2.5 Pro (Google) were evaluated zero-shot on balanced pediatric and adult test sets of frontal chest radiographs (n = 1000 each; 500 pneumonia/500 normal). Cohort-specific InceptionV3 CNNs were trained on the remaining development-pool images (pediatric n = 4715; adult n = 13,863) and evaluated on the same test sets. Performance was assessed using the Matthews correlation coefficient (MCC) with 95% bootstrap confidence intervals (CIs); domain shift was quantified as Δ = Adult - Pediatric.
RESULTS: In pediatrics, the CNN outperformed all LLMs (MCC 0.799, 95% CI 0.766-0.832) vs. GPT-5.2 (0.484, 0.436-0.532), Claude Opus 4.5 (0.470, 0.418-0.521), and Gemini 2.5 Pro (0.272, 0.224-0.316). In adults, all models improved, but the CNN remained best (MCC 0.850, 0.816-0.882). Age-related gains were larger for LLMs (ΔMCC +0.220 [95% CI 0.160-0.281] to +0.466 [0.407-0.525]) than for the CNN (ΔMCC +0.051 [0.005-0.098]), driven mainly by specificity increases.
CONCLUSION: Zero-shot multimodal LLMs show large age-related generalization gaps and clinically relevant error asymmetries in pediatric chest radiography, whereas a domain-trained CNN remains robust within its training domain. Rigorous subgroup evaluation, including pediatric populations, is essential before clinical deployment of multimodal LLMs.
引用
- Haupt, Maurer. Pediatric vs. Adult Pneumonia Detection: Quantifying Age-related Generalization Gaps in Zero-shot Multimodal Large Language Models. Acad Radiol. 2026. PMID: 42586893. doi:10.1016/j.acra.2026.07.076
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。