文献速递胸部技术与安全DR

多模态深度学习整合胸部X线片与临床参数:Transformer模型的应用

Multimodal Deep Learning for Integrating Chest Radiographs and Clinical Parameters: A Case for Transformers.
PubMed 文献 · Radiology · 2023-01-01

本回顾性研究从MIMIC数据库和内部数据库提取ICU患者的胸部X线片及临床参数,训练基于Transformer的神经网络,用单一模态或多模态数据诊断最多25种病理状况,并以AUC评估诊断性能;多模态模型对所有病理状况的诊断性能均有所提升。

要点

  • 在MIMIC数据集中,联合胸部X线片与临床参数的平均AUC为0.77,高于单独使用胸部X线片的0.70和单独使用临床参数的0.72。
  • 内部数据集验证了上述发现,多模态模型在所有病理状况上的诊断性能均有所改善。
  • 该Transformer架构可整合影像与非影像数据,用于ICU患者多种疾病的诊断。

核心结论

  • 在ICU患者中,基于影像与非影像数据联合训练的模型在诊断多种疾病时优于仅使用单一类型数据训练的模型。

研究设计与样本

  • 回顾性研究,数据来自MIMIC数据库和内部数据库,内部数据库包含ICU住院患者的胸部X线片与临床参数(2008年1月至2020年12月)。
  • MIMIC数据集划分为训练集n=33893、验证集n=740、测试集n=1909;内部数据集划分为训练集n=28809、验证集n=7203、测试集n=9004。
  • MIMIC数据集共36542例患者(平均年龄63岁±17[SD];男性20567例),内部数据集共45016例患者(平均年龄66岁±16;男性27577例)。
  • 训练基于Transformer的神经网络架构,分别使用仅非影像数据、仅影像数据或多模态数据诊断最多25种病理状况,并以受试者工作特征曲线下面积(AUC)评估诊断性能。

诊断标准与关键数值

  • 在MIMIC数据集中,联合胸部X线片与临床参数的多模态模型平均AUC为0.77(95%CI:0.77,0.78)。
  • 在MIMIC数据集中,仅使用胸部X线片的模型平均AUC为0.70(95%CI:0.69,0.71;P<.001),仅使用临床参数的模型平均AUC为0.72(95%CI:0.72,0.73;P<.001)。
  • 上述发现在内部数据集中得到验证。

临床应用与阅片要点

  • 原文未据此给出推荐。

局限性

  • 原文未报告局限性。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

英文原文(摘要)· 点击展开

Background Clinicians consider both imaging and nonimaging data when diagnosing diseases; however, current machine learning approaches primarily consider data from a single modality. Purpose To develop a neural network architecture capable of integrating multimodal patient data and compare its performance to models incorporating a single modality for diagnosing up to 25 pathologic conditions. Materials and Methods In this retrospective study, imaging and nonimaging patient data were extracted from the Medical Information Mart for Intensive Care (MIMIC) database and an internal database comprised of chest radiographs and clinical parameters inpatients in the intensive care unit (ICU) (January 2008 to December 2020). The MIMIC and internal data sets were each split into training (n = 33 893, n = 28 809), validation (n = 740, n = 7203), and test (n = 1909, n = 9004) sets. A novel transformer-based neural network architecture was trained to diagnose up to 25 conditions using nonimaging data alone, imaging data alone, or multimodal data. Diagnostic performance was assessed using area under the receiver operating characteristic curve (AUC) analysis. Results The MIMIC and internal data sets included 36 542 patients (mean age, 63 years ± 17 [SD]; 20 567 male patients) and 45 016 patients (mean age, 66 years ± 16; 27 577 male patients), respectively. The multimodal model showed improved diagnostic performance for all pathologic conditions. For the MIMIC data set, the mean AUC was 0.77 (95% CI: 0.77, 0.78) when both chest radiographs and clinical parameters were used, compared with 0.70 (95% CI: 0.69, 0.71; P < .001) for only chest radiographs and 0.72 (95% CI: 0.72, 0.73; P < .001) for only clinical parameters. These findings were confirmed on the internal data set. Conclusion A model trained on imaging and nonimaging data outperformed models trained on only one type of data for diagnosing multiple diseases in patients in an ICU setting. © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Kitamura and Topol in this issue.

来源:Radiology,查看原文。版权归原作者所有。

引用

  1. Khader F, Müller-Franzes G, Wang T, et al. Multimodal Deep Learning for Integrating Chest Radiographs and Clinical Parameters: A Case for Transformers. Radiology. 2023. PMID: 37787671. doi:10.1148/radiol.230806

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。