文献速递胸部DR

胸管及胸膜分离范围对人工智能气胸检测在胸部X线片中的影响

Impact of Confounding Thoracic Tubes and Pleural Dehiscence Extent on Artificial Intelligence Pneumothorax Detection in Chest Radiographs.
PubMed 文献 · Invest Radiol · 2020-01-01

本研究回顾性纳入6434例仰卧位胸部X线片(气胸阳性1652例、阴性4782例),对气胸大小、位置及留置胸管进行放射学标注,评估两种人工智能算法(AI_CheXNet、AI_1.5)的诊断性能。结果显示算法整体AUROC为0.704/0.765,单侧气胸小于1 cm时降至0.666/0.722,大于2 cm时升至0.735/0.818;胸管是强混杂因素,可完全消除算法对无胸管气胸阳性病例与有胸管气胸阴性对照的判别力,而有胸管的大气胸阳性病例对照无胸管阴性病例时AUROC最高达0.875(AI_CheXNet)。

要点

  • AI气胸检测性能随气胸大小变化,小于1 cm单侧气胸AUROC为0.666/0.722,大于2 cm为0.735/0.818。
  • 胸管是强混杂因素,无胸管气胸阳性病例对照有胸管气胸阴性病例时算法判别力完全消除。
  • 有胸管的大气胸阳性病例对照无胸管阴性病例时AUROC最高达0.875(AI_CheXNet)。

核心结论

  • 基于公共数据集训练的现有AI气胸检测算法性能强烈依赖气胸大小,并被胸管等混杂影像特征显著偏倚。
  • 本研究建立的具有临床相关性和放射学标注的基准队列可能对正在进行的算法开发大有裨益。

研究设计与样本

  • 回顾性研究,共纳入6434例仰卧位胸部X线片,其中气胸阳性1652例、气胸阴性4782例。
  • 对仰卧位胸部X线片的气胸大小、气胸位置及留置胸管进行放射学标注。
  • 评估两种AI算法(AI_CheXNet、AI_1.5)的诊断性能,二者均在公开数据集上训练,标签来自自动报告解读。
  • 以受试者工作特征曲线下面积(AUROC)量化算法对气胸检测的判别力,显著性分析基于相应95%置信区间。
  • 进行详细亚组分析以量化气胸大小的影响及留置胸管所致的混杂效应。

诊断标准与关键数值

  • 单侧气胸整体性能AUROC为0.704(AI_1.5)/0.765(AI_CheXNet)。
  • 单侧气胸小于1 cm时AUROC为0.666(AI_1.5)/0.722(AI_CheXNet)。
  • 单侧气胸大于2 cm时AUROC为0.735(AI_1.5)/0.818(AI_CheXNet)。
  • 无胸管气胸阳性病例对照有胸管气胸阴性病例时,算法判别力完全消除。
  • 有胸管的大气胸阳性病例对照无胸管阴性病例时,AUROC最高达0.875(AI_CheXNet)。

临床应用与阅片要点

  • 原文未据此给出推荐。

局限性

  • 原文未报告局限性。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

英文原文(摘要)· 点击展开

OBJECTIVES: We hypothesized that published performances of algorithms for artificial intelligence (AI) pneumothorax (PTX) detection in chest radiographs (CXRs) do not sufficiently consider the influence of PTX size and confounding effects caused by thoracic tubes (TTs). Therefore, we established a radiologically annotated benchmarking cohort (n = 6446) allowing for a detailed subgroup analysis.

MATERIALS AND METHODS: We retrospectively identified 6434 supine CXRs, among them 1652 PTX-positive cases and 4782 PTX-negative cases. Supine CXRs were radiologically annotated for PTX size, PTX location, and inserted TTs. The diagnostic performances of 2 AI algorithms ("AI_CheXNet" [Rajpurkar et al], "AI_1.5" [Guendel et al]), both trained on publicly available datasets with labels obtained from automatic report interpretation, were quantified. The algorithms' discriminative power for PTX detection was quantified by the area under the receiver operating characteristics (AUROC), and significance analysis was based on the corresponding 95% confidence interval. A detailed subgroup analysis was performed to quantify the influence of PTX size and the confounding effects caused by inserted TTs.

RESULTS: Algorithm performance was quantified as follows: overall performance with AUROCs of 0.704 (AI_1.5) / 0.765 (AI_CheXNet) for unilateral PTXs, AUROCs of 0.666 (AI_1.5) / 0.722 (AI_CheXNet) for unilateral PTXs smaller than 1 cm, and AUROCs of 0.735 (AI_1.5) / 0.818 (AI_CheXNet) for unilateral PTXs larger than 2 cm. Subgroup analysis identified TTs to be strong confounders that significantly influence algorithm performance: Discriminative power is completely eliminated by analyzing PTX-positive cases without TTs referenced to control PTX-negative cases with inserted TTs. Contrarily, AUROCs increased up to 0.875 (AI_CheXNet) for large PTX-positive cases with inserted TTs referenced to control cases without TTs.

CONCLUSIONS: Our detailed subgroup analysis demonstrated that the performance of established AI algorithms for PTX detection trained on public datasets strongly depends on PTX size and is significantly biased by confounding image features, such as inserted TTS. Our established, clinically relevant and radiologically annotated benchmarking cohort might be of great benefit for ongoing algorithm development.

来源:Invest Radiol,查看原文。版权归原作者所有。

引用

  1. Rueckel J, Trappmann L, Schachtner B, et al. Impact of Confounding Thoracic Tubes and Pleural Dehiscence Extent on Artificial Intelligence Pneumothorax Detection in Chest Radiographs. Invest Radiol. 2020. PMID: 32694453. doi:10.1097/RLI.0000000000000707

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。