文献速递胸部技术与安全DR

深度学习模型解读胸部X线片:基于放射科医师裁定参考标准与人群校正评估

Chest Radiograph Interpretation with Deep Learning Models: Assessment with Radiologist-adjudicated Reference Standards and Population-adjusted Evaluation.
PubMed 文献 · Radiology · 2019-12-03

本研究开发深度学习模型检测胸片上的气胸、渗出/实变、结节或肿块及骨折四种表现,采用放射科医师裁定的参考标准,在包含759611张图像的多城市医院网络数据集和112120张图像的ChestX-ray14公开数据集上评估,并对DS1行逆概率加权估计人群水平性能。

要点

  • 气胸和渗出/实变的模型性能较高,结节或肿块检测性能相对较低。
  • DS1人群校正后气胸AUC为0.95,结节或肿块AUC为0.72。
  • ChestX-ray14测试集气胸AUC为0.94,结节或肿块为0.91。

核心结论

  • 研究开发了检测胸片上临床相关表现的专家级深度学习模型,采用裁定参考标准并在人群水平估计性能。

研究设计与样本

  • 回顾性研究,使用两个数据集:DS1含759611张图像,来自多城市医院网络;ChestX-ray14含112120张图像。
  • 通过自然语言处理与部分图像专家复核为657954张训练图像提供标签。
  • 测试集分别来自DS1的1818张图像和ChestX-ray14的1962张图像。
  • 参考标准由放射科医师裁定图像复核定义,四名放射科医师复核测试集图像以进行性能比较。
  • 对DS1应用逆概率加权以校正阳性胸片富集并估计人群水平性能。

诊断标准与关键数值

  • DS1人群校正后气胸AUC为0.95(95%CI:0.91,0.99),结节或肿块为0.72(95%CI:0.66,0.77),渗出/实变为0.91(95%CI:0.88,0.93),骨折为0.86(95%CI:0.79,0.92)。
  • ChestX-ray14测试集气胸AUC为0.94(95%CI:0.93,0.96),结节或肿块为0.91(95%CI:0.89,0.93),渗出/实变为0.94(95%CI:0.93,0.95),骨折为0.81(95%CI:0.75,0.86)。

临床应用与阅片要点

  • 原文未据此给出推荐。

局限性

  • 原文未报告局限性。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

英文原文(摘要)· 点击展开

BackgroundDeep learning has the potential to augment the use of chest radiography in clinical radiology, but challenges include poor generalizability, spectrum bias, and difficulty comparing across studies.PurposeTo develop and evaluate deep learning models for chest radiograph interpretation by using radiologist-adjudicated reference standards.Materials and MethodsDeep learning models were developed to detect four findings (pneumothorax, opacity, nodule or mass, and fracture) on frontal chest radiographs. This retrospective study used two data sets. Data set 1 (DS1) consisted of 759 611 images from a multicity hospital network and ChestX-ray14 is a publicly available data set with 112 120 images. Natural language processing and expert review of a subset of images provided labels for 657 954 training images. Test sets consisted of 1818 and 1962 images from DS1 and ChestX-ray14, respectively. Reference standards were defined by radiologist-adjudicated image review. Performance was evaluated by area under the receiver operating characteristic curve analysis, sensitivity, specificity, and positive predictive value. Four radiologists reviewed test set images for performance comparison. Inverse probability weighting was applied to DS1 to account for positive radiograph enrichment and estimate population-level performance.ResultsIn DS1, population-adjusted areas under the receiver operating characteristic curve for pneumothorax, nodule or mass, airspace opacity, and fracture were, respectively, 0.95 (95% confidence interval [CI]: 0.91, 0.99), 0.72 (95% CI: 0.66, 0.77), 0.91 (95% CI: 0.88, 0.93), and 0.86 (95% CI: 0.79, 0.92). With ChestX-ray14, areas under the receiver operating characteristic curve were 0.94 (95% CI: 0.93, 0.96), 0.91 (95% CI: 0.89, 0.93), 0.94 (95% CI: 0.93, 0.95), and 0.81 (95% CI: 0.75, 0.86), respectively.ConclusionExpert-level models for detecting clinically relevant chest radiograph findings were developed for this study by using adjudicated reference standards and with population-level performance estimation. Radiologist-adjudicated labels for 2412 ChestX-ray14 validation set images and 1962 test set images are provided.© RSNA, 2019Online supplemental material is available for this article.See also the editorial by Chang in this issue.

来源:Radiology,查看原文。版权归原作者所有。

引用

  1. Majkowska A, Mittal S, Steiner DF, et al. Chest Radiograph Interpretation with Deep Learning Models: Assessment with Radiologist-adjudicated Reference Standards and Population-adjusted Evaluation. Radiology. 2019. PMID: 31793848. doi:10.1148/radiol.2019191293

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。