基于胸部X线片的COVID-19深度学习诊断支持系统:多读者验证研究
研究利用五个公开数据库的胸部X线片训练深度学习系统,并在各类别各100例的测试集上与11名不同年资放射科医生独立阅片比较。系统总体诊断准确率94.3%,高于放射科医生的61.4%±5.3%,对COVID-19肺炎的敏感度97.0%也显著高于医生的53.2%±11.2%。
要点
- 系统总体准确率94.3%,医生为61.4%±5.3%,系统对COVID-19肺炎检测敏感度97.0%。
- 将其他肺炎与COVID-19肺炎合并时,系统准确率95.7%,医生为88.8%。
- 顾问医师与住院医师总体准确率无显著差异,但顾问医师检出更多COVID-19肺炎病例(P=0.008)。
核心结论
- 该系统在胸部X线片上检测COVID-19肺炎表现出稳健的准确率,并在不同培训水平的放射科医生之上。
研究设计与样本
- 多读者验证研究,训练集包括正常7966例、其他肺炎5451例、COVID-19肺炎258例胸部X线片,测试集中每类各100例;11名盲法放射科医生独立阅读测试集,并与基于人工智能的系统分别分析比较。
诊断标准与关键数值
- 系统总体诊断准确率94.3%,医生为61.4%±5.3%;系统对正常、其他肺炎、COVID-19肺炎的敏感度分别为98.0%、88.0%、97.0%,医生分别为85.0%±12.8%、60.1%±12.2%、53.2%±11.2%(P<0.00032)。
临床应用与阅片要点
- 原文未据此给出推荐。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
MATERIALS AND METHODS: Five publicly available databases comprising normal CXR, confirmed COVID-19 pneumonia cases, and other pneumonias were used. After the harmonization of the data, the training set included 7966 normal cases, 5451 with other pneumonia, and 258 CXRs with COVID-19 pneumonia, whereas in the testing data set, each category was represented by 100 cases. Eleven blinded radiologists with various levels of expertise independently read the testing data set. The data were analyzed separately with the newly proposed artificial intelligence-based system and by consultant radiologists and residents, with respect to positive predictive value (PPV), sensitivity, and F-score (harmonic mean for PPV and sensitivity). The χ2 test was used to compare the sensitivity, specificity, accuracy, PPV, and F-scores of the readers and the system.
RESULTS: The proposed system achieved higher overall diagnostic accuracy (94.3%) than the radiologists (61.4% ± 5.3%). The radiologists reached average sensitivities for normal CXR, other type of pneumonia, and COVID-19 pneumonia of 85.0% ± 12.8%, 60.1% ± 12.2%, and 53.2% ± 11.2%, respectively, which were significantly lower than the results achieved by the algorithm (98.0%, 88.0%, and 97.0%; P < 0.00032). The mean PPVs for all 11 radiologists for the 3 categories were 82.4%, 59.0%, and 59.0% for the healthy, other pneumonia, and COVID-19 pneumonia, respectively, resulting in an F-score of 65.5% ± 12.4%, which was significantly lower than the F-score of the algorithm (94.3% ± 2.0%, P < 0.00001). When other pneumonia and COVID-19 pneumonia cases were pooled, the proposed system reached an accuracy of 95.7% for any pathology and the radiologists, 88.8%. The overall accuracy of consultants did not vary significantly compared with residents (65.0% ± 5.8% vs 67.4% ± 4.2%); however, consultants detected significantly more COVID-19 pneumonia cases (P = 0.008) and less healthy cases (P < 0.00001).
CONCLUSIONS: The system showed robust accuracy for COVID-19 pneumonia detection on CXR and surpassed radiologists at various training levels.
引用
- Fontanellaz M, Ebner L, Huber A, et al. A Deep-Learning Diagnostic Support System for the Detection of COVID-19 Using Chest Radiographs: A Multireader Validation Study. Invest Radiol. 2021. PMID: 33259441. doi:10.1097/RLI.0000000000000748
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。