人工智能在急诊胸片报告准确性中的附加临床价值:无24/7放射科覆盖场景下的验证研究
本研究回顾性纳入563张急诊胸片,由3名board-certified放射科医师(BCRs)、3名放射科住院医师(RRs)和3名有急诊经验的非放射科住院医师(NRRs)分别评估4种病变(胸腔积液、气胸、疑似肺炎的实变、肺部病灶),并以BCRs不同敏感度的置信评分构建4种二分类参考标准。AI系统在气胸、胸腔积液和肺部病灶上AUC分别为0.940、0.953、0.883,与经验丰富的RRs相当并显著优于NRRs;对实变检测AI系统AUC为0.847,达到NRRs共识水平。
要点
- AI系统在气胸、胸腔积液、肺部病灶检测上AUC达0.940、0.953、0.883,与放射科住院医师相当。
- 有急诊经验的非放射科住院医师诊断准确性随参考标准敏感度提高而下降,AI可弥补此差距。
- 对疑似肺炎实变,AI系统AUC为0.847,仅达非放射科住院医师共识水平,需谨慎解读。
局限性
- 回顾性研究设计
- AI系统训练数据来自不同临床机构的非公开数据,存在外部验证但样本量有限
- 参考标准基于BCRs置信评分转换,非临床最终诊断
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
OBJECTIVES: Chest radiographs (CXRs) are commonly performed in emergency units (EUs), but the interpretation requires radiology experience. We developed an artificial intelligence (AI) system (precommercial) that aims to mimic board-certified radiologists' (BCRs') performance and can therefore support non-radiology residents (NRRs) in clinical settings lacking 24/7 radiology coverage. We validated by quantifying the clinical value of our AI system for radiology residents (RRs) and EU-experienced NRRs in a clinically representative EU setting.
MATERIALS AND METHODS: A total of 563 EU CXRs were retrospectively assessed by 3 BCRs, 3 RRs, and 3 EU-experienced NRRs. Suspected pathologies (pleural effusion, pneumothorax, consolidations suspicious for pneumonia, lung lesions) were reported on a 5-step confidence scale (sum of 20,268 reported pathology suspicions [563 images × 9 readers × 4 pathologies]) separately by every involved reader. Board-certified radiologists' confidence scores were converted into 4 binary reference standards (RFSs) of different sensitivities. The RRs' and NRRs' performances were statistically compared with our AI system (trained on nonpublic data from different clinical sites) based on receiver operating characteristics (ROCs) and operating point metrics approximated to the maximum sum of sensitivity and specificity (Youden statistics).
RESULTS: The NRRs lose diagnostic accuracy to RRs with increasingly sensitive BCRs' RFSs for all considered pathologies. Based on our external validation data set, the AI system/NRRs' consensus mimicked the most sensitive BCRs' RFSs with areas under ROC of 0.940/0.837 (pneumothorax), 0.953/0.823 (pleural effusion), and 0.883/0.747 (lung lesions), which were comparable to experienced RRs and significantly overcomes EU-experienced NRRs' diagnostic performance. For consolidation detection, the AI system performed on the NRRs' consensus level (and overcomes each individual NRR) with an area under ROC of 0.847 referenced to the BCRs' most sensitive RFS.
CONCLUSIONS: Our AI system matched RRs' performance, meanwhile significantly outperformed NRRs' diagnostic accuracy for most of considered CXR pathologies (pneumothorax, pleural effusion, and lung lesions) and therefore might serve as clinical decision support for NRRs.
引用
- Rudolph J, Huemmer C, Ghesu FC, et al. Artificial Intelligence in Chest Radiography Reporting Accuracy: Added Clinical Value in the Emergency Unit Setting Without 24/7 Radiology Coverage. Invest Radiol. 2022. PMID: 34352804. doi:10.1097/RLI.0000000000000813
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。