使用人工智能生成的血管严重程度评分改善早产儿视网膜病变附加病变诊断
该多阅片者诊断准确性影像研究纳入11名早产儿视网膜病变(ROP)专家,对110名婴儿的150次眼检查进行附加病变诊断和血管严重程度评分(VSS)。与基线相比,提供基于人工智能(AI)的VSS后,专家间一致性κ值从0.69升至0.81,平均AUC从0.94升至0.98,平均AUPR从0.86升至0.95。
要点
- AI生成的VSS辅助使ROP专家诊断附加病变的κ值从0.69提高至0.81。
- AI辅助使平均AUC从0.94提升至0.98,平均AUPR从0.86提升至0.95。
- 作者认为,若用于实践,AI-based VSS可减少观察者间变异并标准化ROP治疗。
核心结论
- 向ROP临床医生提供基于AI的血管严重程度测量,与附加病变诊断改善及以附加病变参考标准诊断为参照的连续严重程度标注改善相关。
- 作者认为,若在实践中实施,基于AI的VSS可减少观察者间变异,并可能标准化ROP婴儿的治疗。
研究设计与样本
- 多阅片者诊断准确性影像研究,纳入11名ROP专家,其中9人从业10年或以上。
- 回顾2012年1月至2020年7月期间作为ROP影像与信息学研究的一部分,从早产儿常规ROP筛查中获取的RetCam眼底图像。
- 从所有可用检查中选取110名婴儿的150次眼检查进行分级。
- 使用i-ROP DL系统(Siloam Vision)为每组图像分配基于AI的VSS。
- 临床医生在基线和1个月后分别在有AI-based VSS辅助的情况下诊断附加病变并估计VSS(范围1-9)。
主要发现
- 专家一致性从显著(κ值0.69 [0.59, 0.75])改善至接近完美(κ值0.81 [0.71, 0.86])。
- 平均AUC从0.94(95%CI 0.92-0.96)提高至0.98(95%CI 0.96-0.99),差值0.04(95%CI 0.01-0.06)。
- 平均AUPR从0.86(95%CI 0.81-0.90)提高至0.95(95%CI 0.91-0.97),差值0.09(95%CI 0.03-0.14)。
临床怎么用
- 作者认为,若在实践中实施,基于AI的VSS可减少观察者间变异,并可能标准化ROP婴儿的治疗。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
PURPOSE: To evaluate whether providing clinicians with an artificial intelligence (AI)-based vascular severity score (VSS) improves consistency in the diagnosis of plus disease in retinopathy of prematurity (ROP). DESIGN: Multireader diagnostic accuracy imaging study. PARTICIPANTS: Eleven ROP experts, 9 of whom had been in practice for 10 years or more. METHODS: RetCam (Natus Medical Incorporated) fundus images were obtained from premature infants during routine ROP screening as part of the Imaging and Informatics in ROP study between January 2012 and July 2020. From all available examinations, a subset of 150 eye examinations from 110 infants were selected for grading. An AI-based VSS was assigned to each set of images using the i-ROP DL system (Siloam Vision). The clinicians were asked to diagnose plus disease for each examination and to assign an estimated VSS (range, 1-9) at baseline, and then again 1 month later with AI-based VSS assistance. A reference standard diagnosis (RSD) was assigned to each eye examination from the Imaging and Informatics in ROP study based on 3 masked expert labels and the ophthalmoscopic diagnosis. MAIN OUTCOME MEASURES: Mean linearly weighted κ value for plus disease diagnosis compared with RSD. Area under the receiver operating characteristic curve (AUC) and area under the precision-recall curve (AUPR) for labels 1 through 9 compared with RSD for plus disease. RESULTS: Expert agreement improved significantly, from substantial (κ value, 0.69 [0.59, 0.75]) to near perfect (κ value, 0.81 [0.71, 0.86]), when AI-based VSS was integrated. Additionally, a significant improvement in plus disease discrimination was achieved as measured by mean AUC (from 0.94 [95% confidence interval (CI), 0.92-0.96] to 0.98 [95% CI, 0.96-0.99]; difference, 0.04 [95% CI, 0.01-0.06]) and AUPR (from 0.86 [95% CI, 0.81-0.90] to 0.95 [95% CI, 0.91-0.97]; difference, 0.09 [95% CI, 0.03-0.14]). CONCLUSIONS: Providing ROP clinicians with an AI-based measurement of vascular severity in ROP was associated with both improved plus disease diagnosis and improved continuous severity labeling as compared with an RSD for plus disease. If implemented in practice, AI-based VSS could reduce interobserver variability and could standardize treatment for infants with ROP. FINANCIAL DISCLOSURE(S): Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.
引用
- Coyner AS, Young BK, Ostmo SR, et al. Use of an Artificial Intelligence-Generated Vascular Severity Score Improved Plus Disease Diagnosis in Retinopathy of Prematurity. Ophthalmology. 2024. PMID: 38866367. doi:10.1016/j.ophtha.2024.06.006
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。