主动脉夹层形态学风险预测指标的观察者间变异:标准化工作流程与培训的作用
本研究评估了专家在CTA上测量主动脉夹层四种形态学风险预测指标的可重复性。研究纳入72例非复杂性Stanford B型主动脉夹层患者,通过三次观察者间变异评估发现,培训后ICC从0.04-0.89提高至0.69-0.97,Bland-Altman偏倚和一致性界限减小,表明标准化工作流程和培训可优化手动测量可靠性。
要点
- 手动测量主动脉夹层形态学特征的可重复性在常规临床评估中较差。
- 标准化工作流程和专项培训可显著提高观察者间一致性至优秀水平。
- 可靠的专家手动标注是构建机器学习模型高质量真实标签的关键。
局限性
- 样本量有限(72例),可能影响普遍性。
- 回顾性设计。
- 评估限于非复杂性Stanford B型主动脉夹层。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
OBJECTIVES: Establishing the reproducibility of expert-derived measurements on CTA exams of aortic dissection is clinically important and paramount for ground-truth determination for machine learning.
METHODS: Four independent observers retrospectively evaluated CTA exams of 72 patients with uncomplicated Stanford type B aortic dissection and assessed the reproducibility of a recently proposed combination of four morphologic risk predictors (maximum aortic diameter, false lumen circumferential angle, false lumen outflow, and intercostal arteries). For the first inter-observer variability assessment, 47 CTA scans from one aortic center were evaluated by expert-observer 1 in an unconstrained clinical assessment without a standardized workflow and compared to a composite of three expert-observers (observers 2-4) using a standardized workflow. A second inter-observer variability assessment on 30 out of the 47 CTA scans compared observers 3 and 4 with a constrained, standardized workflow. A third inter-observer variability assessment was done after specialized training and tested between observers 3 and 4 in an external population of 25 CTA scans. Inter-observer agreement was assessed with intraclass correlation coefficients (ICCs) and Bland-Altman plots.
RESULTS: Pre-training ICCs of the four morphologic features ranged from 0.04 (-0.05 to 0.13) to 0.68 (0.49-0.81) between observer 1 and observers 2-4 and from 0.50 (0.32-0.69) to 0.89 (0.78-0.95) between observers 3 and 4. ICCs improved after training ranging from 0.69 (0.52-0.87) to 0.97 (0.94-0.99), and Bland-Altman analysis showed decreased bias and limits of agreement.
CONCLUSIONS: Manual morphologic feature measurements on CTA images can be optimized resulting in improved inter-observer reliability. This is essential for robust ground-truth determination for machine learning models.
KEY POINTS: • Clinical fashion manual measurements of aortic CTA imaging features showed poor inter-observer reproducibility. • A standardized workflow with standardized training resulted in substantial improvements with excellent inter-observer reproducibility. • Robust ground truth labels obtained manually with excellent inter-observer reproducibility are key to develop reliable machine learning models.
引用
- Willemink, Mastrodicasa, Madani, et al. Inter-observer variability of expert-derived morphologic risk predictors in aortic dissection. Eur Radiol. 2022. PMID: 36029344. doi:10.1007/s00330-022-09056-z
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。