CT中结构手动勾画的观察者间变异性
11名放射科医生在490层临床CT上手动勾画3193个轮廓(肝肿瘤896、肺肿瘤1085、肾轮廓434、脑血肿497),量化观察者间勾画变异性。不同结构体积重叠变异性均值8.8%–20.8%,随机两名观察者间为5%–57%;2、3、5名观察者组分别捕获37%、53%、72%的变异性,8名观察者捕获75%–94%。作者结论:手动勾画变异性大,2名甚至3名观察者不足以建立完整变异范围。
要点
- 肝肿瘤、肺肿瘤、肾轮廓、脑血肿手动勾画的观察者间体积重叠变异性均值分别为17.8%、20.8%、8.8%、18%。
- 随机两名观察者间勾画体积重叠变异性为5%–57%,两名甚至三名观察者仅捕获部分变异。
- 38.5%的勾画不一致源于单一观察者与其他观察者不一致的部分。
核心结论
- 不同结构和观察者的手动勾画变异性大,且在不同结构和病理类型间跨度宽。
- 两名甚至三名观察者可能不足以建立完整的观察者间变异范围。
- 观察者间手动勾画变异性是建立放射科医生培训和评估参考标准及评估自动分割算法所必需的。
研究设计与样本
- 研究类型为观察者间变异性量化研究,11名放射科医生参与手动勾画。
- 在490层临床CT扫描上共勾画3193个轮廓,包括肝肿瘤896个、肺肿瘤1085个、肾轮廓434个、脑血肿497个。
- 采用标准体积指标和新的组间指标对勾画进行对比分析,以量化观察者间勾画变异性。
诊断标准与关键数值
- 两名观察者间勾画平均体积重叠变异性及范围:肝肿瘤17.8%[-5.8%,+7.2%]、肺肿瘤20.8%[-8.8%,+10.2%]、肾轮廓8.8%[-0.8%,+1.2%]、脑血肿18%[-6.0%,+6.0%]。
- 任意两名随机选择的观察者间平均勾画体积重叠变异性为5%–57%。
- 2名、3名和5名观察者组捕获的平均变异性分别为37%、53%和72%;8名观察者捕获总变异性的75%–94%。
- 所有病例中,38.5%的勾画不一致源于单一观察者勾画的部分与其他观察者不一致。
- 基于专业水平的观察者间勾画变异性无统计学差异。
临床应用与阅片要点
- 手动勾画结构时需注意观察者间变异性较大,不同结构和病理类型间变异范围宽。
- 仅由2名甚至3名观察者进行勾画可能不足以覆盖完整的观察者间变异范围。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
PURPOSE: To quantify the inter-observer variability of manual delineation of lesions and organ contours in CT to establish a reference standard for volumetric measurements for clinical decision making and for the evaluation of automatic segmentation algorithms.
MATERIALS AND METHODS: Eleven radiologists manually delineated 3193 contours of liver tumours (896), lung tumours (1085), kidney contours (434) and brain hematomas (497) on 490 slices of clinical CT scans. A comparative analysis of the delineations was then performed to quantify the inter-observer delineation variability with standard volume metrics and with new group-wise metrics for delineations produced by groups of observers.
RESULTS: The mean volume overlap variability values and ranges (in %) between the delineations of two observers were: liver tumours 17.8 [-5.8,+7.2]%, lung tumours 20.8 [-8.8,+10.2]%, kidney contours 8.8 [-0.8,+1.2]% and brain hematomas 18 [-6.0,+6.0] %. For any two randomly selected observers, the mean delineation volume overlap variability was 5-57%. The mean variability captured by groups of two, three and five observers was 37%, 53% and 72%; eight observers accounted for 75-94% of the total variability. For all cases, 38.5% of the delineation non-agreement was due to parts of the delineation of a single observer disagreeing with the others. No statistical difference was found for the delineation variability between the observers based on their expertise.
CONCLUSION: The variability in manual delineations for different structures and observers is large and spans a wide range across a variety of structures and pathologies. Two and even three observers may not be sufficient to establish the full range of inter-observer variability.
KEY POINTS: • This study quantifies the inter-observer variability of manual delineation of lesions and organ contours in CT. • The variability of manual delineations between two observers can be significant. Two and even three observers capture only a fraction of the full range of inter-observer variability observed in common practice. • Inter-observer manual delineation variability is necessary to establish a reference standard for radiologist training and evaluation and for the evaluation of automatic segmentation algorithms.
引用
- Joskowicz L, Cohen D, Caplan N, et al. Inter-observer variability of manual contour delineation of structures in CT. Eur Radiol. 2018. PMID: 30194472. doi:10.1007/s00330-018-5695-5
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。