当病灶位置未预先指定时,采用第三读者法评估病灶检测的观察者间一致性中Kappa统计量的问题
本文指出,当病灶位置未预先指定时,Cohen kappa统计量会低估观察者间对病灶检测的一致性,并提出两种替代指标:特异 agreement 指数(ISA)和改良kappa。在前列腺癌多读者回顾性研究中,九名读者依据PI-RADS v2在163例患者(110例病例、53例对照)的多参数MRI上检测并评分病灶;所有病灶的平均kappa、改良kappa和ISA分别为30%、55%和57%,指数病灶分别为20%、87%和87%。结论推荐使用ISA评估病灶检测的读者间一致性。
要点
- 当病灶位置未预先指定时,kappa可显著低估读者间对病灶检测的一致性。
- 改良kappa经适当校正机会一致后,与ISA近似等价。
- 前列腺癌数据中,所有病灶平均kappa 30%、改良kappa 55%、ISA 57%;指数病灶分别为20%、87%、87%。
核心结论
- 当病灶位置未预先指定时,应用kappa可能导致读者间对病灶检测一致性的实质性向下偏倚;推荐使用ISA评估病灶检测的读者间一致性。
研究设计与样本
- 方法学论述并应用于一项回顾性多读者研究;九名读者使用PI-RADS v2指南在163例连续患者(110例病例、53例对照)的多参数MRI上检测并评分前列腺癌病灶。
诊断标准与关键数值
- 所有病灶:平均kappa 30%、改良kappa 55%、ISA 57%。
- 指数病灶:平均kappa 20%、改良kappa 87%、ISA 87%。
- 改良kappa经适当校正机会一致量后,与ISA近似等价。
临床应用与阅片要点
- 原文推荐使用ISA评估病灶检测的读者间一致性。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
RATIONALE AND OBJECTIVES: To point out the problems with Cohen kappa statistic and to explore alternative metrics to determine interobserver agreement on lesion detection when locations are not prespecified.
MATERIALS AND METHODS: Use of kappa and two alternative methods, namely index of specific agreement (ISA) and modified kappa, for measuring interobserver agreement on the location of detected lesions are presented. These indices of agreement are illustrated by application to a retrospective multireader study in which nine readers detected and scored prostate cancer lesions in 163 consecutive patients (n = 110 cases, n = 53 controls) using the guideline of Prostate Imaging Reporting and Data System version 2 on multiparametric magnetic resonance imaging.
RESULTS: The proposed modified kappa, which properly corrects for the amount of agreement by chance, is shown to be approximately equivalent to the ISA. In the prostate cancer data, average kappa, modified kappa, and ISA equaled 30%, 55%, and 57%, respectively, for all lesions and 20%, 87%, and 87%, respectively, for index lesions.
CONCLUSIONS: The application of kappa could result in a substantial downward bias in reader agreement on lesion detection when locations are not prespecified. ISA is recommended for assessment of reader agreement on lesion detection.
引用
- Shih JH, Greer MD, Turkbey B, et al. The Problems with the Kappa Statistic as a Metric of Interobserver Agreement on Lesion Detection Using a Third-reader Approach When Locations Are Not Prespecified. Acad Radiol. 2018. PMID: 29551463. doi:10.1016/j.acra.2018.01.030
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。