测量诊断中的一致性:研究者实用指南
本文针对诊断评估中一致性研究的方法学问题,以5名产科医生评估20份胎心监护(CTG)为例,说明二分类结局下一致性的定义、计算与解读,并阐述其与可靠性的关系、观察者内与观察者间研究的区别以及统计推断和样本量计算方法,同时提供辅助软件。
要点
- 一致性与可靠性概念不同,多评估者时需明确一致性测度的计算方式。
- 二分类结局可计算总一致率、特异一致率和kappa系数,并应报告置信区间。
- 观察者内与观察者间研究设计不同,统计推断和样本量计算需分别考虑。
核心结论
- 本文旨在阐明二分类结局下一致性测度的计算与解读,帮助研究者提高评估临床测试一致性的研究的方法学质量。
研究设计与样本
- 本文为方法学指导文章,以5名有经验的产科医生评估20份胎心监护(CTG)为例进行演示。
主要发现
- 近期一项关于产时胎心监护可靠性与一致性研究的系统综述指出三个方法学问题:一致性与可靠性概念混淆、多评估者时一致性和可靠性测度的计算不清晰、置信区间很少报告。
- 本文演示了在不同场景下如何定义、计算和解读一致性,并解释了一致性测度与可靠性概念的关系、观察者内与观察者间研究的区别以及统计推断和样本量计算方法。
- 本文重点强调总一致率、特异一致率和kappa系数,并开发了一个shiny应用以支持研究者开展一致性研究。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
Healthcare professionals routinely perform clinical examinations and diagnostic assessments. How the findings of these assessments are interpreted can have significant implications for patient care and outcomes. A recent systematic review on reliability and agreement studies in intrapartum fetal heart rate monitoring highlighted three methodological issues: (1) confusion between the concepts of agreement and reliability, (2) lack of clarity on how agreement and reliability measures are calculated when more than two raters are involved, and (3) confidence intervals seldom reported. This paper aims to clarify how agreement measures can be computed and interpreted when the outcome is binary (e.g., normal/abnormal test result). Using a motivating example in which five experienced obstetricians assessed 20 CTGs, we demonstrate how agreement can be defined, computed, and interpreted in various scenarios. The paper further explains the relationship between agreement measures and the concept of reliability, the distinction between intra- and inter-observer studies, and approaches to make statistical inference and sample size calculations. Particular emphasis is placed on the proportion of agreement, the proportion of specific agreement and kappa coefficients. A shiny application has also been developed to support researchers in their agreement studies. This work completes existing tools such as the Guidelines for Reporting Reliability and Agreement Studies (GRRAS), the Quality Appraisal Tool for Studies of Diagnostic Reliability (QAREL) and STARD guidelines for reporting diagnostic accuracy studies. It is intended to help researchers improve the methodological quality of studies that evaluate the agreement of clinical tests.
引用
- Vanbelle S, Engelhart CH, Blix E, et al. Measuring Agreement in Diagnostics: A Practical Guide for Researchers. Stat Med. 2025. PMID: 41103021. doi:10.1002/sim.70299
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。