指南与共识流行病学与生物统计

GRADE与CINeMA在评估证据确定性方面显示中度至高度一致性

Grading of Recommendations Assessment, Development, and Evaluation and Confidence in Network Meta-Analysis showed moderate to substantial concordance in the evaluation of certainty of the evidence.
指南 / 共识 · J Clin Epidemiol · 2025-05-05

本研究比较GRADE与CINeMA两种方法评估网状Meta分析证据确定性的评定者间信度、一致性和应用时间。选取4个网状Meta分析中的2个二分类和2个连续结局,13名评定者随机分组应用两种工具。结果显示两种方法评定者间信度多为中度至高度,一致性随比较数量和间接证据增加而下降,应用时间均较长。

要点

  • GRADE评定者间信度Gwet's AC为0.49至0.70,CINeMA为0.02至0.73。
  • 两种方法总体一致性在癫痫发作应答为1,注意缺陷多动障碍症状为0.90,体重减轻为0.68,缺血性卒中为0.42。
  • GRADE评估每个网络的中位时间为160至481分钟,CINeMA为150至330分钟。

核心结论

  • 两种方法评定者间信度多为中度至高度,但CINeMA应用于最大最复杂网络(缺血性卒中)时例外。
  • 小网络且无或极少间接比较时一致性良好,随比较数量和间接证据增加一致性下降。
  • 两种方法应用时间均较长,GRADE用于大网络时尤为明显。

研究设计与样本

  • 从4个网状Meta分析中选取2个二分类结局(癫痫发作应答、缺血性卒中)和2个连续结局(注意缺陷多动障碍症状变化、体重减轻),网络大小、结构和复杂性不同。
  • 13名评定者随机分为4组,每组对1个连续结局和1个二分类结局应用GRADE和CINeMA。
  • 使用Gwet's AC测量总体评价及各工具领域的评定者间信度和一致性,并计算评估每个网络和工具的时间,包括查阅指导文献的时间。

主要发现

  • GRADE评定者间信度范围为0.49(癫痫发作应答)至0.70(缺血性卒中)。
  • CINeMA评定者间信度范围为0.02(缺血性卒中)至0.73(注意缺陷多动障碍症状变化)。
  • 总体一致性:癫痫发作应答为1,注意缺陷多动障碍症状为0.90,体重减轻为0.68,缺血性卒中为0.42。
  • GRADE评估每个网络的中位时间为160至481分钟,CINeMA为150至330分钟。

局限性

  • 原文未报告局限性

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

OBJECTIVES: Grading of Recommendations Assessment, Development, and Evaluation (GRADE) and Confidence in Network Meta-Analysis (CINeMA) are available to assess the confidence in network meta-analysis (NMA) results. They share common aspects, but their operationalization differs. We evaluated interrater reliability (IRR) among assessors, the approaches' concordance, and application time. METHODS: Two dichotomous ("seizure response", "ischemic stroke") and two continuous ("change in attention deficit hyperactivity disorder (ADHD)", "weight loss") outcomes with networks of different sizes, structures and complexities were chosen from four NMAs. Thirteen assessors were randomly assigned to four groups to apply both GRADE and CINeMA on one continuous and one dichotomous outcome. We measured IRR and concordance using Gwet's AC on the overall evaluation and each tool's domain. We calculated time spent evaluating each network and tool, including time to consult the guidance papers. RESULTS: IRR ranged from 0.49 ("seizure response") to 0.70 ("ischemic stroke") with GRADE, from 0.02 ("ischemic stroke") to 0.73 ("change in ADHD symptoms") with CINeMA. Overall concordance was 1, 0.90, 0.68, and 0.42 for "seizure response", "ADHD symptoms", "weight loss", and "ischemic stroke", respectively. The median time spent to assess each network ranged from 160 to 481 minutes with GRADE, from 150 to 330 minutes with CINeMA. CONCLUSION: IRR was moderate to substantial with both approaches, except for CINeMA when applied to the largest and most complex network ("ischemic stroke"). Concordance was good for small networks with no or few indirect comparisons ("seizure response") and decreased as the number of comparisons and indirect evidence increased. Application time was long with both approaches, particularly with GRADE for large networks.

来源:J Clin Epidemiol,查看原文。版权归原作者所有。

引用

  1. Minozzi S, Cinquini M, Arienti C, et al. Grading of Recommendations Assessment, Development, and Evaluation and Confidence in Network Meta-Analysis showed moderate to substantial concordance in the evaluation of certainty of the evidence. J Clin Epidemiol. 2025. PMID: 40334720. doi:10.1016/j.jclinepi.2025.111811

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。