指南与共识流行病学与生物统计

在营养研究Meta分析样本中评估缺失证据偏倚风险时评审对共识判断的差异

Variation observed in consensus judgments between pairs of reviewers when assessing the risk of bias due to missing evidence in a sample of published meta-analyses of nutrition research.
指南 / 共识 · J Clin Epidemiol · 2023-12-23

该研究使用ROB-ME工具评估42篇营养研究Meta分析中缺失证据偏倚风险,8名评估者随机分为4对,每对评估21篇,每篇由两对评估。结果显示信号问题原始一致率52%至100%,Gwet AC 0.39至0.76;总体偏倚风险判断原始一致率76%(95%CI 60%至92%),Gwet AC 0.47(95%CI 0.14至0.80),提示评估者间共识判断存在较大差异,需更多培训。

要点

  • ROB-ME工具信号问题的评估者间一致率从52%到100%不等。
  • 总体偏倚风险判断的原始一致率为76%,Gwet AC为0.47。
  • 7篇(17%)Meta分析被至少一对评估者判为低风险偏倚。

核心结论

  • 评估者配对间对ROB-ME信号问题及总体偏倚风险判断存在较大差异,需更多教程和培训以促进工具的一致应用。

研究设计与样本

  • 研究纳入42篇营养研究Meta分析的随机样本,8名评估者随机分为4对,每对评估21篇随机分配的Meta分析,每篇Meta分析由两对评估者评估。

主要发现

  • ROB-ME工具8个信号问题的原始一致率范围为52%至100%,Gwet AC范围为0.39至0.76。
  • 总体偏倚风险判断的原始一致率为76%(95%CI 60%至92%),Gwet AC为0.47(95%CI 0.14至0.80)。
  • 7篇(17%)Meta分析被一对或两对评估者判定为缺失证据偏倚低风险。

局限性

  • 原文未报告局限性。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

OBJECTIVES: To evaluate the risk of bias due to missing evidence in a sample of published meta-analyses of nutrition research using the Risk Of Bias due to Missing Evidence (ROB-ME) tool and determine inter-rater agreement in assessments. STUDY DESIGN AND SETTING: We assembled a random sample of 42 meta-analyses of nutrition research. Eight assessors were randomly assigned to one of four pairs. Each pair assessed 21 randomly assigned meta-analyses, and each meta-analysis was assessed by two pairs. We calculated raw percentage agreement and chance corrected agreement using Gwet's Agreement Coefficient (AC) in consensus judgments between pairs. RESULTS: Across the eight signaling questions in the ROB-ME tool, raw percentage agreement ranged from 52% to 100%, and Gwet's AC ranged from 0.39 to 0.76. For the risk-of-bias judgment, the raw percentage agreement was 76% (95% confidence interval 60% to 92%) and Gwet's AC was 0.47 (95% confidence interval 0.14 to 0.80). In seven (17%) meta-analyses, either one or both pairs judged the risk of bias due to missing evidence as "low risk". CONCLUSION: Our findings indicated substantial variation in assessments in consensus judgments between pairs for the signaling questions and overall risk-of-bias judgments. More tutorials and training are needed to help researchers apply the ROB-ME tool more consistently.

来源:J Clin Epidemiol,查看原文。版权归原作者所有。

引用

  1. Kanukula R, McKenzie JE, Cashin AG, et al. Variation observed in consensus judgments between pairs of reviewers when assessing the risk of bias due to missing evidence in a sample of published meta-analyses of nutrition research. J Clin Epidemiol. 2023. PMID: 38142761. doi:10.1016/j.jclinepi.2023.111244

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。