在营养研究Meta分析样本中评估缺失证据偏倚风险时评审对共识判断的差异
该研究使用ROB-ME工具评估42篇营养研究Meta分析中缺失证据偏倚风险,8名评估者随机分为4对,每对评估21篇,每篇由两对评估。结果显示信号问题原始一致率52%至100%,Gwet AC 0.39至0.76;总体偏倚风险判断原始一致率76%(95%CI 60%至92%),Gwet AC 0.47(95%CI 0.14至0.80),提示评估者间共识判断存在较大差异,需更多培训。
要点
- ROB-ME工具信号问题的评估者间一致率从52%到100%不等。
- 总体偏倚风险判断的原始一致率为76%,Gwet AC为0.47。
- 7篇(17%)Meta分析被至少一对评估者判为低风险偏倚。
核心结论
- 评估者配对间对ROB-ME信号问题及总体偏倚风险判断存在较大差异,需更多教程和培训以促进工具的一致应用。
研究设计与样本
- 研究纳入42篇营养研究Meta分析的随机样本,8名评估者随机分为4对,每对评估21篇随机分配的Meta分析,每篇Meta分析由两对评估者评估。
主要发现
- ROB-ME工具8个信号问题的原始一致率范围为52%至100%,Gwet AC范围为0.39至0.76。
- 总体偏倚风险判断的原始一致率为76%(95%CI 60%至92%),Gwet AC为0.47(95%CI 0.14至0.80)。
- 7篇(17%)Meta分析被一对或两对评估者判定为缺失证据偏倚低风险。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
OBJECTIVES: To evaluate the risk of bias due to missing evidence in a sample of published meta-analyses of nutrition research using the Risk Of Bias due to Missing Evidence (ROB-ME) tool and determine inter-rater agreement in assessments. STUDY DESIGN AND SETTING: We assembled a random sample of 42 meta-analyses of nutrition research. Eight assessors were randomly assigned to one of four pairs. Each pair assessed 21 randomly assigned meta-analyses, and each meta-analysis was assessed by two pairs. We calculated raw percentage agreement and chance corrected agreement using Gwet's Agreement Coefficient (AC) in consensus judgments between pairs. RESULTS: Across the eight signaling questions in the ROB-ME tool, raw percentage agreement ranged from 52% to 100%, and Gwet's AC ranged from 0.39 to 0.76. For the risk-of-bias judgment, the raw percentage agreement was 76% (95% confidence interval 60% to 92%) and Gwet's AC was 0.47 (95% confidence interval 0.14 to 0.80). In seven (17%) meta-analyses, either one or both pairs judged the risk of bias due to missing evidence as "low risk". CONCLUSION: Our findings indicated substantial variation in assessments in consensus judgments between pairs for the signaling questions and overall risk-of-bias judgments. More tutorials and training are needed to help researchers apply the ROB-ME tool more consistently.
引用
- Kanukula R, McKenzie JE, Cashin AG, et al. Variation observed in consensus judgments between pairs of reviewers when assessing the risk of bias due to missing evidence in a sample of published meta-analyses of nutrition research. J Clin Epidemiol. 2023. PMID: 38142761. doi:10.1016/j.jclinepi.2023.111244
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。