一种半自动化方法有助于评估网络Meta分析中证据确定性:第1部分——直接比较
该研究在两个动态网络Meta分析中实施并评估一种半自动化方法,以辅助评估直接比较的GRADE证据确定性。针对三个GRADE域制定决策规则并生成自动化判断,由两名独立评价者确认或修改。在374个直接比较中,仅纳入单项研究时自动化判断与评价者共识完全一致(100%),纳入多项研究时一致性接近完美。
要点
- 针对研究局限性、间接性和不一致性三个GRADE域制定决策规则并生成自动化判断。
- 单项研究直接比较(n=292)中自动化判断与评价者共识完全一致(100%)。
- 多项研究直接比较中三个GRADE域一致性为99%-100%,总体评级一致性为96%。
核心结论
- 使用既定决策规则的自动化判断在绝大多数GRADE证据确定性评级中与专家判断一致。
研究设计与样本
- 在两个动态网络Meta分析中实施并评估半自动化方法,针对三个GRADE域制定决策规则并生成自动化判断,由两名独立评价者通过在线工具确认或修改。
主要发现
- 在374个直接比较中,仅纳入单项研究时(n=292)自动化判断与评价者共识完全一致(100%)。
- 纳入多项研究时,三个GRADE域的自动化判断与评价者共识一致性为99%-100%,总体评级一致性为96%。
- 评价者间信度接近完美,Gwet's AC1 kappa值范围为96%至100%。
关键数值与推荐
- GRADE证据确定性评估涉及研究局限性、间接性和不一致性三个域,输入包括每项研究的偏倚风险和间接性评级以及异质性指标。
- 间接性评级由两名独立评价者作出并通过共识解决分歧。
临床怎么用
- 原文未据此给出推荐。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
OBJECTIVES: To implement and evaluate a semi-automated approach to facilitate rating the Grading, Recommendation, Assessment, Development and Evaluation (GRADE) certainty of evidence (CoE) for direct comparisons within two living network meta-analysis. METHODS: For each of three GRADE domains (study limitations, indirectness, and inconsistency), decision rules were developed and used to generate automated judgments for each domain and the overall certainty. Inputs included risk of bias and indirectness ratings for each study and measures of heterogeneity. Indirectness ratings were made by two independent reviewers and resolved through consensus. With the help of an online tool (customized to our project), two independent raters viewed forest plots and additional data and could confirm or modify the suggested rating. Disagreements were resolved by consensus. We evaluated inter-rater reliability and accuracy. RESULTS: Across 374 direct comparisons, there was perfect agreement (100%) between the automated judgment and reviewer consensus, when only a single study was available (n = 292), and near-perfect agreement when more than one study was available (99%-100% for the three GRADE domains and 96% for overall rating). Inter-rater reliability was near perfect (Gwet's AC1 kappa score ranging from 96% to 100%). CONCLUSION: Automated judgments using established decision rules agreed with expert judgment for the vast majority of GRADE CoE ratings.
引用
- Kamso MM, Whittle SL, Pardo JP, et al. A semi-automated approach facilitated the assessment of the certainty of evidence in a network meta-analysis: Part 1 - Direct comparisons. J Clin Epidemiol. 2025. PMID: 41421719. doi:10.1016/j.jclinepi.2025.112109
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。