偏倚风险评估工具开发中的常见挑战与建议:方法学研究系统综述
本系统综述通过前向引文检索,评估PROBAST、RoB2、ROBINS-I、QUADAS-2等关键偏倚风险评估工具的设计与可用性研究,共纳入13项工具评估。研究发现6/14个主题存在跨工具的共同挑战,包括困难条目、总体偏倚判断、工具复杂性、耗时、用户一致性和未来使用建议,并指出需指导未来工具评估的设计与实施。
要点
- 共纳入13项评估,覆盖PROBAST、RoB2、ROBINS-I和QUADAS-2,其他工具无评估。
- 6/14个主题呈现跨工具共同发现:困难条目、总体判断、可用性、耗时、用户一致性和建议。
- 其余8个主题中7个仅QUADAS-2有发现,限制跨工具比较;1个主题无发现。
核心结论
- 关键偏倚风险评估工具的评估提出了工具使用与开发的共同挑战和建议,可能对工具使用者和开发者有帮助。
- 需要指导以支持未来偏倚风险评估工具评估的设计与实施。
研究设计与样本
- 系统综述方法学研究,从每个工具的主要报告进行前向引文检索,由两名评价者评估纳入。
- 共纳入13项工具评估:PROBAST(3项)、RoB2(3项)、ROBINS-I(4项)、QUADAS-2(3项),其他工具无评估。
- 提取工具特征并使用NVivo进行文档分析,采用演绎与归纳相结合的方法。
主要发现
- 评估在临床主题领域、方法学、偏倚评估方法和工具使用者背景方面存在差异,部分评估存在影响普遍性的局限。
- 6/14个主题存在跨工具共同发现:困难条目(如RoB2/ROBINS-I的「偏离预期干预」域)、总体偏倚判断(PROBAST/ROBINS-I的总体风险计算顾虑)、工具可用性(复杂性顾虑)、完成工具时间(需求不一,如取决于评估结局数量)、用户一致性(各工具不一)、未来使用建议(如预试验)和开发建议(为QUADAS-2/PROBAST增加中间域答案;为所有工具提供更清晰指导)。
- 其余8个主题中7个仅QUADAS-2有发现,限制跨工具比较;1个主题(「问题重组」)无发现。
临床怎么用
- 原文未据此给出推荐。
局限性
- 部分评估存在影响普遍性的局限。
- 其余8个主题中7个仅QUADAS-2有发现,限制跨工具比较。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
OBJECTIVES: To review the findings of studies that have evaluated the design and/or usability of key risk of bias (RoB) tools for the assessment of RoB in primary studies, as categorized by the Library of Assessment Tools and InsTruments Used to assess Data validity in Evidence Synthesis Network (a searchable library of RoB tools for evidence synthesis): Prediction model Risk Of Bias ASessment Tool (PROBAST) , Risk of Bias-2 (RoB2), Risk Of Bias In Non-randomised Studies of Interventions (ROBINS-I), Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2), Quality Assessment of Diagnostic Accuracy Studies-Comparative (QUADAS-C), Quality Assessment of Prognostic Accuracy Studies (QUAPAS), Risk Of Bias in Non-randomised Studies of Exposures (ROBINS-E), and the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) RoB checklist. STUDY DESIGN AND SETTING: Systematic review of methodological studies. We conducted a forward citation search from the primary report of each tool, to identify primary studies that aimed to evaluate the design and/or usability of the tool. Two reviewers assessed studies for inclusion. We extracted tool features into Microsoft Word and used NVivo for document analysis, comprising a mix of deductive and inductive approaches. We summarized findings within each tool and explored common findings across tools. RESULTS: We identified 13 tool evaluations meeting our inclusion criteria: PROBAST (3), RoB2 (3), ROBINS-I (4), and QUADAS-2 (3). We identified no evaluations for the other tools. Evaluations varied in clinical topic area, methodology, approach to bias assessment, and tool user background. Some had limitations affecting generalizability. We identified common findings across tools for 6/14 themes: (1) challenging items (eg, RoB2/ROBINS-I "deviations from intended interventions" domain), (2) overall RoB judgment (concerns with overall risk calculation in PROBAST/ROBINS-I), (3) tool usability (concerns about complexity), (4) time to complete tool (varying demands on time, eg, depending on number of outcomes assessed), (5) user agreement (varied across tools), and (6) recommendations for future use (eg, piloting) and development (add intermediate domain answer to QUADAS-2/PROBAST; provide clearer guidance for all tools). Of the other eight themes, seven only had findings for the QUADAS-2 tool, limiting comparison across tools, and one ("reorganization of questions") had no findings. CONCLUSION: Evaluations of key RoB tools have posited common challenges and recommendations for tool use and development. These findings may be helpful to people who use or develop RoB tools. Guidance is necessary to support the design and implementation of future RoB tool evaluations.
引用
- Tomlinson E, Cooper C, Davenport C, et al. Common challenges and suggestions for risk of bias tool development: a systematic review of methodological studies. J Clin Epidemiol. 2024. PMID: 38670243. doi:10.1016/j.jclinepi.2024.111370
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。