高确定性证据稳定可靠,而中等或更低确定性证据可能同样易于不稳定
该meta-流行病学研究评估证据质量与干预效应估计更新后变化的关系,纳入2016年1月至2021年5月更新的Cochrane系统评价。高确定性证据更新后效应估计无系统改变且精度提高,中等及以下确定性证据的绝对偏差更大、方向或显著性改变更常见,且各确定性等级间无清晰一致差异。
要点
- 高确定性证据更新后效应估计稳定,93%(27/29)维持相同方向和统计显著性水平。
- 中等、低、极低确定性证据更新后绝对偏差更大,方向或显著性改变更常见。
- 证据局限性越多,效应估计偏差越大,方向或显著性越不稳定。
核心结论
- 无方法学缺陷的高质量证据值得信赖且稳定,更新新数据时能提供可靠的干预效应估计。
- 中等及更低质量证据可能同样易于不稳定,无法判断现有效应估计是真实、夸大还是低估。
研究设计与样本
- 采用meta-流行病学研究设计。
- 从Cochrane综述中获取针对同一卫生保健问题的随机试验meta分析证据,这些综述在2016年1月至2021年5月间更新并纳入额外数据。
- 提取首次和最后更新版本中主要结局的效应估计及95%CI,以及相应的GRADE证据确定性评估。
- 共纳入150篇原始Cochrane综述。
主要发现
- 高确定性且无局限性者占19.3%(29/150),更新后效应估计无系统改变(平均ROR 1.00;95%CI 0.99-1.02),与旧估计的中位绝对偏差aROR为1.06(IQR 1.01-1.15),精度提高(中位RoSE 0.87;IQR 0.76-1.00),93%(27/29)维持相同方向和统计显著性水平。
- 中等确定性占30.0%(45/150),低确定性占32.0%(48/150),极低确定性占18.7%(28/150),其更新后中位aROR为1.12至1.33,中位RoSE为0.78至0.86,各确定性等级间无清晰一致差异。
- 中等、低、极低确定性证据中无方向或显著性改变的比例分别为75.6%、64.6%、75.0%。
- 随着局限性数量增加,效应估计偏差增大(aROR:零个局限性1.05,一个1.11,两个1.25,三个1.24),方向或显著性稳定比例下降(无局限性93.2%,一个74.5%,两个68.2%,三个61.5%)。
关键数值与推荐
- 高确定性证据:无局限性,更新后平均ROR 1.00(95%CI 0.99-1.02),中位aROR 1.06(IQR 1.01-1.15),中位RoSE 0.87(IQR 0.76-1.00),93%(27/29)维持相同方向和统计显著性水平。
- 中等确定性证据:中位aROR 1.12至1.33,中位RoSE 0.78至0.86,75.6%无方向或显著性改变。
- 低确定性证据:中位aROR 1.12至1.33,中位RoSE 0.78至0.86,64.6%无方向或显著性改变。
- 极低确定性证据:中位aROR 1.12至1.33,中位RoSE 0.78至0.86,75.0%无方向或显著性改变。
- 局限性数量与aROR:零个1.05,一个1.11,两个1.25,三个1.24;与稳定比例:无局限性93.2%,一个74.5%,两个68.2%,三个61.5%。
临床怎么用
- 原文未据此给出推荐。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
OBJECTIVES: To assess to what extent the overall quality of evidence indicates changes to observe intervention effect estimates when new data become available. METHODS: We conducted a meta-epidemiological study. We obtained evidence from meta-analyses of randomized trials of Cochrane reviews addressing the same health-care question that was updated with inclusion of additional data between January 2016 and May 2021. We extracted the reported effect estimates with 95% confidence intervals (CIs) from meta-analyses and corresponding GRADE (Grading of Recommendations Assessment, Development, and Evaluation) assessments of any intervention comparison for the primary outcome in the first and the last updated review version. We considered the reported overall quality (certainty) of evidence (CoE) and specific evidence limitations (no, serious or very serious for risk of bias, imprecision, inconsistency, and/or indirectness). We assessed the change in pooled effect estimates between the original and updated evidence using the ratio of odds ratio (ROR), absolute ratio of odds ratio (aROR), ratio of standard errors (RoSE), direction of effects, and level of statistical significance. RESULTS: High CoE without limitations characterized 19.3% (n = 29) out of 150 included original Cochrane reviews. The update with additional data did not systematically change the effect estimates (mean ROR 1.00; 95% CI 0.99-1.02), which deviated 1.06-fold from the older estimates (median aROR; interquartile range [IQR]: 1.01-1.15), gained precision (median RoSE 0.87; IQR 0.76-1.00), and maintained the same direction with the same level of statistical significance in 93% (27 of 29) of cases. Lower CoE with limitations characterized 121 original reviews and graded as moderate CoE in 30.0% (45 of 150), low CoE in 32.0% (48 of 150), and very low CoE in 18.7% (28 of 150) reviews. Their update had larger absolute deviations (median aROR 1.12 to 1.33) and larger gains in precision (median RoSE 0.78-0.86) without clear and consistent differences between these categories of CoE. Changes in effect direction or statistical significance were also more common in the lower quality evidence, again with a similar extent across categories (without change in 75.6%, 64.6%, and 75.0% for moderate, low, very low CoE). As limitations increased, effect estimates deviated more (aROR 1.05 with zero, 1.11 with one, 1.25 with two, 1.24 with three limitations) and changes in direction or significance became more frequent (93.2% stable with no limitations, 74.5% with one, 68.2% with two, and 61.5% with three limitations). CONCLUSION: High-quality evidence without methodological deficiencies is trustworthy and stable, providing reliable intervention effect estimates when updated with new data. Evidence of moderate and lower quality may be equally prone to being unstable and cannot indicate if available effect estimates are true, exaggerated, or underestimated.
引用
- Djulbegovic B, Koletsi D, Hozo I, et al. High certainty evidence is stable and trustworthy, whereas evidence of moderate or lower certainty may be equally prone to being unstable. J Clin Epidemiol. 2024. PMID: 38740313. doi:10.1016/j.jclinepi.2024.111392
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。