将足踝随机对照试验的统计显著性阈值从0.05降至0.005会产生什么影响?
该研究手工检索2019年至2024年排名前10的骨科期刊和前3的足踝专科期刊,纳入123项足踝随机对照试验。若将显著性阈值从0.05降至0.005,仅19%的主要终点仍显著,58%的显著终点降为「提示性」,维持80%把握度平均需增加69%样本量。
要点
- 在0.005阈值下,123项试验中仅25%维持主要终点显著。
- 原显著终点中58%落入0.005至0.05之间,将被重分类为「提示性」。
- 维持80%把握度需平均增加69%样本量。
核心结论
- 采用0.005阈值将实质影响已发表足踝随机对照试验的解读,超过一半的试验在一个或多个主要终点上会被重分类为仅有「提示性」或无差异发现。
研究设计与样本
- 手工审查2019年至2024年基于2023年两年期刊影响因子排名前10的骨科期刊和前3的足踝专科期刊中聚焦足踝的研究,纳入标准为随机对照试验设计、聚焦足踝疾病或干预、英文发表、报告主要结局的p值,最终123项随机对照试验纳入分析。
主要发现
- 从123项试验中识别281个主要终点,按原文阈值p<0.05有44%(124/281)为统计学显著。
- 124个显著终点中仅42%(52/124)满足p<0.005,58%(72/124)落在0.005至0.05之间,按Benjamin等分类将被重分类为「提示性」而非统计学显著。
- 全部281个终点中仅19%(52/281)在0.005阈值下仍统计学显著。
- 123项试验中25%(31/123)维持主要终点统计学显著。
- 123项试验中54%(66/123)具备足够把握度分析数据,假设α为0.005、把握度80%、效应量取自报告组间差异,维持统计把握度需平均增加69%样本量。
关键数值与推荐
- p值按Ioannidis分类:p<0.005为「统计学显著」,0.005≤p<0.05为「提示性」,p≥0.05为「不显著」。
临床怎么用
- 原文未据此给出推荐。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
BACKGROUND: The threshold for statistical significance (p < 0.05) has been debated in recent years, with proposals to lower it to p < 0.005 to reduce the frequency of papers concluding with false-positive results, which can result in patients receiving overtreatment, and potentiating the problem of nonreplicable results in medical research. However, to our knowledge the impact of modeling that suggestion-in terms of how many studies might be reclassified as no-difference studies and how much larger studies would need to become to implement that suggestion-has not been assessed in orthopaedic surgery. QUESTIONS/PURPOSES: We used randomized trials in foot and ankle research to answer the question: If the threshold for statistical significance were lowered from p < 0.05 to p < 0.005, (1) what proportion of foot and ankle RCTs would be reclassified as no-difference trials under a stricter p value threshold, and (2) how much larger would studies have needed to be to retain or obtain 80% power at the p < 0.005 level? METHODS: We manually reviewed all articles published between 2019 and 2024 in the top 10 ranked orthopaedic journals and the top three foot and ankle-specific journals, both selected based on their 2023 two-year journal Impact Factor, focusing on foot and ankle studies. Studies were included if they met the following criteria: (1) RCT design, (2) focus on foot and ankle conditions or interventions, (3) published in English, and (4) reported p values for primary outcomes. After screening, a total of 123 RCTs met these criteria and were included in the final analysis. Those studies' p values for primary endpoints were extracted and analyzed under both thresholds. If a study had multiple primary endpoints or evaluated the primary endpoint from multiple domains, all p values were included. We categorized p values into three groups based on the classification proposed by Ioannidis: (1) p < 0.005 as "statistically significant," (2) 0.005 ≤ p < 0.05 as "suggestive," and (3) p ≥ 0.05 as "nonsignificant." For studies with sufficient power analysis data, we calculated the required sample size increase needed to maintain 80% statistical power (1 - beta) at an alpha level of 0.005, using the variance reported in the source studies. The effect size (delta) was inferred from the between-group differences reported in each study. Additionally, multivariable logistic regression analysis was performed to identify factors associated with maintaining statistical significance under the p < 0.005 threshold. RESULTS: Among 281 primary endpoints identified from 123 trials, 44% (124 of 281) were statistically significant using the threshold defined in those articles (p < 0.05). Of these significant endpoints, only 42% (52 of 124) of endpoints met the proposed threshold (p < 0.005), whereas 58% (72 of 124) fell between 0.005 and 0.05. Following the classification proposed by Benjamin et al., these endpoints would be reclassified as "suggestive" rather than statistically significant. Overall, only 19% (52 of 281) of all endpoints remained statistically significant under the threshold of 0.005 proposed. Twenty-five percent (31 of 123) of trials maintained statistically significant primary endpoints. Among the 123 trials, 54% (66 of 123) had sufficient power analysis data. Assuming an alpha of 0.005, power of 80%, and effect sizes derived from reported between-group differences, maintaining statistical power under the new threshold would require a mean increase of 69% in the sample size. Logistic regression analysis revealed that extracorporeal shock wave therapy (OR 6.8; p < 0.001) and injection therapy (OR 3.3; p = 0.008) were associated with maintaining significance under the stricter threshold. CONCLUSION: Adopting a threshold of p < 0.005 would substantially impact the interpretation of published foot and ankle RCTs; using that threshold, more than one-half of published RCTs in foot and ankle surgery would have been reclassified as having only "suggestive" or no-difference findings on one or more primary study endpoints. CLINICAL RELEVANCE: Lowering the p value threshold to 0.005 would require larger sample sizes, posing feasibility challenges in foot and ankle surgery because of smaller patient populations. While this shift aims to reduce false-positives, it risks excluding meaningful findings from underpowered studies. More importantly, this debate highlights that no single p value threshold is universally appropriate. Instead of rigidly applying 0.05 or 0.005, researchers should adjust thresholds based on study context-allowing more relaxed thresholds for exploratory studies and stricter ones for high-risk interventions.
引用
- Shimozono Y, Shinya Y, Matsuda S, et al. What Would Be the Effect of Lowering the Threshold of Statistical Significance From 0.05 to 0.005 in Foot and Ankle Randomized Controlled Trials? Clin Orthop Relat Res. 2025. PMID: 41002082. doi:10.1097/CORR.0000000000003689
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。