针对具有共同主要终点的诊断准确性研究的多重检验框架
本文针对以敏感度和特异度为共同主要终点的(比较性)III期诊断准确性研究,提出一个多重检验框架,允许在同一评价研究中评估多个诊断模型,并证明该参数同时检验过程渐近地实现家族错误率的强控制。模拟研究表明该策略平均获得更好的最终诊断模型并提高统计功效,并给出贝叶斯方法以确定同时评估的最优模型数。
要点
- 提出多重检验框架,允许在同一III期诊断准确性研究中评估多个诊断模型。
- 参数同时检验过程渐近地实现家族错误率的强控制,并提供点估计与区间估计的多重性校正。
- 模拟研究表明该策略平均获得更好的最终诊断模型并提高统计功效。
核心结论
- 在采用合适的多重比较校正时,在同一评价研究中评估多个有前景的诊断模型具有若干优势。
研究设计与样本
- 提出针对以敏感度和特异度为共同主要终点的(比较性)III期诊断准确性研究的多重检验框架。
- 该框架挑战了通常建议将模型选择与评价严格分开、即在评价研究中只评估单一诊断模型的做法。
- 通过广泛模拟研究评估该多重检验策略的表现。
- 提出贝叶斯方法以确定同时评估的最优模型数,算法在给定模型开发阶段先前(留出)数据的情况下优化预期最终模型性能。
主要发现
- 参数同时检验过程渐近地实现家族错误率的强控制。
- 点估计与区间估计可获得多重性校正。
- 模拟研究表明该多重检验策略平均获得更好的最终诊断模型并提高统计功效。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
Major advances have been made regarding the utilization of machine learning techniques for disease diagnosis and prognosis based on complex and high-dimensional data. Despite all justified enthusiasm, overoptimistic assessments of predictive performance are still common in this area. However, predictive models and medical devices based on such models should undergo a throughout evaluation before being implemented into clinical practice. In this work, we propose a multiple testing framework for (comparative) phase III diagnostic accuracy studies with sensitivity and specificity as co-primary endpoints. Our approach challenges the frequent recommendation to strictly separate model selection and evaluation, that is, to only assess a single diagnostic model in the evaluation study. We show that our parametric simultaneous test procedure asymptotically allows strong control of the family-wise error rate. A multiplicity correction is also available for point and interval estimates. Moreover, we demonstrate in an extensive simulation study that our multiple testing strategy on average leads to a better final diagnostic model and increased statistical power. To plan such studies, we propose a Bayesian approach to determine the optimal number of models to evaluate simultaneously. For this purpose, our algorithm optimizes the expected final model performance given previous (hold-out) data from the model development phase. We conclude that an assessment of multiple promising diagnostic models in the same evaluation study has several advantages when suitable adjustments for multiple comparisons are employed.
引用
- Westphal M, Zapf A, Brannath W, et al. A multiple testing framework for diagnostic accuracy studies with co-primary endpoints. Stat Med. 2022. PMID: 35075684. doi:10.1002/sim.9308
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。