使用聚合数据或个体参与者数据的预测模型网状Meta分析——范围综述及报告与实施建议
该范围综述检索PubMed和Embase至2025年9月1日,纳入28项使用网状Meta分析(NMA)评估诊断或预后预测模型性能的研究,其中26项(92.9%)使用聚合数据,仅2项(7.1%)使用个体参与者数据;关键NMA假设和模型性能指标的报告不一致,作者据此提出改进报告与实施的建议。
要点
- 28项NMA中26项(92.9%)使用聚合数据,仅2项(7.1%)使用个体参与者数据。
- 14.3%(n=4)讨论了传递性、相似性或可交换性,53.6%(n=15)检验了一致性。
- 39.3%(n=11)报告区分度指标如C统计量,无研究报告校准指标。
核心结论
- 本范围综述表明NMA方法在预测模型评价中的应用有限且多样,主要依赖聚合数据而非个体参与者数据,对关键NMA假设和模型性能指标考虑不一致。
- 作者提出了针对预测模型验证性能NMA的报告与实施建议。
研究设计与样本
- 范围综述,检索PubMed和Embase至2025年9月1日,纳入使用NMA比较和评估至少三种预测模型的研究,最终纳入28篇文章。
主要发现
- 28项研究中26项(92.9%)使用聚合数据,2项(7.1%)使用个体参与者数据。
- 医院照护是最常见场景(n=22;78.6%),呼吸科(n=7;25.0%)和心脏科(n=5;17.9%)是最常研究的临床领域。
- 14.3%(n=4)讨论了传递性、相似性或可交换性,53.6%(n=15)检验了一致性。
- 60.7%(n=17)报告贝叶斯方法,17.9%(n=5)报告频率学派方法;累积排名曲线下面积是最主要排序方法(n=18)。
- 39.3%(n=11)报告区分度指标如C统计量,无研究报告校准指标;64.3%(n=18)提供敏感度或特异度,无文章报告决策曲线分析等先进决策分析指标。
关键数值与推荐
- 多数NMA网络中纳入5至10个模型,5项NMA分析了超过20个模型。
临床怎么用
- 作者建议改进报告、共享数据和分析代码,并仔细检查假设,以帮助研究者和医生选择最可靠的模型。
局限性
- 原文未报告局限性
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
BACKGROUND: Prediction models are essential in clinical decision-making for estimating the probability of current (diagnosis, screening) or future (prognosis) outcomes. Network meta-analysis (NMA) serves as a powerful tool to compare the performance of multiple prediction models simultaneously. However, there is hardly any guidance on methods and reporting for studies employing NMA to evaluate prediction models. OBJECTIVE: To provide an overview of NMAs assessing prediction model (external validation) performance, regardless of whether they use aggregate data (AD) or individual participant data (IPD). In addition, we offer recommendations for improving the reporting and conduct of NMAs in prediction model research. METHODS: We searched PubMed and Embase up to September 1, 2025, to identify studies that addressed the evaluation of diagnostic or prognostic prediction model performance using NMA. We included articles that employed NMA to compare and assess at least three prediction models. We summarized the identified studies based on, eg, their application (diagnostic vs prognostic), data use (AD vs IPD), medical contexts in which the models were assessed, and evaluation metrics applied (eg, discrimination, calibration, and (re)classification). In addition, we examined the statistical approaches employed, the NMA assumptions (such as consistency, transitivity, and exchangeability), and the ranking methods used for model comparison. RESULTS: After screening 2436 articles, 28 were included. Twenty-six studies (92.9%) used AD, while two (7.1%) used IPD. Hospital care was the most common setting (n = 22; 78.6%), with respirology (n = 7; 25.0%) and cardiology (n = 5; 17.9%) as the most frequently studied clinical domains. Key NMA assumptions were addressed differently across the 28 NMAs: 14.3% (n = 4) discussed transitivity, similarity, or exchangeability, and 53.6% (n = 15) tested for consistency. The statistical approach also varied, with 60.7% of studies (n = 17) reporting Bayesian methods and 17.9% (n = 5) reporting frequentist approaches. Surface under the cumulative ranking was the predominant ranking method (n = 18). Most NMAs included 5 to 10 models in the network, with 5 NMAs analyzing more than 20 models. Performance metrics varied, with 39.3% of studies (n = 11) reporting discrimination measures, such as C statistics, while none reported calibration metrics. Sensitivity or specificity was provided in 64.3% of studies (n = 18), and no articles reported advanced decision-analytic metrics like the decision curve analysis. CONCLUSION: This scoping review highlights the limited and diverse use of NMA methods in evaluating prediction models, with a predominant reliance on aggregate rather than IPD, and inconsistent consideration of key NMA assumptions and model performance metrics. We provide recommendations for the reporting and conduct of an NMA of prediction model validation performance. PLAIN LANGUAGE SUMMARY: Prediction models are tools used in medicine to estimate a patient's risk of developing a disease or experiencing a health outcome. Many different prediction models exist for the same condition, and it can be difficult for doctors and researchers to know which model performs best. One way to compare multiple models is through a statistical method called network meta-analysis (NMA), which is commonly used to compare treatments but has rarely been applied to prediction models. In our study, we reviewed all published NMAs that evaluated prediction models to see how they were conducted and reported. We looked at whether the studies reported important performance measures, such as how well the models could distinguish between patients with and without the outcome (discrimination), and how well the predictions matched actual outcomes (calibration). We also checked if key NMA assumptions were considered and how analyses were conducted. We found that most studies used summary data instead of patient-level data. Many did not report crucial performance measures and rarely checked NMA assumptions. There was also limited transparency in how the models were analyzed, making it difficult for others to reproduce the results. Our findings show that while NMA has great potential to help compare prediction models and identify the most reliable ones, current practice often lacks the detailed reporting needed to make these comparisons fully trustworthy. We recommend better reporting, sharing of data and analysis code, and careful checking of assumptions to help researchers and doctors choose the most reliable models, ultimately improving patient care.
引用
- Yusufujiang M, Damen JAA, Idema DL, et al. Network meta-analysis of prediction models using aggregate or individual participant data - a scoping review and recommendations for reporting and conduct. J Clin Epidemiol. 2025. PMID: 41047088. doi:10.1016/j.jclinepi.2025.112006
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。