统计与机器学习模型预测接受免疫检查点抑制剂治疗的晚期非小细胞肺癌患者总生存的区分度、校准度与变量重要性
本研究基于七项临床试验中接受阿替利珠单抗治疗的晚期非小细胞肺癌患者(3203例),比较两种统计模型与六种机器学习模型在预测总生存方面的区分度、校准度与变量重要性,发现各模型区分度多为中等(聚合C指数0.69-0.70),支持向量机区分度差(聚合C指数0.57),校准度数值上XGBoost较优,无单一模型在所有评估队列中持续最优,所有模型均将治疗前中性粒细胞与淋巴细胞比值和ECOG体能状态评分列为前五位重要预测因子。
要点
- 在3203例患者中,统计模型与6种机器学习模型中的5种区分度中等(聚合C指数0.69-0.70),支持向量机区分度差(聚合C指数0.57)。
- 校准度方面聚合图显示各模型大体可比(LASSO除外),XGBoost数值上校准较优。
- 所有模型均将治疗前中性粒细胞与淋巴细胞比值和ECOG体能状态评分列为前五位重要预测因子。
核心结论
- 在区分度或校准度上均无明确最佳模型,但XGBoost模型在数值上可能校准较优。
- 给定模型的性能因评估队列而异,凸显使用多个独立数据集进行模型评估的重要性。
- 所有模型均将治疗前中性粒细胞与淋巴细胞比值和ECOG体能状态评分确定为关键预后因素。
研究设计与样本
- 研究纳入来自七项临床试验的接受阿替利珠单抗治疗的晚期非小细胞肺癌患者,共3203例。
- 比较两种统计模型(Cox比例风险模型和加速失效时间模型)与6种机器学习模型(CoxBoost、极端梯度提升XGBoost、梯度提升机GBMs、随机生存森林、正则化Coxph模型含LASSO、支持向量机SVM)。
- 采用留一项研究嵌套交叉验证框架评估区分度(Harrell一致性指数Cindex)和校准度(集成校准指数ICI及校准图),并用Shapley加性解释SHAP值评估变量重要性。
主要发现
- 两种统计模型和6种机器学习模型中的5种区分度中等且可比,聚合C指数为0.69-0.70;支持向量机区分度差,聚合C指数为0.57。
- 校准度方面,除LASSO外,各模型在聚合校准图上大体可比,XGBoost模型数值上校准较优。
- 各评估队列中个体性能指标存在差异,无单一模型持续优于其他模型。
- 治疗前中性粒细胞与淋巴细胞比值和ECOG体能状态评分在所有模型中均位列前五位最重要预测因子。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
BACKGROUND AND OBJECTIVES: Prognostic models can enhance clinician-patient communication and guide treatment decisions. Numerous machine learning (ML) algorithms are available and offer a novel approach to predicting survival in patients treated with immune checkpoint inhibitors. However, large-scale benchmarking of their performances-particularly in terms of calibration-has not been evaluated across multiple independent cohorts. This study aimed to develop, evaluate, and compare statistical and ML models regarding discrimination, calibration, and variable importance for predicting overall survival across seven clinical trial cohorts of advanced non-small cell lung cancer (NSCLC) undergoing immune checkpoint inhibitor treatment. METHODS: This study included atezolizumab-treated patients with advanced NSCLC from seven clinical trials. We compared two statistical models: Cox proportional-hazard (Coxph) and accelerated failure time models, and 6 ML models: CoxBoost, extreme gradient-boosting (XGBoost), gradient-boosting machines (GBMs), random survival forest, regularized Coxph models (least absolute shrinkage and selection operator [LASSO]), and support vector machines (SVMs). Models were evaluated on discrimination and calibration using a leave-one-study-out nested cross-validation (nCV) framework. Discrimination was assessed using Harrell's concordance index (Cindex), while calibration was assessed using integrated calibration index (ICI) and plot. Variable importance was assessed using Shapley Additive exPlanations (SHAP) values. RESULTS: In a cohort of 3203 patients, the two statistical models and 5 of the 6 ML models demonstrated comparable and moderate discrimination performances (aggregated Cindex: 0.69-0.70), while SVM exhibited poor discrimination (aggregated Cindex: 0.57). Regarding calibration, the models appeared largely comparable in aggregated plots, except for LASSO, although the XGBoost models demonstrated superior calibration numerically. Across the evaluation cohorts, individual performance measures varied and no single model consistently outperforming the others. Pretreatment neutrophil-to-lymphocyte ratios (NLRs) and Eastern Cooperative Oncology Group Performance Status (ECOGPS) were ranked among the top five most important predictors across all models. CONCLUSION: There was no clear best-performing model for either discrimination or calibration, although XGBoost models showed possible superior calibration numerically. Performance of a given model varied across evaluation cohorts, highlighting the importance of model assessment using multiple independent datasets. All models identified pretreatment NLR and ECOGPS as the key prognostic factors.
引用
- Li LX, Hopkins AM, Woodman R, et al. Discrimination, calibration, and variable importance in statistical and machine learning models for predicting overall survival in advanced non-small cell lung cancer patients treated with immune checkpoint inhibitors. J Clin Epidemiol. 2025. PMID: 41276092. doi:10.1016/j.jclinepi.2025.112082
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。