危重症死亡预测模型的外部验证:区分度保持但校准度差
该多中心研究对近期范围综述识别的43个危重症死亡预测模型进行外部验证,在SICS-I队列(1,075例)和FINNAKI队列(2,901例)中评估模型性能及重新校准效果。仅11个模型(26%)可外部验证。APACHE II、APACHE IV、SAPS-R和SAPS模型的缩放Brier评分最佳,分别为0.11、0.10、0.10和0.06。SAPS II、APACHE II和APACHE IV区分度最佳,AUC范围为0.63(0.61-0.66)至0.83(0.81-0.85)。大多数模型校准度差,重新校准截距和斜率后至少改善至中等。决策曲线显示APACHE IV和SAPS-R在0-60%阈值概率范围内有正净获益。
要点
- 43个危重症死亡预测模型中仅11个(26%)可进行外部验证,多数模型因变量不可得无法验证。
- APACHE II、APACHE IV、SAPS-R和SAPS模型的缩放Brier评分最佳,分别为0.11、0.10、0.10和0.06。
- SAPS II、APACHE II和APACHE IV区分度最佳,所有模型AUC范围为0.63(0.61-0.66)至0.83(0.81-0.85)。
- 大多数模型校准度差,重新校准截距和斜率后至少改善至中等校准。
- 决策曲线显示APACHE IV和SAPS-R在0-60%阈值概率范围内有正净获益。
适用人群与场景
- SICS-I研究纳入1,075例患者,FINNAKI研究纳入2,901例危重症患者,用于外部验证死亡预测模型。
处置流程与药物剂量
- 使用SICS-I数据对模型进行重新校准,随后在FINNAKI数据中验证。
- 评估内容包括:原始文献中模型重建所需数据、变量可得性、两个独立队列中的模型性能以及重新校准对模型性能的影响。
- 采用缩放Brier评分、受试者工作特征曲线下面积(AUC)、校准截距和斜率以及决策曲线评估模型性能。
研究设计与关键结果
- 研究设计为多中心研究,在SICS-I(1,075例)和FINNAKI(2,901例)两个队列中对43个危重症死亡预测模型进行外部验证。
- 11个模型(26%)可外部验证。
- APACHE II、APACHE IV、SAPS-R和SAPS模型的缩放Brier评分分别为0.11、0.10、0.10和0.06,为最佳。
- SAPS II、APACHE II和APACHE IV区分度最佳;所有模型总体AUC范围为0.63(0.61-0.66)至0.83(0.81-0.85)。
- 大多数模型校准度差,重新校准截距和斜率后至少改善至中等校准。
- 决策曲线显示APACHE IV和SAPS-R在0-60%阈值概率范围内有正净获益。
证据类型、等级与局限
- 证据类型:多中心外部验证研究。
- 局限:原文未报告局限性。
对床旁实践的改变
- 作者认为,APACHE II、APACHE IV和SAPS II的区分能力可接受至优秀,但校准度差;在临床使用这些模型预测个体患者死亡风险时需注意校准问题,重新校准可改善校准度。
提炼说明
- 本条依据论文摘要整理;摘要未涉及「诊断标准与关键阈值」「红旗征与不能漏的情况」「指南或共识出处」,故未列出这些栏目;全文请点「阅读原文」。
英文原文(摘要)· 点击展开
OBJECTIVES: In a recent scoping review, we identified 43 mortality prediction models for critically ill patients. We aimed to assess the performances of these models through external validation.
DESIGN: Multicenter study.
SETTING: External validation of models was performed in the Simple Intensive Care Studies-I (SICS-I) and the Finnish Acute Kidney Injury (FINNAKI) study.
PATIENTS: The SICS-I study consisted of 1,075 patients, and the FINNAKI study consisted of 2,901 critically ill patients.
MEASUREMENTS AND MAIN RESULTS: For each model, we assessed: 1) the original publications for the data needed for model reconstruction, 2) availability of the variables, 3) model performance in two independent cohorts, and 4) the effects of recalibration on model performance. The models were recalibrated using data of the SICS-I and subsequently validated using data of the FINNAKI study. We evaluated overall model performance using various indexes, including the (scaled) Brier score, discrimination (area under the curve of the receiver operating characteristics), calibration (intercepts and slopes), and decision curves. Eleven models (26%) could be externally validated. The Acute Physiology And Chronic Health Evaluation (APACHE) II, APACHE IV, Simplified Acute Physiology Score (SAPS)-Reduced (SAPS-R)' and Simplified Mortality Score for the ICU models showed the best scaled Brier scores of 0.11' 0.10' 0.10' and 0.06' respectively. SAPS II, APACHE II, and APACHE IV discriminated best; overall discrimination of models ranged from area under the curve of the receiver operating characteristics of 0.63 (0.61-0.66) to 0.83 (0.81-0.85). We observed poor calibration in most models, which improved to at least moderate after recalibration of intercepts and slopes. The decision curve showed a positive net benefit in the 0-60% threshold probability range for APACHE IV and SAPS-R.
CONCLUSIONS: In only 11 out of 43 available mortality prediction models, the performance could be studied using two cohorts of critically ill patients. External validation showed that the discriminative ability of APACHE II, APACHE IV, and SAPS II was acceptable to excellent, whereas calibration was poor.
参考来源
- Cox EGM, Wiersema R, Eck RJ, et al. External Validation of Mortality Prediction Models for Critical Illness Reveals Preserved Discrimination but Poor Calibration. Crit Care Med. 2022. PMID: 36378565. doi:10.1097/CCM.0000000000005712
供执业医师抢救与诊疗参考,不替代临床判断;剂量请结合体重、肝肾功能和本院药品规格复核,以指南原文为准。