日本成人ICU患者死亡风险预测模型的开发与验证:一种提高医疗质量评价准确性的方法
该研究利用日本全国ICU临床注册数据,回顾性纳入2018年4月1日至2019年3月31日收治的成人ICU患者,对APACHE III-j模型进行三种方式的重校准,并以AUROC、Brier评分和改良Hosmer-Lemeshow检验评价模型性能。共纳入44个ICU的33,557例患者,ICU死亡率为3.8%,住院死亡率为8.1%。原始模型AUROC为0.915、Brier评分为0.062、改良Hosmer-Lemeshow检验P<.001;模型3的AUROC为0.917、Brier评分为0.047、改良Hosmer-Lemeshow检验P=0.84,校准良好。除模型3外,漏斗图均显示过度离散。作者认为模型3可作为JROD模型用于ICU质量监测,但异常值检测的临床有效性需进一步研究。
要点
- 研究人群为日本44个ICU的33,557例成人患者,ICU死亡率为3.8%,住院死亡率为8.1%。
- 原始APACHE III-j模型AUROC为0.915(区分度好),但Brier评分为0.062、改良Hosmer-Lemeshow检验P<.001,校准差且高估死亡风险。
- 模型3(分层广义可加模型)AUROC为0.917、Brier评分为0.047、改良Hosmer-Lemeshow检验P=0.84,校准良好。
- 模型1和模型2的AUROC均为0.915、Brier评分均为0.047,改良Hosmer-Lemeshow检验P分别为<.001和.002,校准仍不佳。
- 除模型3外,漏斗图均显示过度离散;重校准模型的EWMA图有效性通过视觉检查判定。
适用人群与场景
- 2018年4月1日至2019年3月31日期间入住日本ICU的成人患者。
研究设计与关键结果
- 研究类型为回顾性分析,使用日本全国ICU临床注册数据,纳入44个ICU的33,557例成人患者。
- 原始APACHE III-j模型的AUROC为0.915,Brier评分为0.062,改良Hosmer-Lemeshow检验P<.001。
- 模型1(广义线性模型,含APACHE III-j线性预测值的偏移变量)的AUROC为0.915,Brier评分为0.047,改良Hosmer-Lemeshow检验P<.001。
- 模型2(广义线性模型,以APACHE III-j线性预测值为预测变量)的AUROC为0.915,Brier评分为0.047,改良Hosmer-Lemeshow检验P=.002。
- 模型3(分层广义可加模型,以APACHE III-j线性预测值为预测变量)的AUROC为0.917,Brier评分为0.047,改良Hosmer-Lemeshow检验P=.84。
- 除模型3外,其余模型的漏斗图均显示过度离散;重校准模型的EWMA图有效性通过视觉检查判定。
证据类型、等级与局限
- 证据类型:回顾性分析,使用日本全国ICU临床注册数据。
- 局限:作者指出,虽然模型3表现良好并可作为JROD模型用于ICU质量监测,但异常值检测的临床有效性仍需进一步研究。
对床旁实践的改变
- 作者认为,模型3表现良好,可作为JROD模型用于ICU质量监测,但异常值检测的临床有效性需进一步研究。
- 作者认为,这种更新方法可能也适用于其他场景。
提炼说明
- 本条依据论文摘要整理;摘要未涉及「诊断标准与关键阈值」「处置流程与药物剂量」「红旗征与不能漏的情况」「指南或共识出处」,故未列出这些栏目;全文请点「阅读原文」。
英文原文(摘要)· 点击展开
BACKGROUND: The Acute Physiology and Chronic Health Evaluation (APACHE) III-j model is widely used to predict mortality in Japanese intensive care units (ICUs). Although the model's discrimination is excellent, its calibration is poor. APACHE III-j overestimates the risk of death, making its evaluation of healthcare quality inaccurate. This study aimed to improve the calibration of the model and develop a Japan Risk of Death (JROD) model for benchmarking purposes.
METHODS: A retrospective analysis was conducted using a national clinical registry of ICU patients in Japan. Adult patients admitted to an ICU between April 1, 2018, and March 31, 2019, were included. The APACHE III-j model was recalibrated with the following models: Model 1, predicting mortality with an offset variable for the linear predictor of the APACHE III-j model using a generalized linear model; model 2, predicting mortality with the linear predictor of the APACHE III-j model using a generalized linear model; and model 3, predicting mortality with the linear predictor of the APACHE III-j model using a hierarchical generalized additive model. Model performance was assessed with the area under the receiver operating characteristic curve (AUROC), the Brier score, and the modified Hosmer-Lemeshow test. To confirm model applicability to evaluating quality of care, funnel plots of the standardized mortality ratio and exponentially weighted moving average (EWMA) charts for mortality were drawn.
RESULTS: In total, 33,557 patients from 44 ICUs were included in the study population. ICU mortality was 3.8%, and hospital mortality was 8.1%. The AUROC, Brier score, and modified Hosmer-Lemeshow p value of the original model and models 1, 2, and 3 were 0.915, 0.062, and < .001; 0.915, 0.047, and < .001; 0.915, 0.047, and .002; and 0.917, 0.047, and .84, respectively. Except for model 3, the funnel plots showed overdispersion. The validity of the EWMA charts for the recalibrated models was determined by visual inspection.
CONCLUSIONS: Model 3 showed good performance and can be adopted as the JROD model for monitoring quality of care in an ICU, although further investigation of the clinical validity of outlier detection is required. This update method may also be useful in other settings.
参考来源
- Endo H, Uchino S, Hashimoto S, et al. Development and validation of the predictive risk of death model for adult patients admitted to intensive care units in Japan: an approach to improve the accuracy of healthcare quality measures. J Intensive Care. 2021. PMID: 33588956. doi:10.1186/s40560-021-00533-z
供执业医师抢救与诊疗参考,不替代临床判断;剂量请结合体重、肝肾功能和本院药品规格复核,以指南原文为准。