通过自动化设计提升ICU风险预测模型对记录偏倚的韧性
该回顾性多中心研究基于eICU研究机构数据库,纳入2017年1月1日至2018年12月31日出院、共计572,985例成人ICU住院,最终队列509,586例,按3:2随机分为开发队列305,590例和验证队列203,996例。研究采用广义相加模型(GAM),纳入反映ICU入科后第一个24小时的临床特征,并针对入科诊断、格拉斯哥昏迷评分(GCS)和极端生命体征的记录偏倚进行设计。在验证队列中,CCOPM对ICU死亡的AUROC为0.925(对比APACHE IVa/IVb 0.88),对住院死亡的AUROC为0.90(对比0.86),并在不同入科诊断、ICU类型和ICU-年亚组中校准更佳;在两个GCS记录发生已知变化的卫生系统、超过120,000例患者的外部队列中,CCOPM的预测较APACHE IVa更稳定。作者认为这些结果支持使用自动化捕获而非经过培训的人员来捕获用于ICU死亡率标杆比较的GCS数据。
要点
- 在203,996例验证队列中,CCOPM对ICU死亡的AUROC为0.925,APACHE IVa/IVb为0.88;对住院死亡的AUROC为0.90 vs 0.86。
- CCOPM在不同入科诊断、ICU类型和不同ICU-年亚组中校准表现均优于APACHE IVa/IVb。
- 在两个GCS记录发生已知变化的卫生系统、超过120,000例患者的外部队列中,CCOPM的预测较APACHE IVa更稳定。
- 模型基于ICU入科后第一个24小时的临床特征,并针对入科诊断、GCS和极端生命体征的记录偏倚进行设计。
- 作者结论:这些结果提供证据支持使用自动化捕获而非经过培训的人员来捕获用于ICU死亡率标杆比较的GCS数据。
适用人群与场景
- 2017年1月1日至2018年12月31日出院、来自参与Philips eICU远程重症监护项目的ICU的成人ICU住院,共572,985例,最终队列509,586例(开发队列305,590例,验证队列203,996例)。
- 适用场景为ICU死亡率标杆比较,基于ICU入科后第一个24小时的临床特征进行风险预测。
研究设计与关键结果
- 研究设计:回顾性、多中心研究,ICU患者按3:2随机分为开发队列和验证队列;采用广义相加模型(GAM),纳入反映ICU入科后第一个24小时的临床特征,并针对入科诊断、GCS和极端生命体征的记录偏倚进行设计。
- 验证队列中,CCOPM对ICU死亡的AUROC为0.925,APACHE IVa/IVb为0.88;对住院死亡的AUROC为0.90,APACHE IVa/IVb为0.86。
- CCOPM在不同入科诊断、ICU类型和不同ICU-年亚组中校准表现均优于APACHE IVa/IVb。
- 在两个GCS记录发生已知变化的卫生系统、超过120,000例患者的外部队列中,CCOPM的预测较APACHE IVa更稳定。
证据类型、等级与局限
- 证据类型:回顾性、多中心研究,基于eICU研究机构数据库。
对床旁实践的改变
- 作者认为,这些结果提供证据支持使用自动化捕获而非经过培训的人员来捕获用于ICU死亡率标杆比较的GCS数据。
提炼说明
- 本条依据论文摘要整理;摘要未涉及「诊断标准与关键阈值」「处置流程与药物剂量」「红旗征与不能漏的情况」「指南或共识出处」,故未列出这些栏目;全文请点「阅读原文」。
英文原文(摘要)· 点击展开
OBJECTIVES: Electronic health records enable automated data capture for risk models but may introduce bias. We present the Philips Critical Care Outcome Prediction Model (CCOPM) focused on addressing model features sensitive to data drift to improve benchmarking ICUs on mortality performance.
DESIGN: Retrospective, multicenter study of ICU patients randomized in 3:2 fashion into development and validation cohorts. Generalized additive models (GAM) with features designed to mitigate biases introduced from documentation of admission diagnosis, Glasgow Coma Scale (GCS), and extreme vital signs were developed using clinical features representing the first 24 hours of ICU admission.
SETTING: eICU Research Institute database derived from ICUs participating in the Philips eICU telecritical care program.
PATIENTS: A total of 572,985 adult ICU stays discharged from the hospital between January 1, 2017, and December 31, 2018, were included, yielding 509,586 stays in the final cohort; 305,590 and 203,996 in development and validation cohorts, respectively.
INTERVENTIONS: None.
MEASUREMENTS AND MAIN RESULTS: Model discrimination was compared against Acute Physiology and Chronic Health Evaluation (APACHE) IVa/IVb models on the validation cohort using the area under the receiver operating characteristic (AUROC) curve. Calibration assessed by actual/predicted ratios, calibration-in-the-large statistics, and visual analysis. Performance metrics were further stratified by subgroups of admission diagnosis and ICU characteristics. Historic data from two health systems with abrupt changes in Glasgow Coma Scale (GCS) documentation were assessed in the year prior to and after data shift. CCOPM outperformed APACHE IVa/IVb for ICU mortality (AUROC, 0.925 vs 0.88) and hospital mortality (AUROC, 0.90 vs 0.86). Better calibration performance was also attained among subgroups of different admission diagnoses, ICU types, and over unique ICU-years. The CCOPM provided more stable predictions compared with APACHE IVa within an external cohort of greater than 120,000 patients from two health systems with known changes in GCS documentation.
CONCLUSIONS: These mortality risk models demonstrated excellent performance compared with APACHE while appearing to mitigate bias introduced through major shifts in GCS documentation at two large health systems. This provides evidence to support using automated capture rather than trained personnel for capture of GCS data used in benchmarking ICUs on mortality performance.
参考来源
- Liu X, Armaignac DL, Becker C, et al. Improving ICU Risk Predictive Models Through Automation Designed for Resiliency Against Documentation Bias. Crit Care Med. 2022. PMID: 36576215. doi:10.1097/CCM.0000000000005750
供执业医师抢救与诊疗参考,不替代临床判断;剂量请结合体重、肝肾功能和本院药品规格复核,以指南原文为准。