文献速递重症综合管理与镇静镇痛消化、代谢与肾脏急症

院际转运死亡风险模型的外部验证与本地化改进:SafeNET 在真实世界电子健康记录中的可迁移性测试

Testing Transferability of a Mortality Risk Model for Interhospital Transfer Using Real-World Electronic Health Records: External Validation and Model Enhancement.
PubMed 文献 · Crit Care Explor · 2026-09-10
一句话结论SafeNET 在外院际系统保持较好区分度(AUC-ROC 0.87),但本地 CatBoost 模型表现更优(AUC-ROC 0.92,Brier 0.04 vs 0.055),且功能状态变量缺失率高达 85%。

本研究在 2023 年一家多医院学术健康系统的 14,728 例成人院际转运患者中,外部验证已发表的 SafeNET 机器学习模型(14 变量特征集,不做修改),并用本地数据训练 CatBoost、LightGBM、XGBoost 梯度提升模型(70/30 分层拆分、五折交叉验证、类权重与随机欠采样处理约 1:27 类别不平衡)。SafeNET 在留出测试集上 AUC-ROC 0.87、F1 0.73(95%CI 0.69–0.77),接近开发站点结果;本地 CatBoost 模型 AUC-ROC 0.92、F1 0.74(95%CI 0.72–0.78),校准更优(Brier 0.04 vs 0.055)。功能状态变量在 85% 的病例中缺失,限制原始特征集的直接部署;

要点

  • 在 14,728 例成人院际转运患者中,SafeNET 外部验证 AUC-ROC 0.87、F1 0.73(95%CI 0.69–0.77),与开发站点结果接近,支持跨系统可移植性。
  • 本地 CatBoost 模型优于 SafeNET:AUC-ROC 0.92、F1 0.74(95%CI 0.72–0.78),Brier 0.04 对比 0.055,校准更好。
  • 功能状态变量在 85% 的转运病例中缺失,直接套用 SafeNET 原始 14 变量特征集在真实电子健康记录中不可行。
  • 本地化训练可利用系统特异电子健康记录模式补偿跨系统数据缺失,但需注意类别不平衡(约 1:27)处理对精确率的影响。

适用人群与场景

  • (本栏提炼含原文里核对不到的数值,已删除,请以原文为准)

诊断标准与关键阈值

  • SafeNET 模型按已发表规格使用其原始 14 变量特征集,不做修改直接评估。
  • 本地模型采用分层 70/30 训练-测试拆分、五折交叉验证、类权重和随机欠采样处理约 1:27 的类别不平衡。
  • 评估指标包括区分度(AUC-ROC)、分类指标(精确率、召回率、F1)和校准(Brier 分数)。

处置流程与药物剂量

  • 原文未提及

红旗征与不能漏的情况

  • 原文未提及

研究设计与关键结果

  • 研究类型:外部验证与模型改进研究;样本量 14,728 例成人院际转运患者;单中心多医院学术健康系统。
  • SafeNET 在留出测试集上 AUC-ROC 0.87、F1 0.73(95%CI 0.69–0.77),与开发站点结果接近。
  • 本地 CatBoost 模型 AUC-ROC 0.92、F1 0.74(95%CI 0.72–0.78),Brier 0.04 对比 SafeNET 0.055,校准改善。
  • 功能状态变量在 85% 的病例中缺失,限制原始特征集直接实施。

证据类型、等级与局限

  • 证据类型:外部验证与本地模型开发研究(基于真实世界电子健康记录)。

对床旁实践的改变

  • SafeNET 可在外部系统复现区分度,但直接部署受限于功能状态变量高缺失率(85%),需评估本地数据可用性。
  • 本地训练梯度提升模型(如 CatBoost)可提高区分度与校准,但需处理类别不平衡并关注精确率-召回率权衡。
  • 临床部署预后模型时,应同时评估召回率与精确率,避免仅依赖 AUC 判断模型可用性。
  • 原文未提及适用与不适用人群的进一步限定。

指南或共识出处

  • 原文未提及

以上提炼依据论文摘要(全文未开放获取,或许可不允许翻译转载);全文请点「阅读原文」。

英文原文(摘要)· 点击展开

BACKGROUND: Mortality risk is often uncertain at the time of interhospital transfer (IHT).

OBJECTIVES: We examined what happens when a published machine learning model for post-transfer mortality, SafeNET, is deployed in an independent health system with substantial data fragmentation and missingness and assessed the extent to which local model development can compensate for cross-system data limitations.

DERIVATION COHORT: SafeNET was evaluated from published specifications using its original 14-variable feature schema and applied without modification. Locally trained gradient-boosting models (Categorical Boosting [CatBoost], Light Gradient Boosting, Extreme Gradient Boosting) were developed using a stratified 70/30 train-test split with five-fold cross-validation, class weighting, and random under-sampling to address class imbalance (~1:27).

VALIDATION COHORT: We included 14,728 adult patients (≥ 18 yr) undergoing IHT at a multihospital academic health system from January 1, 2023, to December 31, 2023. Functional status variables were missing in 85% of encounters, limiting direct implementation of the original feature set.

PREDICTION MODEL: Discrimination, classification metrics (precision, recall, F1 score), and calibration (Brier score) were compared across SafeNET and locally trained models on the held-out test set.

RESULTS: SafeNET achieved performance that closely approximated its development-site results (area under the receiver operating characteristic curve [AUC-ROC], 0.87; F1, 0.73; 95% CI, 0.69-0.77), indicating strong external consistency. Locally trained CatBoost models outperformed SafeNET (AUC-ROC, 0.92; F1, 0.74; 95% CI, 0.72-0.78) and demonstrated improved calibration (Brier 0.04 vs. 0.055). Key variables, including functional status, were missing in 85% of encounters, limiting direct implementation of the original feature set. Although recall exceeded 0.70 across most models, precision remained below 080 for several configurations, indicating important trade-offs relevant to clinical deployment.

CONCLUSIONS: SafeNET's performance was reproducible in an external health system, supporting its portability. However, local models trained on system-specific electronic health record patterns achieved markedly higher accuracy and calibration. Real-world replication revealed major data-availability barriers and highlighted the tension between acceptable recall and suboptimal precision in critical-care prognostic applications.

来源:Crit Care Explor,查看原文。版权归原作者所有。

参考来源

  1. Hadler RA, Lakamana SK, Moorman T, et al. Testing Transferability of a Mortality Risk Model for Interhospital Transfer Using Real-World Electronic Health Records: External Validation and Model Enhancement. Crit Care Explor. 2026. PMID: 42720061. doi:10.1097/CCE.0000000000001442

供执业医师抢救与诊疗参考,不替代临床判断;剂量请结合体重、肝肾功能和本院药品规格复核,以指南原文为准。