专有恶化指数模型的验证及其在住院成人中的表现
这项回顾性预后研究纳入美国中西部8家不同规模与类型的医院(学术、社区、城市、农村)中2021年1月1日至5月31日收治的13737例18岁及以上住院患者,评估每15分钟生成的DTI预测值。恶化定义为住院期间发生机械通气、ICU转科或死亡,发生率为10.3%(n=1436)。DTI在观察水平的AUROC为0.759(95%CI 0.756-0.762)、就诊水平为0.685(95%CI 0.671-0.700),对应AUPRC分别为0.039(95%CI 0.037-0.040)和0.248(95%CI 0.227-0.273)。偏倚在不同人口学亚组间存在差异,自认美洲印第安人或阿拉斯加原住民的患者偏倚指标差14.0%,选择不披露族裔者差19.0%。作者认为DTI预测能力中等,亚组表现差异提示需要更透明的模型训练数据并强化对外部开发预测模型的本地验证。
要点
- DTI预测住院恶化的观察水平AUROC为0.759(95%CI 0.756-0.762),就诊水平AUROC为0.685(95%CI 0.671-0.700)。
- 观察水平AUPRC仅0.039(95%CI 0.037-0.040),就诊水平AUPRC为0.248(95%CI 0.227-0.273)。
- 恶化(机械通气、ICU转科或死亡)发生率为10.3%(n=1436)。
- 偏倚指标在自认美洲印第安人或阿拉斯加原住民的患者中差14.0%,在拒绝披露族裔者中差19.0%。
- 作者认为DTI预测能力中等,亚组表现差异提示需要更透明的模型训练数据并强化本地验证。
适用人群与场景
- 美国中西部8家不同规模与类型的医院(学术、社区、城市、农村)中2021年1月1日至5月31日收治的18岁及以上住院患者,共13737例、14834次住院。
诊断标准与关键阈值
- 恶化定义为住院期间发生机械通气、ICU转科或死亡中的任意一项。
研究设计与关键结果
- 回顾性预后研究,纳入8家医院13737例患者、14834次住院,共生成5143513次DTI预测(每15分钟一次)。
- 13918次就诊中,患者平均(SD)年龄60.3(19.2)岁;7636例(54.9%)为女性,11345例(81.5%)为白人,12392例(89.0%)为非西班牙裔或拉丁裔的其他族裔。
- 恶化发生率为10.3%(n=1436)。
- 观察水平AUROC为0.759(95%CI 0.756-0.762),就诊水平AUROC为0.685(95%CI 0.671-0.700)。
- 观察水平AUPRC为0.039(95%CI 0.037-0.040),就诊水平AUPRC为0.248(95%CI 0.227-0.273)。
- 偏倚指标在不同人口学亚组间存在差异:自认美洲印第安人或阿拉斯加原住民的患者偏倚指标差14.0%,选择不披露族裔者差19.0%。
证据类型、等级与局限
- 证据类型:回顾性预后研究。
- 局限:原文未报告局限性。
对床旁实践的改变
- 作者认为DTI预测患者恶化的能力中等,在观察水平和就诊水平表现不同,且在不同人口学组间存在差异。
- 作者认为亚组表现差异提示需要更透明的模型训练数据,并强化对外部开发预测模型的本地验证。
提炼说明
- 本条依据论文摘要整理;摘要未涉及「处置流程与药物剂量」「红旗征与不能漏的情况」「指南或共识出处」,故未列出这些栏目;全文请点「阅读原文」。
英文原文(摘要)· 点击展开
IMPORTANCE: The Deterioration Index (DTI), used by hospitals for predicting patient deterioration, has not been extensively validated externally, raising concerns about performance and equitable predictions.
OBJECTIVE: To locally validate DTI performance and assess its potential for bias in predicting patient clinical deterioration.
DESIGN, SETTING, AND PARTICIPANTS: This retrospective prognostic study included 13 737 patients admitted to 8 heterogenous Midwestern US hospitals varying in size and type, including academic, community, urban, and rural hospitals. Patients were 18 years or older and admitted between January 1 and May 31, 2021.
EXPOSURE: DTI predictions made every 15 minutes.
MAIN OUTCOMES AND MEASURES: Deterioration, defined as the occurrence of any of the following while hospitalized: mechanical ventilation, intensive care unit transfer, or death. Performance of the DTI was evaluated using area under the receiver operating characteristic curve (AUROC) and area under the precision recall curve (AUPRC). Bias measures were calculated across demographic subgroups.
RESULTS: A total of 5 143 513 DTI predictions were made for 13 737 patients across 14 834 hospitalizations. Among 13 918 encounters, the mean (SD) age of patients was 60.3 (19.2) years; 7636 (54.9%) were female, 11 345 (81.5%) were White, and 12 392 (89.0%) were of other ethnicity than Hispanic or Latino. The prevalence of deterioration was 10.3% (n = 1436). The DTI produced AUROCs of 0.759 (95% CI, 0.756-0.762) at the observation level and 0.685 (95% CI, 0.671-0.700) at the encounter level. Corresponding AUPRCs were 0.039 (95% CI, 0.037-0.040) at the observation level and 0.248 (95% CI, 0.227-0.273) at the encounter level. Bias measures varied across demographic subgroups and were 14.0% worse for patients identifying as American Indian or Alaska Native and 19.0% worse for those who chose not to disclose their ethnicity.
CONCLUSIONS AND RELEVANCE: In this prognostic study, the DTI had modest ability to predict patient deterioration, with varying degrees of performance at the observation and encounter levels and across different demographic groups. Disparate performance across subgroups suggests the need for more transparency in model training data and reinforces the need to locally validate externally developed prediction models.
参考来源
- Byrd TF, Southwell B, Ravishankar A, et al. Validation of a Proprietary Deterioration Index Model and Performance in Hospitalized Adults. JAMA Netw Open. 2023. PMID: 37486632. doi:10.1001/jamanetworkopen.2023.24176
供执业医师抢救与诊疗参考,不替代临床判断;剂量请结合体重、肝肾功能和本院药品规格复核,以指南原文为准。