机器学习跨中心应用:手术部位感染检测算法的外部验证
该研究将在明尼苏达大学医学中心开发的自动化手术部位感染(SSI)检测算法,在加州大学旧金山分校进行外部验证。结果显示,表浅、器官/腔隙及总SSI的AUC值在内外验证间无统计学差异,表明算法具有跨机构可推广性,可用于加速和聚焦病历审查。
要点
- SSI检测机器学习算法在不同中心间具有可推广性。
- 外部验证的AUC与内部验证无统计学差异。
- 提高病例审查量可降低假阴性率,可排除低概率病例。
核心结论
- 在一个中心开发的SSI检测机器学习算法可推广至另一机构。
- SSI检测模型可实际应用于加速和聚焦病历审查。
研究设计与样本
- 研究基于NSQIP患者,在Site A(明尼苏达大学医学中心)提取4年电子健康记录数据用于模型开发和内部验证,在Site B(加州大学旧金山分校)提取随后2年数据用于外部验证。
- Site A纳入8,883例患者,Site B纳入1,473例患者。
- 模型用于检测术后30天内的表浅、器官/腔隙及总SSI,验证指标为AUC及95%置信区间。
主要发现
- 表浅SSI外部验证AUC为0.804,内部验证AUC为0.784(95%CI 0.874),两者无统计学差异。
- 器官/腔隙SSI外部验证AUC为0.905,内部验证AUC为0.867(95%CI 0.941),两者无统计学差异。
- 总SSI外部验证AUC为0.855,内部验证AUC为0.854(95%CI 0.908),两者无统计学差异。
- 假阴性率随病例审查量的增加而降低,可采取排除低预测概率SSI病例的策略。
临床怎么用
- 该SSI检测算法可跨机构应用,帮助加速和聚焦病历审查。
- 可根据预测概率排除低风险病例,减少人工审查负担。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
BACKGROUND: Surgical complications have tremendous consequences and costs. Complication detection is important for quality improvement, but traditional manual chart review is burdensome. Automated mechanisms are needed to make this more efficient. To understand the generalizability of a machine learning algorithm between sites, automated surgical site infection (SSI) detection algorithms developed at one center were tested at another distinct center. STUDY DESIGN: NSQIP patients had electronic health record (EHR) data extracted at one center (University of Minnesota Medical Center, Site A) over a 4-year period for model development and internal validation, and at a second center (University of California San Francisco, Site B) over a subsequent 2-year period for external validation. Models for automated NSQIP SSI detection of superficial, organ space, and total SSI within 30 days postoperatively were validated using area under the curve (AUC) scores and corresponding 95% confidence intervals. RESULTS: For the 8,883 patients (Site A) and 1,473 patients (Site B), AUC scores were not statistically different for any outcome including superficial (external 0.804, internal [0.784, 0.874] AUC); organ/space (external 0.905, internal [0.867, 0.941] AUC); and total (external 0.855, internal [0.854, 0.908] AUC) SSI. False negative rates decreased with increasing case review volume and would be amenable to a strategy in which cases with low predicted probabilities of SSI could be excluded from chart review. CONCLUSIONS: Our findings demonstrated that SSI detection machine learning algorithms developed at 1 site were generalizable to another institution. SSI detection models are practically applicable to accelerate and focus chart review.
引用
- Zhu Y, Simon GJ, Wick EC, et al. Applying Machine Learning Across Sites: External Validation of a Surgical Site Infection Detection Algorithm. J Am Coll Surg. 2021. PMID: 33831539. doi:10.1016/j.jamcollsurg.2021.03.026
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。