文献速递流行病学与生物统计医学人工智能与信息学

排序一致性作为预测模型稳定性新指标:纵向队列研究

Consistency of ranking was evaluated as new measure for prediction model stability: longitudinal cohort study.
PubMed 文献 · J Clin Epidemiol · 2021-07-03

15个模型(含机器学习和统计模型)群体水平C统计量约0.88相似,但个体水平上高风险患者排序一致而绝对风险不一致,低风险患者反之。建议TRIPOD和PROBAST指南纳入排序以评估个体水平稳定性。

要点

  • 排序一致性可补充绝对风险稳定性,评估个体水平预测稳定性。
  • 高风险患者模型间排序一致但绝对风险差异大,低风险患者相反。
  • 模型开发指南TRIPOD和PROBAST应纳入排序评估个体水平稳定性。

核心结论

  • 排序一致性是评估风险预测模型个体水平稳定性的有用指标,提供绝对风险稳定性之外的补充信息。
  • 模型开发指南(如TRIPOD和PROBAST)应纳入排序以评估个体水平稳定性。

主要发现

  • 15个模型(含机器学习和统计模型)群体水平C统计量约0.88,表现相似。
  • 高风险患者模型间排序差异百分位四分位距为-0.6至1.0,绝对风险差异四分位距为-18.8至9.0。
  • 低风险患者排序不一致但绝对风险更一致。

关键数值与推荐

  • 高风险患者排序差异百分位IQR为-0.6至1.0,绝对风险差异IQR为-18.8至9.0。
  • 15个模型群体水平C统计量约0.88。

临床怎么用

  • 在模型比较和选择时,除群体水平性能外,应评估个体水平排序一致性和绝对风险稳定性。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

OBJECTIVE: Clinical risk prediction models are generally assessed on population level with a lack of measures that evaluate their stability at predicting risks of individual patients. This study evaluated the use of ranking as a measure to assess individual level stability between risk prediction models. STUDY DESIGN AND SETTING: A large patient cohort (3.66 million patients with 0.11 million cardiovascular events) extracted from the Clinical Practice Research Datalink was used in the exemplar of cardiovascular disease risk prediction. RESULTS: It was found that 15 models (including machine learning and statistical models) had similar population-level model performance (C statistics about 0.88). For patients with high absolute risks, the models were more consistent in ranking of risk predictions (interquartile range (IQR) of differences in rank percentiles -0.6 to 1.0), but inconsistent in absolute risk (IQR of differences in absolute risk -18.8 to 9.0). At low risk, the reverse was true with inconsistent ranking but more consistent absolute risk. CONCLUSION: Consistency of ranking of individual risk predictions is a useful measure to assess risk prediction models providing complementary information to absolute risk stability. Model developing guidelines including "TRIPOD" and "PROBAST" should incorporate ranking to assess individual level stability between risk prediction models.

来源:J Clin Epidemiol,查看原文。版权归原作者所有。

引用

  1. Li Y, Sperrin M, Ashcroft DM, et al. Consistency of ranking was evaluated as new measure for prediction model stability: longitudinal cohort study. J Clin Epidemiol. 2021. PMID: 34224835. doi:10.1016/j.jclinepi.2021.06.026

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。