文献速递流行病学与生物统计医学人工智能与信息学

结局错分下风险预测模型的验证方法

Validation of a Risk-Prediction Model in the Presence of Outcome Misclassification.
PubMed 文献 · Stat Med · 2026-01-01

电子健康记录易出现结局测量误差,直接以错分结局验证预测模型会得到不可靠的性能估计。作者提出利用小样本病历审查金标准数据校正错分,推导TPR、FPR、PPV、NPV和AUC公式,并模拟比较不同错分场景。结果表明所提方法在所有考察场景下准确性好且精度提高。

要点

  • 结局错分会使预测模型验证结果不可靠
  • 可用小样本病历审查金标准数据校正错分
  • 所提方法在模拟场景中准确且精度提高

核心结论

  • 评估预测模型性能时应考虑结局错分,以准确指导临床决策。

研究设计与样本

  • 方法学研究:利用小样本病历审查金标准结局测量校正验证中的结局错分。
  • 推导存在结局错分时TPR、FPR、PPV、NPV和AUC的估计公式。
  • 通过模拟研究比较所提方法与未校正错分方法及小样本病历审查方法的性能。

主要发现

  • 模拟研究覆盖错分独立或依赖特征、单向或双向等不同场景。
  • 所提方法在所有考察的错分场景下估计准确性好且精度提高。

关键数值与推荐

  • 推导的校正指标包括真阳性率、假阳性率、阳性预测值、阴性预测值和ROC曲线下面积。

临床怎么用

  • 在利用电子健康记录验证预测模型时,应收集小样本病历审查金标准数据以校正结局错分。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

Electronic health records (EHRs) provide a rich data source for building prediction models to improve the quality of care. However, EHR data are prone to measurement error, and outcomes used to evaluate prediction model performance may be misclassified. Comparing risk predictions to misclassified outcomes will result in unreliable estimates of a prediction model's performance. We propose a method that leverages a smaller chart review sample with gold-standard outcome measurements to adjust validation for outcome misclassification and provide more accurate prediction model evaluation. We derive formulae to estimate true positive rate (TPR), false positive rate (FPR), positive predictive value (PPV), negative predictive value (NPV), and area under the receiver operating characteristic curve (AUC) in the presence of outcome misclassification. Different scenarios of misclassification are explored, including when the misclassification is independent or dependent on features, and when misclassification is unidirectional (e.g., only missed diagnoses) or bidirectional. In simulation studies, we compare the bias and 95% confidence interval coverage of performance estimates obtained using our proposed method to those estimated with misclassified outcomes (without accounting for misclassification) or in the smaller chart review sample. Simulation results indicate that, across all misclassification scenarios examined, our proposed estimates have good accuracy and improved precision. Outcome misclassification should be considered when evaluating a prediction model's performance in order to accurately inform decision-making about whether and how to use a clinical prediction model.

来源:Stat Med,查看原文。版权归原作者所有。

引用

  1. Zou R, D Williamson B, M Shortreed S, et al. Validation of a Risk-Prediction Model in the Presence of Outcome Misclassification. Stat Med. 2026. PMID: 41930736. doi:10.1002/sim.70377

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。