针对二分类结局错分校正的ROC分析
Creating accurate risk prediction models from Big Data resources such as Electronic Health Records (EHRs) is a critical step toward achieving precision medicine.…
要点
- 结局错分导致标准ROC分析的AUC产生偏倚,偏倚程度取决于假阳性率、假阴性率及疾病患病率。
- 仅在建模阶段校正错分不足以消除AUC偏倚。
- 所提错分校正ROC方法计算简便,适用于大数据集中高效比较多个预测模型。
核心结论
- 结局错分会使标准ROC分析的AUC产生偏倚,仅校正建模阶段不足以消除该偏倚。
- 所提错分校正ROC方法可产生真实AUC的偏倚校正估计。
主要发现
- 错分导致标准ROC分析的AUC偏倚,偏倚程度由假阳性和假阴性错分率及疾病患病率决定。
- 仅校正建模阶段的错分不足以消除AUC偏倚。
- 错分校正ROC方法可产生真实AUC的偏倚校正估计,计算简便,适用于大数据集中高效比较多个预测模型。
关键数值与推荐
- 错分校正ROC方法要求错分率已知或可估计,这些量通常是建模步骤所需的量。
临床怎么用
- 原文未据此给出推荐。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
Creating accurate risk prediction models from Big Data resources such as Electronic Health Records (EHRs) is a critical step toward achieving precision medicine. A major challenge in developing these tools is accounting for imperfect aspects of EHR data, particularly the potential for misclassified outcomes. Misclassification, the swapping of case and control outcome labels, is well known to bias effect size estimates for regression prediction models. In this paper, we study the effect of misclassification on accuracy assessment for risk prediction models and find that it leads to bias in the area under the curve (AUC) metric from standard ROC analysis. The extent of the bias is determined by the false positive and false negative misclassification rates as well as disease prevalence. Notably, we show that simply correcting for misclassification while building the prediction model is not sufficient to remove the bias in AUC. We therefore introduce an intuitive misclassification-adjusted ROC procedure that accounts for uncertainty in observed outcomes and produces bias-corrected estimates of the true AUC. The method requires that misclassification rates are either known or can be estimated, quantities typically required for the modeling step. The computational simplicity of our method is a key advantage, making it ideal for efficiently comparing multiple prediction models on very large datasets. Finally, we apply the correction method to a hospitalization prediction model from a cohort of over 1 million patients from the Veterans Health Administrations EHR. Implementations of the ROC correction are provided for Stata and R. Published 2017. This article is a U.S. Government work and is in the public domain in the USA.
引用
- Zawistowski M, Sussman JB, Hofer TP, et al. Corrected ROC analysis for misclassified binary outcomes. Stat Med. 2017. PMID: 28245528. doi:10.1002/sim.7260
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。