基于机器学习的预测模型类别不平衡校正的危害:一项模拟研究
该研究通过大量蒙特卡洛模拟比较了在不同数据生成场景下,使用类别不平衡校正与未校正的机器学习模型的样本外预测性能,并使用MIMIC-III数据进行了案例研究。结果显示,在所有模拟场景中,未校正类别不平衡的预测模型校准性能等于或优于校正后的模型;校正引入的校准不良表现为风险高估,且并不总能通过重新校准纠正。结论认为,校正类别不平衡并非总是必要,甚至可能对旨在提供个体可靠风险估计的临床预测模型有害。
要点
- 在所有模拟场景中,未校正类别不平衡的模型校准性能等于或优于校正模型。
- 校正类别不平衡引入的校准不良表现为风险高估,且不总能通过重新校准纠正。
- 校正类别不平衡并非总是必要,甚至可能对临床预测模型有害。
核心结论
- 校正类别不平衡并非总是必要,甚至可能对旨在产生个体可靠风险估计的临床预测模型有害。
研究设计与样本
- 使用大量蒙特卡洛模拟,在不同数据生成场景(样本量、预测变量数量和事件比例变化)下比较使用与不使用类别不平衡校正的机器学习模型的样本外预测性能。
- 使用MIMIC-III数据进行案例研究说明发现。
主要发现
- 在所有模拟场景中,未校正类别不平衡的预测模型校准性能等于或优于校正类别不平衡的预测模型。
- 校正类别不平衡引入的校准不良表现为风险高估,且并不总能通过重新校准纠正。
临床怎么用
- 作者认为,校正类别不平衡并非总是必要,甚至可能对旨在产生个体可靠风险估计的临床预测模型有害。
局限性
- 原文未报告局限性
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
INTRODUCTION: Risk prediction models are increasingly used in healthcare to aid in clinical decision-making. In most clinical contexts, model calibration (i.e., assessing the reliability of risk estimates) is critical. Data available for model development are often not perfectly balanced with the modeled outcome (i.e., individuals with vs. without the event of interest are not equally prevalent in the data). It is common for researchers to correct for class imbalance, yet, the effect of such imbalance corrections on the calibration of machine learning models is largely unknown. METHODS: We studied the effect of imbalance corrections on model calibration for a variety of machine learning algorithms. Using extensive Monte Carlo simulations we compared the out-of-sample predictive performance of models developed with an imbalance correction to those developed without a correction for class imbalance across different data-generating scenarios (varying sample size, the number of predictors, and event fraction). Our findings were illustrated in a case study using MIMIC-III data. RESULTS: In all simulation scenarios, prediction models developed without a correction for class imbalance consistently had equal or better calibration performance than prediction models developed with a correction for class imbalance. The miscalibration introduced by correcting for class imbalance was characterized by an over-estimation of risk and was not always able to be corrected with re-calibration. CONCLUSION: Correcting for class imbalance is not always necessary and may even be harmful to clinical prediction models which aim to produce reliable risk estimates on an individual basis.
引用
- Carriero A, Luijken K, de Hond A, et al. The Harms of Class Imbalance Corrections for Machine Learning Based Prediction Models: A Simulation Study. Stat Med. 2025. PMID: 39865585. doi:10.1002/sim.10320
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。