外部验证表明机器学习模型在预测客观和患者报告膀胱过度活动症治疗结局方面优于人类专家
研究基于ROSETTA试验队列训练算子分裂神经网络,用于预测膀胱内肉毒毒素A注射后膀胱过度活动症的客观应答和患者报告症状改善,并在ABC试验队列中外部验证。验证集中,该网络预测临床应答者状态的AUC为0.66,优于XGBoost(AUC 0.58)、逻辑回归(AUC 0.55)和人类专家(AUC 0.47-0.53);预测患者主观症状改善的AUC为0.64,同样优于其他算法和人类专家(AUC 0.41-0.62)。
要点
- 算子分裂神经网络外部验证预测OAB患者OBTX-A临床应答者状态AUC为0.66,优于XGBoost、逻辑回归和人类专家。
- 该网络预测患者报告症状改善的AUC为0.64,优于其他算法和人类专家(AUC 0.41-0.62)。
- 作者认为临床实施可能改善咨询和治疗选择。
核心结论
- 在具有挑战性的独立验证队列中,神经网络在预测OAB患者OBTX-A客观和患者报告结局方面优于人类专家和其他机器学习方法。
- 作者认为临床实施可能改善咨询和治疗选择。
研究设计与样本
- 使用ROSETTA试验队列训练算子分裂算法,以预测客观应答和患者报告症状改善,并使用接受OBTX-A的ABC试验队列进行外部验证。
- 在外部验证集中比较算法、公开可用的顶级机器学习分类器XGBoost、交叉验证逻辑回归和人类专家的AUC。
主要发现
- 在验证集中,算子分裂神经网络预测临床应答者状态的AUC为0.66,优于XGBoost with DART(AUC 0.58)、逻辑回归(AUC 0.55)和人类专家(AUC 0.47-0.53)。
- 该网络预测OBTX-A后患者主观症状改善的AUC为0.64,优于其他算法和人类专家(AUC 0.41-0.62)。
临床怎么用
- 原文未据此给出推荐。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
OBJECTIVE: To predict treatment response for overactive bladder (OAB) for a specific patient remains elusive. We sought to develop accurate models using machine learning for prediction of objective and patient-reported treatment response to intravesical botulinum toxin (OBTX-A) injection. We sought to validate the models in a challenging setting using an external dataset of a markedly different patient cohort and dosing regimen. We hypothesized the model would outperform human experts and top available algorithms. METHODS: Algorithms using "operator splitting" designed for accuracy and efficiency even in small training datasets with variable completeness, were trained to predict objective response and patient-reported symptomatic improvement using the ROSETTA trial cohort and validated using the ABC trial cohort of patients who underwent OBTX-A. Areas under the curve (AUC) of algorithms were compared to the top publicly-available machine-learning classifier XGBoost, logistic regression with cross validation, and human expert predictions in the external validation set. RESULTS: In the validation set, the operator splitting neural network had AUC of 0.66 and outperformed XGBoost with DART (top available machine-learning classifier, AUC: 0.58), logistic regression (AUC 0.55), and human experts (AUC 0.47-0.53) for prediction of clinical responder status. It was similarly accurate in prediction of patient subjective improvement in symptoms following OBTX-A (AUC: 0.64), again outperforming other algorithms and human experts (AUC 0.41-0.62). CONCLUSION: The neural network outperformed human experts and other machine-learning approaches in prediction of objective and patient-reported OBTX-A outcomes for OAB in a challenging independent validation cohort. Clinical implementation could improve counseling and treatment selection.
引用
- Werneburg GT, Werneburg EA, Goldman HB, et al. External Validation Demonstrates Machine Learning Models Outperform Human Experts in Prediction of Objective and Patient-reported Overactive Bladder Treatment Outcomes. Urology. 2024. PMID: 39242047. doi:10.1016/j.urology.2024.08.071
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。