文献速递医学人工智能与信息学流行病学与生物统计

使用机器学习的预测模型研究对缺失数据的处理与报告不佳:一项文献综述

Missing data is poorly handled and reported in prediction model studies using machine learning: a literature review.
PubMed 文献 · J Clin Epidemiol · 2021-11-16

该系统综述检索2018至2019年发表的、使用监督式机器学习开发或验证临床预测模型的原著研究,共纳入152篇。其中56篇未报告缺失数据,96篇报告了处理方法但46篇未报告缺失比例,删除法(65/96,其中完整病例分析43/96)最常见,多重插补仅8/96、替代分割等内置机制仅7/96。作者认为多数研究对缺失数据的报告不充分。

要点

  • 152篇机器学习临床预测模型研究中,56篇完全未报告缺失数据情况。
  • 96篇报告处理方法的论文中,46篇未报告缺失比例,仅7篇报告可能原因、8篇报告缺失机制。
  • 删除法最常见(65/96),其中完整病例分析43/96;多重插补仅8/96,替代分割等内置机制仅7/96。

核心结论

  • 多数使用机器学习的预测模型研究未充分报告缺失数据的存在与处理方式,删除法虽被普遍认为可能造成偏倚并损失分析效能,却仍是最常用的方法。
  • 作者认为预测模型研究者应更加了解处理缺失数据的替代方法。

研究设计与样本

  • 文献综述:系统检索2018年至2019年发表的、在各医学领域使用任一监督式机器学习方法开发或验证临床预测模型的原著研究。
  • 共纳入152篇基于机器学习的临床预测模型研究。
  • 提取信息包括缺失数据的数量与性质(如完全随机缺失、缺失的可能原因)及其处理方式。

主要发现

  • 152篇论文中有56篇未报告任何缺失数据相关信息(n=56/152)。
  • 多数论文(n=96/152)报告了缺失数据处理细节(如所用方法),但其中许多(n=46/96)未报告数据中缺失的比例。
  • 这96篇论文中,仅有时报告缺失的可能原因(n=7/96)和缺失数据机制信息(n=8/96)。
  • 最常见的缺失数据处理方法是删除法(n=65/96),多数为完整病例分析(n=43/96)。
  • 极少数研究使用多重插补(n=8/96)或在预测模型开发、验证或实施中直接处理缺失数据的内置机制如替代分割(n=7/96)。

局限性

  • 原文未报告局限性

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

OBJECTIVES: Missing data is a common problem during the development, evaluation, and implementation of prediction models. Although machine learning (ML) methods are often said to be capable of circumventing missing data, it is unclear how these methods are used in medical research. We aim to find out if and how well prediction model studies using machine learning report on their handling of missing data. STUDY DESIGN AND SETTING: We systematically searched the literature on published papers between 2018 and 2019 about primary studies developing and/or validating clinical prediction models using any supervised ML methodology across medical fields. From the retrieved studies information about the amount and nature (e.g. missing completely at random, potential reasons for missingness) of missing data and the way they were handled were extracted. RESULTS: We identified 152 machine learning-based clinical prediction model studies. A substantial amount of these 152 papers did not report anything on missing data (n = 56/152). A majority (n = 96/152) reported details on the handling of missing data (e.g., methods used), though many of these (n = 46/96) did not report the amount of the missingness in the data. In these 96 papers the authors only sometimes reported possible reasons for missingness (n = 7/96) and information about missing data mechanisms (n = 8/96). The most common approach for handling missing data was deletion (n = 65/96), mostly via complete-case analysis (CCA) (n = 43/96). Very few studies used multiple imputation (n = 8/96) or built-in mechanisms such as surrogate splits (n = 7/96) that directly address missing data during the development, validation, or implementation of the prediction model. CONCLUSION: Though missing values are highly common in any type of medical research and certainly in the research based on routine healthcare data, a majority of the prediction model studies using machine learning does not report sufficient information on the presence and handling of missing data. Strategies in which patient data are simply omitted are unfortunately the most often used methods, even though it is generally advised against and well known that it likely causes bias and loss of analytical power in prediction model development and in the predictive accuracy estimates. Prediction model researchers should be much more aware of alternative methodologies to address missing data.

来源:J Clin Epidemiol,查看原文。版权归原作者所有。

引用

  1. Nijman S, Leeuwenberg AM, Beekers I, et al. Missing data is poorly handled and reported in prediction model studies using machine learning: a literature review. J Clin Epidemiol. 2021. PMID: 34798287. doi:10.1016/j.jclinepi.2021.11.023

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。