指南与共识细胞病理与分子病理

美国病理学家学会关于病理学机器学习性能评估的建议

Recommendations for Performance Evaluation of Machine Learning in Pathology: A Concept Paper From the College of American Pathologists.
指南 / 共识 · Arch Pathol Lab Med · 2024-01-01

美国病理学家学会专家小组基于文献、CLSI指南及监管框架,提出病理临床领域机器学习系统性能评估的建议,涵盖风险评估、预部署要求、数据来源与整理、验证与确认、变更控制管理、人机交互、人员培训及能力评估等主题,旨在指导实验室对含机器学习的体外诊断检测进行验证和/或确认。

要点

  • 机器学习系统用于临床病理决策支持前需进行性能评估。
  • 评估应覆盖分析前、分析中、分析后全流程及人机交互。
  • 建议包括风险评估、数据管理、验证确认和变更控制。

核心结论

  • 机器学习模型的性能评估对于使用机器学习的体外诊断检测的验证和/或确认至关重要。
  • 本文提出的建议适用于拟用于临床患者检测的机器学习系统的性能评估。
  • 鼓励进一步研究真实世界临床数据以支持这些建议。

研究设计与样本

  • 专家小组对文献、临床和实验室标准协会(CLSI)指南以及实验室和政府监管框架进行了审查。
  • 基于审查结果提出了关于在患者样本体外诊断测试中纳入机器学习的性能评估建议。

主要发现

  • 文献和现有文件的审查促成了建议的制定。
  • 建议涵盖的主题包括机器学习模型评估的考虑因素:风险评估、预部署要求、数据来源与整理、验证与确认、变更控制管理、人机交互、人员培训及能力评估。

关键数值与推荐

  • 建议涉及对患者样本的体外诊断测试进行性能评估,这些测试在实验室工作流的分析前、分析中或分析后阶段纳入了机器学习。

临床怎么用

  • 病理实验室在临床实践中评估机器学习系统性能时可参考这些建议。
  • 实验室应关注风险评估、数据管理、验证确认、变更控制、人机交互及人员培训等环节。

局限性

  • 本文为概念性文件,建议基于文献和现有框架,尚需真实世界临床数据进一步支持。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

CONTEXT.—: Machine learning applications in the pathology clinical domain are emerging rapidly. As decision support systems continue to mature, laboratories will increasingly need guidance to evaluate their performance in clinical practice. Currently there are no formal guidelines to assist pathology laboratories in verification and/or validation of such systems. These recommendations are being proposed for the evaluation of machine learning systems in the clinical practice of pathology. OBJECTIVE.—: To propose recommendations for performance evaluation of in vitro diagnostic tests on patient samples that incorporate machine learning as part of the preanalytical, analytical, or postanalytical phases of the laboratory workflow. Topics described include considerations for machine learning model evaluation including risk assessment, predeployment requirements, data sourcing and curation, verification and validation, change control management, human-computer interaction, practitioner training, and competency evaluation. DATA SOURCES.—: An expert panel performed a review of the literature, Clinical and Laboratory Standards Institute guidance, and laboratory and government regulatory frameworks. CONCLUSIONS.—: Review of the literature and existing documents enabled the development of proposed recommendations. This white paper pertains to performance evaluation of machine learning systems intended to be implemented for clinical patient testing. Further studies with real-world clinical data are encouraged to support these proposed recommendations. Performance evaluation of machine learning models is critical to verification and/or validation of in vitro diagnostic tests using machine learning intended for clinical practice.

来源:Arch Pathol Lab Med,查看原文。版权归原作者所有。

引用

  1. Hanna MG, Olson NH, Zarella M, et al. Recommendations for Performance Evaluation of Machine Learning in Pathology: A Concept Paper From the College of American Pathologists. Arch Pathol Lab Med. 2024. PMID: 38041522. doi:10.5858/arpa.2023-0042-CP

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。