人工智能诊断影像算法开发与评估的监管框架:总结与建议
本文综述了软件作为医疗器械应用的主要监管框架,指出其在诊断任务与算法混淆、任务定义肤浅、缺乏直接比较机制、安全与性能表征不足、各安装站点性能评估资源缺乏及固有利益冲突等六方面缺陷,并建议分离诊断任务与算法、定义超越准确性的性能要素、分步评估、引入第三方评估并将这些要素纳入制造商开发流程,推荐四阶段开发与评估以保障所有安装站点算法达到世界级性能。
要点
- 现有监管框架未充分区分诊断任务与诊断算法,可能影响算法可信度。
- 建议定义超越准确性的性能要素,并将评估过程分为离散步骤。
- 推荐四阶段开发与评估流程,以确保所有安装站点算法性能达到世界级水平。
核心结论
- 当前监管框架存在六项主要缺陷,需补充措施以改进诊断人工智能算法的开发与评估。
- 建议分离诊断任务与算法、定义超越准确性的性能要素、分步评估、鼓励第三方评估并将要素纳入制造商开发流程。
- 推荐四阶段开发与评估,类比于药物和软件应用已应用或提议的流程,以帮助确保所有安装站点所有算法达到世界级性能。
研究设计与样本
- 本文为综述,回顾了软件作为医疗器械应用的主要监管框架,识别主要差距并提出额外策略。
- 原文未报告系统检索范围、纳入文献数量或证据分级方法。
诊断标准与关键数值
- 原文未报告敏感度、特异度、AUC、阈值或分级标准等诊断标准与关键数值。
临床应用与阅片要点
- 作者建议将诊断任务与算法分离,并定义超越准确性的性能要素。
- 作者建议将评估过程分为离散步骤,并鼓励由第三方评估者进行评估。
- 作者建议将这些要素纳入制造商的开发流程,并推荐四阶段开发与评估。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
Although artificial intelligence (AI)-based algorithms for diagnosis hold promise for improving care, their safety and effectiveness must be ensured to facilitate wide adoption. Several recently proposed regulatory frameworks provide a solid foundation but do not address a number of issues that may prevent algorithms from being fully trusted. In this article, we review the major regulatory frameworks for software as a medical device applications, identify major gaps, and propose additional strategies to improve the development and evaluation of diagnostic AI algorithms. We identify the following major shortcomings of the current regulatory frameworks: (1) conflation of the diagnostic task with the diagnostic algorithm, (2) superficial treatment of the diagnostic task definition, (3) no mechanism to directly compare similar algorithms, (4) insufficient characterization of safety and performance elements, (5) lack of resources to assess performance at each installed site, and (6) inherent conflicts of interest. We recommend the following additional measures: (1) separate the diagnostic task from the algorithm, (2) define performance elements beyond accuracy, (3) divide the evaluation process into discrete steps, (4) encourage assessment by a third-party evaluator, (5) incorporate these elements into the manufacturers' development process. Specifically, we recommend four phases of development and evaluation, analogous to those that have been applied to pharmaceuticals and proposed for software applications, to help ensure world-class performance of all algorithms at all installed sites. In the coming years, we anticipate the emergence of a substantial body of research dedicated to ensuring the accuracy, reliability, and safety of the algorithms.
引用
- Larson DB, Harvey H, Rubin DL, et al. Regulatory Frameworks for Development and Evaluation of Artificial Intelligence-Based Diagnostic Imaging Algorithms: Summary and Recommendations. J Am Coll Radiol. 2020. PMID: 33096088. doi:10.1016/j.jacr.2020.09.060
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。