指南与共识技术与安全

人工智能在行动:放射学人工智能委员会关于有效模型评估与部署的路线图

AI in Action: A Road Map From the Radiology AI Council for Effective Model Evaluation and Deployment.
PubMed 文献 · J Am Coll Radiol · 2025-05-23

本文介绍在一家大型学术中心创建放射学人工智能委员会,并制定评分表框架以规范放射学AI模型的评估和临床工作流接入。该评分表旨在应对真实世界模型性能、工作流实施、资源配置、投资回报及对更广泛卫生系统影响等部署挑战,文章概述了评分表的建立步骤、组成部分及8个月内评估13个模型的初步结果。

要点

  • 放射学AI委员会制定评分表框架,标准化、透明化地评估AI模型并接入临床工作流。
  • 评估维度涵盖真实世界模型性能、工作流实施、资源配置、投资回报及对卫生系统的影响。
  • 8个月内评估13个模型,强调超越性能指标的整体评估及评估的透明性与客观性。

核心结论

  • 放射学AI委员会通过评分表框架标准化和透明化AI模型选择过程,强调超越性能指标的整体评估以及评估的透明性和客观性,以提高放射学AI模型的疗效和安全性。

研究设计与样本

  • 本文描述在一家大型学术中心创建放射学AI委员会并制定评分表框架,用于评估放射学AI模型并将其接入临床工作流,报告了8个月内评估13个模型的初步结果。

局限性

  • 原文未报告局限性

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

英文原文(摘要)· 点击展开

As the integration of artificial intelligence (AI) into radiology workflows continues to evolve, establishing standardized processes for the evaluation and deployment of AI models is crucial to ensure success. This article outlines the creation of a Radiology AI Council at a large academic center and subsequent development of framework in the form of a rubric to formalize the evaluation of radiology AI models and onboard them into clinical workflows. The rubric aims to address the challenges faced during the deployment of AI models, such as real-world model performance, workflow implementation, resource allocation, return on investment, and impact to the broader health system. Using this comprehensive rubric, the council aims to ensure that the process for selecting AI models is both standardized and transparent. This article outlines the steps taken to establish this rubric, its components, and the initial results from evaluation of 13 models over an 8-month period. We emphasize the importance of holistic model evaluation beyond performance metrics, and transparency and objectivity in AI model evaluation, with the goal of improving the efficacy and safety of AI models in radiology.

来源:J Am Coll Radiol,查看原文。版权归原作者所有。

引用

  1. Trivedi H, Khosravi B, Gichoya J, et al. AI in Action: A Road Map From the Radiology AI Council for Effective Model Evaluation and Deployment. J Am Coll Radiol. 2025. PMID: 40414408. doi:10.1016/j.jacr.2025.05.016

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。