创建标准化工具以评估和比较结肠镜中基于人工智能的计算机辅助检测程序:改良德尔菲法
通过改良德尔菲法,25位全球结肠镜计算机辅助检测(CADe)领域的专家经8个月在线调查,从121条评分标准中筛选出6项最高优先级的CADe评估指标,包括敏感性和独立验证(并列第一)、腺瘤检出率、假阳性率、延迟时间和腺瘤漏检率,旨在为CADe软件开发和验证提供参考。
要点
- 改良德尔菲法筛选出6项最高优先级CADe评估指标。
- 敏感性与独立验证并列第一,平均优先级评分均为4.16。
- 其余指标为腺瘤检出率(4.08)、假阳性率(4.00)、延迟时间(3.84)、腺瘤漏检率(3.68)。
核心结论
- 这是首份关于结肠镜CADe优先评分指标的国际共识声明,这些评分标准应为CADe软件开发和改进提供参考。
研究设计与样本
- 采用改良德尔菲法,25位全球结肠镜CADe领域专家(包括内镜医师、研究人员和行业代表)参与为期8个月的在线调查。
主要发现
- 第一轮排序后54条标准的平均优先级评分为2.25至4.38。
- 第一轮排序后排名前11的标准平均优先级评分为3.04至4.16。
- 最终6项最高优先级标准为:敏感性与独立验证并列第一(平均评分均为4.16),腺瘤检出率(平均4.08),假阳性率(平均4.00),延迟时间(平均3.84),腺瘤漏检率(平均3.68)。
关键数值与推荐
- 参与者共生成121条评分标准,其中54条被认为在研究范围内并分发审阅。
- 参与者使用5点李克特量表对标准按优先级排序,最终确定6项最高优先级标准。
临床怎么用
- 这些评分标准应为CADe软件开发和改进提供参考。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
BACKGROUND AND AIMS: Multiple computer-aided detection (CADe) software programs have now achieved regulatory approval in the United States, Europe, and Asia and are being used in routine clinical practice to support colorectal cancer screening. There is uncertainty regarding how different CADe algorithms may perform. No objective methodology exists for comparing different algorithms. We aimed to identify priority scoring metrics for CADe evaluation and comparison. METHODS: A modified Delphi approach was used. Twenty-five global leaders in CADe in colonoscopy, including endoscopists, researchers, and industry representatives, participated in an online survey over the course of 8 months. Participants generated 121 scoring criteria, 54 of which were deemed within the study scope and distributed for review and asynchronous e-mail-based open comment. Participants then scored criteria in order of priority on a 5-point Likert scale during ranking round 1. The top 11 highest priority criteria were re-distributed, with another opportunity for open comment, followed by a final round of priority scoring to identify the final 6 criteria. RESULTS: Mean priority scores for the 54 criteria ranged from 2.25 to 4.38 after the first ranking round. The top 11 criteria after round 1 of ranking yielded mean priority scores ranging from 3.04 to 4.16. The final 6 highest priority criteria, including a tie for first-place ranking, were (1, tied) sensitivity (average, 4.16) and (1, tied) separate and independent validation of the CADe algorithm (average, 4.16); (3) adenoma detection rate (average, 4.08); (4) false-positive rate (average, 4.00); (5) latency (average, 3.84); and (6) adenoma miss rate (average, 3.68). CONCLUSIONS: This is the first reported international consensus statement of priority scoring metrics for CADe in colonoscopy. These scoring criteria should inform CADe software development and refinement. Future research should validate these metrics on a benchmark video dataset to develop a validated scoring instrument.
引用
- Gadi SRV, Mori Y, Misawa M, et al. Creating a standardized tool for the evaluation and comparison of artificial intelligence-based computer-aided detection programs in colonoscopy: a modified Delphi approach. Gastrointest Endosc. 2024. PMID: 39608592. doi:10.1016/j.gie.2024.11.042
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。