商用放射AI产品人口学亚组报告现状:范围综述
该范围综述纳入545项验证252个商用放射AI产品的研究,分析按性别、年龄、种族/民族报告亚组表现的现状。结果显示仅77项研究报告了亚组表现,骨骼与肺部X线研究相对最常见。近年发表时间与厂商资助并未提高亚组报告比例,且多数结核检测数据集可能不足以支持事后亚组荟萃分析。
要点
- 商用放射AI验证研究常未描述人口学亚组数据,更少给出各亚组表现结果。
- 阅片时不宜默认AI在少数群体中与总体表现一致,应关注亚组证据缺失。
- 骨与胸部X线类AI验证中亚组表现报告相对较多,其他部位证据更薄弱。
核心结论
- 商用放射AI验证报告分散,人口学描述与亚组表现均不足以估计亚组偏倚。
- 当前文献不足以确定这些产品在人口学少数群体中的安全性与表现。
研究设计与样本
- 共纳入2010年后发表的545项研究,涉及252个商用放射AI产品。
- 采用回归分析评估报告趋势,并应用Wilson置信区间方程估计结核检测研究的事后亚组荟萃分析把握度。
关键影像表现/征象
- 骨骼研究(20/88)与肺部研究(30/139)最常报告亚组表现数据。
- 使用胸部X线(24/79)或骨X线(19/63)的研究最常报告亚组表现数据。
诊断标准与关键数值
- 更近发表年份与亚组报告增加无显著关联(OR 1.039,95%CI 0.959-1.127)。
- 厂商资助与亚组报告增加无显著关联(OR 1.010,95%CI 0.492-1.920)。
- 21个结核检测数据集中14个可能不足以进行事后亚组荟萃分析。
临床应用与阅片要点
- 阅读商用AI辅助报告时应意识到其在少数人口群体中的亚组表现证据普遍不足。
- 对缺乏亚组验证证据的AI结果不宜直接推广至与本机构人群特征不同的个体。
- 报告AI辅助发现时可注明该产品在相关人口学亚组中的验证证据缺失。
局限性
- 原文未具体描述纳入验证研究本身的方法学质量或参考标准异质性。
- 结果仅基于发表于同行评审期刊的商用AI验证研究,未纳入非公开验证数据。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
OBJECTIVE: Though subgroup performance reporting helps ensure the safety of artificial intelligence (AI) products, the extent of this reporting remains unclear. This scoping review identifies studies validating commercially available AI-based products and reports the trends in performance reporting across sex, age, and race/ethnicity demographic subgroups.
MATERIALS AND METHODS: Peer-reviewed validation studies of commercially available products published after 2010 were collected from the Health AI Register and PubMed on 29 November 2024. Study trends in the reporting of sex, age, and race/ethnicity were mapped with regression analysis. We apply the Wilson confidence interval equation to estimate which tuberculosis detection studies are underpowered for subgroup meta-analysis.
RESULTS: Three hundred ninety-two of 545 studies validating 252 products reported subgroup demographic data for any of the three groups. Only 77 of these presented subgroup performance results. Skeletal (20/88) and lung (30/139) studies, and those utilizing chest (24/79) or bone (19/63) radiographs, most often presented subgroup performance data. We found no evidence that more recent studies (OR: 1.039 [95% CI: 0.959-1.127]) or company sponsorship (OR: 1.010 [95% CI: 0.492-1.920]) led to increased subgroup reporting. We show that 14/21 tuberculosis datasets may be underpowered for post-hoc subgroup meta-analysis.
CONCLUSION: This scoping review quantifies how fragmented the commercial validation landscape is, showing that reporting for both the demographics and per-subgroup performance is inadequate for estimating subgroup bias. This systemic problem requires effort from all stakeholders, from researchers to regulatory agencies, encouraging thorough reporting and commercial product validation to support physician and patient trust in medical AI products.
KEY POINTS: Question The number of studies validating the performance of each commercially available radiology AI product for minority subgroup bias is unclear. Findings The currently available commercial AI validation studies often neglect to describe demographic subgroup data, and fewer provide performance results per subgroup, prohibiting algorithmic bias meta-analysis. Clinical relevance Physician and patient trust in the medical AI already used clinically must be built on peer-reviewed literature and meta-analysis. The current literature is insufficient for determining the safety and performance of these products for demographic minorities.
引用
- Walston, Takita, Mitsuyama, et al. The current state of demographic subgroup reporting for commercially available AI for radiology: a scoping review. Eur Radiol. 2026. PMID: 42286177. doi:10.1007/s00330-026-12652-y
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。