监管批准商用AI用于急性卒中CT的公平性与公平性报告系统评价
结果仅1项研究报告了社会人口学亚组性能,发现女性和黑人患者阳性预测值低于对照组。总体而言,纳入研究极少报告亚组性能和训练人群构成,限制了公平性评估。
要点
- 阅片时需注意:纳入评价中仅1项研究报告了AI在不同性别和种族亚组间的性能差异
- 现有证据不足以判断这些商用AI产品在各亚组间是否存在不公平表现
- 报告AI工具时若缺乏亚组和训练数据信息,不应推断其公平或存在偏倚
核心结论
- 纳入的评价研究中亚组性能报告很少,仅凭1项研究的阳性预测值差异不足以描述各产品的公平性
- 缺失亚组和训练人群报告限制了公平性评估,但不能据此认定所评价系统存在不公平
研究设计与样本
- 检索了PubMed、Scopus、PubMed Central、Embase、Web of Science Core Collection及IEEE Xplore数据库
- 两名评价者独立筛选研究、提取数据并评估偏倚风险
- 依据CLAIM 2024、TRIPOD+AI和算法偏倚相关证据预设的8个报告条目进行评分,分为报告、部分报告或未报告
- 评价对象为监管批准用于成人急性卒中CT的商用AI工具,研究语言限定为英语
诊断标准与关键数值
- 仅1项研究报告了社会人口学亚组性能,结果显示女性患者的阳性预测值低于男性患者,为67.3%对72.7%,p=0.02
- 同一研究发现黑人患者的阳性预测值低于非黑人患者,为66.7%对73.2%,p=0.005
- 另1项研究报告了按中心、扫描仪或扫描方案分层的结果
- 没有纳入的同行评审研究描述AI工具训练数据的人口学构成
- 4项研究承认存在潜在偏倚或数据缺口,但未进行测量
临床应用与阅片要点
- 在日常阅片报告中若使用商用AI辅助急性卒中的CT评估,需意识到其在不同性别和种族亚组间的性能数据极为有限
- 解读AI结果时,不能默认AI在各亚组间表现一致,尤其在阳性预测值层面可能存在差异
- 当前缺乏按扫描设备和扫描方案分层的性能报告,跨机构部署AI时需谨慎
- 影像科室在采购和验证商用AI时应主动获取亚组性能和训练数据构成信息
局限性
- 纳入研究极少报告亚组性能,社会人口学亚组性能仅出自1项研究
- 所有纳入研究均未描述AI工具训练数据的人口学构成
- 仅凭1项研究的阳性预测值差异不足以全面描述各产品的公平性表现
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
RATIONALE AND OBJECTIVES: Commercial artificial intelligence (AI) tools cleared by regulators now triage acute stroke on computed tomography (CT) scans across health systems whose patients differ in age, sex, race, socioeconomic status, and imaging hardware. It is unknown whether studies of these tools report how performance varies across such groups.
MATERIALS AND METHODS: We systematically reviewed English-language studies of commercial, regulator-cleared AI tools for adult acute stroke CT tasks. PubMed, Scopus, and PubMed Central were searched from January 1, 2018, through June 2, 2026, and Embase, Web of Science Core Collection, and IEEE Xplore were searched on September 14, 2026, for records published from 2018 onward; the searches were supplemented by a backward-reference and forward-citation audit. Two reviewers independently screened studies, extracted data, and assessed risk of bias. Eight prespecified reporting items informed by CLAIM 2024, TRIPOD+AI, and evidence on algorithmic bias were scored as reported, partial, or not reported. The review was registered with PROSPERO (CRD420261403480).
RESULTS: Thirteen studies covering nine vendors and four CT tasks met the eligibility criteria. Only one study reported sociodemographic subgroup performance. It found lower positive predictive value for female patients than for male patients (67.3% vs. 72.7%; p = 0.02) and for Black patients than for non-Black patients (66.7% vs. 73.2%; p = 0.005). Performance by site, scanner, or protocol was reported in one other study. No included peer-reviewed study described the demographic composition of the tool's training data, and four studies acknowledged a potential bias or data gap without measuring it.
CONCLUSION: Subgroup performance was rarely reported in the included peer-reviewed evaluations. The finding of positive predictive value differences in one study is insufficient to characterize fairness across products. Missing subgroup and training-population reporting limits assessment of fairness, but does not establish that the evaluated systems are inequitable.
引用
- Adiniaev, Omar, Timor, et al. Reporting of Fairness and Equity in Studies of Regulator-cleared Commercial AI for Acute Stroke CT: A Systematic Review. Acad Radiol. 2026. PMID: 42907264. doi:10.1016/j.acra.2026.09.075
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。