人工智能系统识别胸部X线假阴性判读的价值研究
本研究连续收集单中心2020年3月判读为正常的4208张胸部X线片,回顾性应用商业化AI系统,并以三位胸部放射科医师多数投票为参考标准。AI系统检测出初始被忽略的相关异常,检出率为2.4%,假转诊率为14.0%;在「AI作为顾问」场景下,放射科医师检出率为1.2%,假转诊率为0.97%。结论认为AI系统可识别被放射科医师忽略的相关异常,并通过反馈帮助放射科医师纠正假阴性判读。
要点
- AI系统在判读为正常的胸片中检出2.4%被忽略的相关异常,假转诊率14.0%。
- AI作为顾问反馈后,放射科医师检出率1.2%,假转诊率降至0.97%。
- 决策曲线分析显示,当成本效益比低于1:0.8时,AI作为顾问场景净获益为正。
核心结论
- AI系统可识别放射科医师忽略的相关异常,并通过反馈使放射科医师纠正初始假阴性判读。
研究设计与样本
- 单中心连续收集2020年3月判读为正常的胸部X线片,回顾性应用商业化AI系统;参考标准为三位胸部放射科医师多数投票。
诊断标准与关键数值
- AI系统检测初始被忽略相关异常的检出率为2.4%,假转诊率(FRR)为14.0%。
临床应用与阅片要点
- 在「AI作为顾问」场景下,放射科医师检出初始被忽略相关异常的检出率为1.2%,假转诊率为0.97%。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
OBJECTIVES: To investigate the efficacy of an artificial intelligence (AI) system for the identification of false negatives in chest radiographs that were interpreted as normal by radiologists.
METHODS: We consecutively collected chest radiographs that were read as normal during 1 month (March 2020) in a single institution. A commercialized AI system was retrospectively applied to these radiographs. Radiographs with abnormal AI results were then re-interpreted by the radiologist who initially read the radiograph ("AI as the advisor" scenario). The reference standards for the true presence of relevant abnormalities in radiographs were defined by majority voting of three thoracic radiologists. The efficacy of the AI system was evaluated by detection yield (proportion of true-positive identification among the entire examination) and false-referral rate (FRR, proportion of false-positive identification among all examinations). Decision curve analyses were performed to evaluate the net benefits of applying the AI system.
RESULTS: A total of 4208 radiographs from 3778 patients (M:F = 1542:2236; median age, 56 years) were included. The AI system identified initially overlooked relevant abnormalities with a detection yield and an FRR of 2.4% and 14.0%, respectively. In the "AI as the advisor" scenario, radiologists detected initially overlooked relevant abnormalities with a detection yield and FRR of 1.2% and 0.97%, respectively. In a decision curve analysis, AI as an advisor scenario exhibited a positive net benefit when the cost-to-benefit ratio was below 1:0.8.
CONCLUSION: An AI system could identify relevant abnormalities overlooked by radiologists and could enable radiologists to correct their false-negative interpretations by providing feedback to radiologists.
KEY POINTS: • In consecutive chest radiographs with normal interpretations, an artificial intelligence system could identify relevant abnormalities that were initially overlooked by radiologists. • The artificial intelligence system could enable radiologists to correct their initial false-negative interpretations by providing feedback to radiologists when overlooked abnormalities were present.
引用
- Hwang, Park, Hong, et al. Artificial intelligence system for identification of false-negative interpretations in chest radiographs. Eur Radiol. 2022. PMID: 35195744. doi:10.1007/s00330-022-08593-x
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。