膝骨关节炎X线评估中观察者间一致性与并发人工智能辅助的表现
这项回顾性观察者表现研究纳入三个欧洲中心连续站立位膝关节X线片共225例,由11名不同年资的放射科与骨科读者在有无商业AI工具辅助下按Kellgren-Lawrence分级评估。结果显示AI辅助提高了部分初级读者的分级表现(AUC 0.81±0.017 vs 0.88±0.011等),并使全体读者间一致性从κ=0.77±0.018升至0.85±0.013(P<.001)。
要点
- AI辅助使全体读者KL分级观察者间一致性由κ=0.77升至0.85(P<.001)。
- 6名初级读者中3名在AI辅助下KL分级AUC高于无辅助(P<.001至P=.008)。
- 认证放射科医师AI辅助下一致性κ=0.90±0.01,高于参考读者独立阅片的0.84±0.017(P=.01)。
核心结论
- AI辅助提高了初级读者的膝骨关节炎X线KL分级表现,并提高了所有读者及各经验水平的观察者间一致性。
研究设计与样本
- 回顾性观察者表现研究,2019年4月至2022年5月从三个欧洲中心连续纳入疑似膝骨关节炎患者的站立位膝关节X线片,每中心75例、共225例(平均年龄55岁±15[SD],女性113例)。
- 每中心招募4名放射科与骨科读者,涵盖培训中与认证后经验水平,共11名读者完成阅片,分别在有无商业AI工具辅助下评估正位片KL分级(KL-0=无膝骨关节炎,KL-4=重度膝骨关节炎)。
- 以三名肌肉骨骼放射科顾问医师的多数投票作为参考标准,采用序贯受试者工作特征方法估计分级表现。
诊断标准与关键数值
- KL分级分布:KL-0占24.0%(54例)、KL-1占28.0%(63例)、KL-2占21.8%(49例)、KL-3占18.7%(42例)、KL-4占7.6%(17例)。
- 6名初级读者中3名AI辅助下KL分级AUC高于无辅助:0.81±0.017 vs 0.88±0.011(P<.001);0.76±0.018 vs 0.86±0.013(P<.001);0.89±0.011 vs 0.91±0.009(P=.008)。
- 全体读者KL分级观察者间一致性AI辅助下高于无辅助:κ=0.77±0.018 vs 0.85±0.013(P<.001)。
- 认证放射科医师AI辅助下KL分级一致性κ=0.90±0.01,高于参考读者独立阅片的κ=0.84±0.017(P=.01)。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
Background Due to conflicting findings in the literature, there are concerns about a lack of objectivity in grading knee osteoarthritis (KOA) on radiographs. Purpose To examine how artificial intelligence (AI) assistance affects the performance and interobserver agreement of radiologists and orthopedists of various experience levels when evaluating KOA on radiographs according to the established Kellgren-Lawrence (KL) grading system. Materials and Methods In this retrospective observer performance study, consecutive standing knee radiographs from patients with suspected KOA were collected from three participating European centers between April 2019 and May 2022. Each center recruited four readers across radiology and orthopedic surgery at in-training and board-certified experience levels. KL grading (KL-0 = no KOA, KL-4 = severe KOA) on the frontal view was assessed by readers with and without assistance from a commercial AI tool. The majority vote of three musculoskeletal radiology consultants established the reference standard. The ordinal receiver operating characteristic method was used to estimate grading performance. Light kappa was used to estimate interrater agreement, and bootstrapped t statistics were used to compare groups. Results Seventy-five studies were included from each center, totaling 225 studies (mean patient age, 55 years ± 15 [SD]; 113 female patients). The KL grades were KL-0, 24.0% (n = 54); KL-1, 28.0% (n = 63); KL-2, 21.8% (n = 49); KL-3, 18.7% (n = 42); and KL-4, 7.6% (n = 17). Eleven readers completed their readings. Three of the six junior readers showed higher KL grading performance with versus without AI assistance (area under the receiver operating characteristic curve, 0.81 ± 0.017 [SEM] vs 0.88 ± 0.011 [P < .001]; 0.76 ± 0.018 vs 0.86 ± 0.013 [P < .001]; and 0.89 ± 0.011 vs 0.91 ± 0.009 [P = .008]). Interobserver agreement for KL grading among all readers was higher with versus without AI assistance (κ = 0.77 ± 0.018 [SEM] vs 0.85 ± 0.013; P < .001). Board-certified radiologists achieved almost perfect agreement for KL grading when assisted by AI (κ = 0.90 ± 0.01), which was higher than that achieved by the reference readers independently (κ = 0.84 ± 0.017; P = .01). Conclusion AI assistance increased junior readers' radiographic KOA grading performance and increased interobserver agreement for osteoarthritis grading across all readers and experience levels. Published under a CC BY 4.0 license. Supplemental material is available for this article.
引用
- Brejnebøl MW, Lenskjold A, Ziegeler K, et al. Interobserver Agreement and Performance of Concurrent AI Assistance for Radiographic Evaluation of Knee Osteoarthritis. Radiology. 2024. PMID: 38980184. doi:10.1148/radiol.233341
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。