文献速递骨肌DR

用于X线膝关节骨关节炎严重程度分类的人工智能工具的外部验证

External validation of an artificial intelligence tool for radiographic knee osteoarthritis severity classification.
PubMed 文献 · Eur J Radiol · 2022-03-12

这项回顾性外部验证研究纳入2019年9月24日至10月22日连续50例患者(排除3例)共99个膝关节,以负重非固定屈曲后前位膝关节X线片评估人工智能工具按Kellgren-Lawrence分级与两位肌骨放射学顾问共识的一致性。人工智能工具与顾问共识的二次加权kappa为0.88(95%CI 0.82-0.92),与顾问间一致性0.89(95%CI 0.85-0.93)相近。

要点

  • 人工智能工具与顾问共识的二次加权kappa为0.88(95%CI 0.82-0.92),与顾问间0.89相近。
  • 人工智能工具序数加权准确率为97.8%(95%CI 96.9-98.6%)。
  • 平均多分类准确率为84%(83/99,95%CI 77-91%),F1分数为0.67(95%CI 0.51-0.81)。

核心结论

  • 人工智能工具在X线膝关节骨关节炎严重程度分类上与放射学顾问共识达到与顾问彼此间相同水平的良好至优秀一致性。

研究设计与样本

  • 回顾性、连续患者样本的外部验证研究,使用临床生产PACS中的负重非固定屈曲后前位膝关节X线片,纳入50例连续患者(排除3例)共99个膝关节(排除1个)。

诊断标准与关键数值

  • 参考标准为两位肌骨放射学顾问的共识,按序数Kellgren-Lawrence分级评估。
  • 人工智能工具与顾问共识的二次加权kappa为0.88(95%CI 0.82-0.92)。
  • 顾问间二次加权kappa为0.89(95%CI 0.85-0.93)。
  • 人工智能工具的序数加权准确率为97.8%(95%CI 96.9-98.6%)。
  • 人工智能工具的平均多分类准确率为84%(83/99,95%CI 77-91%),F1分数为0.67(95%CI 0.51-0.81)。

局限性

  • 原文未报告局限性。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

英文原文(摘要)· 点击展开

PURPOSE: To externally validate an artificial intelligence (AI) tool for radiographic knee osteoarthritis severity classification on a clinical dataset.

METHOD: This retrospective, consecutive patient sample, external validation study used weight-bearing, non-fixed-flexion posterior-anterior knee radiographs from a clinical production PACS. The index test was ordinal Kellgren-Lawrence grading by an AI tool, two musculoskeletal radiology consultants, two reporting technologists, and two resident radiologists. Grading was repeated by all readers after at least four weeks. Reference test was the consensus of the two consultants. The primary outcome was quadratic weighted kappa. Secondary outcomes were ordinal weighted accuracy, multiclass accuracy and F1-score.

RESULTS: 50 consecutive patients between September 24, 2019 and October 22, 2019 were retrospectively included (3 excluded) totaling 99 knees (1 excluded). Quadratic weighted kappa for the AI tool and the consultant consensus was 0.88 CI95% (0.82-0.92). Agreement between the consultants was 0.89 CI95% (0.85-0.93). Intra-rater agreements for the consultants were 0.96 CI95% (0.94-0.98) and 0.94 CI95% (0.91-0.96) respectively. For the AI tool it was 1 CI95% (1-1). For the AI tool, ordinal weighted accuracy was 97.8% CI95% (96.9-98.6 %). Average multiclass accuracy and F1-score were 84% (83/99) CI95% (77-91%) and 0.67 CI95% (0.51-0.81).

CONCLUSIONS: The AI tool achieved the same good-to-excellent agreement with the radiology consultant consensus for radiographic knee osteoarthritis severity classification as the consultants did with each other.

来源:Eur J Radiol,查看原文。版权归原作者所有。

引用

  1. Brejnebøl, Hansen, Nybing, et al. External validation of an artificial intelligence tool for radiographic knee osteoarthritis severity classification. Eur J Radiol. 2022. PMID: 35338955. doi:10.1016/j.ejrad.2022.110249

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。