文献速递骨肌DR

多模态大语言模型与临床医生膝关节骨关节炎X线分级对比研究

Performance of multimodal large language models versus clinicians for radiographic knee osteoarthritis grading: A multiobserver study.
PubMed 文献 · Skeletal Radiol · 2026-07-24

回顾性单中心研究纳入348张膝关节X线片,以资深肌骨放射科医生为参考标准,比较临床医生与两代多模态大语言模型(ChatGPT-4o、ChatGPT-5.0)的Kellgren-Lawrence分级表现。结果显示ChatGPT-5.0优于ChatGPT-4o,但两者在详细分级、影像征象解读及可重复性方面仍不及临床医生。

要点

  • ChatGPT-5.0对KOA二元检测敏感度达0.96,但特异度有限,不宜单独用于X线评估。
  • 放射科医生与参考标准一致性最高(κ=0.87),优于骨科医生、住院医师和所有LLM。
  • LLM对KL 0和KL 4分级较准确,对KL 1-2级判断能力有限,需警惕中间级别误判。

核心结论

  • ChatGPT-5.0在KL分级、征象解读和可重复性方面优于ChatGPT-4o,但两者均不及临床医生。
  • 当前多模态LLM对KOA二元检测敏感度高而特异度有限,不适合独立用于膝关节X线骨关节炎分级。

研究设计与样本

  • 回顾性单中心研究,纳入348张膝关节X线片,由资深肌骨放射科医生作为参考标准进行KL分级。
  • 评价者包括放射科医生、骨科医生、放射科住院医师及两个多模态大语言模型(ChatGPT-4o和ChatGPT-5.0)。
  • 评价维度包括二元KOA检测(KL 0-1对比KL≥2)、影像征象解读和观察者内可重复性。

关键影像表现/征象

  • 影像征象解读涵盖关节间隙狭窄、骨赘和软骨下硬化三项X线表现,各征象解读一致性均为中等水平。
  • 按级别分类在KL 0和KL 4最为准确,KL 1-2级的分类准确性仍然有限。

诊断标准与关键数值

  • 与参考标准一致性:放射科医生κ=0.87最高,其次为骨科医生和放射科住院医师,两个LLM为中等一致性。
  • 二元KOA检测中ChatGPT-5.0敏感度为0.96,但特异度有所降低。
  • 观察者内可重复性:参考判读者κ=0.881最高,骨科医生κ=0.634,放射科医生κ=0.626,住院医师κ=0.478。
  • LLM可重复性:ChatGPT-5.0的κ为0.591,高于ChatGPT-4o的κ=0.485。

临床应用与阅片要点

  • 放射科医生KL分级一致性最高,日常工作中应作为膝关节骨关节炎X线分级的核心判读者。
  • LLM可作为高敏感度筛查工具辅助发现KOA,但需由临床医生复核确认,尤其注意假阳性。
  • 阅片时重点关注KL 1-2级别,该区间所有评价者包括LLM的准确性均有限。

局限性

  • 单中心回顾性研究设计,样本量348张X线片,参考标准仅由一位资深肌骨放射科医生确定。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

英文原文(摘要)· 点击展开

OBJECTIVE: To compare the performance of clinicians and two generations of multimodal large language models (LLMs) in Kellgren-Lawrence (KL) grading of knee osteoarthritis (KOA), including feature-level assessment and intraobserver repeatability.

MATERIALS AND METHODS: In this retrospective single-center study, 348 knee radiographs were graded by a senior musculoskeletal radiologist (reference standard), a radiologist, an orthopedic surgeon, a radiology resident, and LLMs (ChatGPT-4o and ChatGPT-5.0). Binary KOA detection (KL 0-1 vs ≥ 2), feature-level interpretation (joint space narrowing, osteophytes, subchondral sclerosis), and intraobserver repeatability were evaluated. Agreement metrics included weighted κ, accuracy, and standard diagnostic measures.

RESULTS: Agreement with the reference standard was highest for the radiologist (κ = 0.87), followed by the orthopedic surgeon and radiology resident. Both LLMs demonstrated moderate agreement, with ChatGPT-5.0 outperforming ChatGPT-4o. For binary KOA detection, ChatGPT-5.0 showed very high sensitivity (0.96) but reduced specificity. Per-grade classification was most accurate for KL 0 and KL 4, but remained limited for KL 1-2. Feature-level concordance was modest across all radiographic findings. Intraobserver repeatability was highest for the reference reader (κ = 0.881), followed by the orthopedic surgeon (κ = 0.634) and radiologist (κ = 0.626), while lower agreement was observed for the resident (κ = 0.478) and LLMs, with ChatGPT-5.0 showing higher consistency than ChatGPT-4o (κ = 0.591 vs. 0.485).

CONCLUSION: Although ChatGPT-5.0 outperformed ChatGPT-4o, both models remained inferior to clinicians in detailed KL grading, feature-level interpretation, and reproducibility. Current multimodal LLMs show high sensitivity but limited specificity and are not suitable for standalone radiographic KOA assessment.

来源:Skeletal Radiol,查看原文。版权归原作者所有。

引用

  1. Akdogan, Akdogan, Tumer, et al. Performance of multimodal large language models versus clinicians for radiographic knee osteoarthritis grading: A multiobserver study. Skeletal Radiol. 2026. PMID: 42498839. doi:10.1007/s00256-026-05322-z

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。