文献速递骨肌MR

使用ChatGPT-4将肌肉骨骼放射学报告翻译为患者友好摘要

Translating musculoskeletal radiology reports into patient-friendly summaries using ChatGPT-4.
PubMed 文献 · Skeletal Radiol · 2024-01-25

本研究评估使用ChatGPT-4为60份肌肉骨骼MR报告(肩、膝、腰椎各20份)生成简明准确的外行摘要的可行性。所有摘要均满足字数和可读性标准,三位独立读者对准确性和完整性的平均评分较高(准确性2.58-2.77,完整性2.73-2.87),仅少数摘要可能造成混淆或不准确。研究提示生成式AI有望自动化生成患者友好的肌肉骨骼MR影像摘要。

要点

  • ChatGPT-4生成的肌肉骨骼MR报告外行摘要总体准确性和完整性评分较高。
  • 读者间对摘要准确性和完整性的评分一致性较低(准确性kappa 0.33,完整性kappa 0.29)。
  • 少数摘要被评分为1分(不正确或不完整,可能造成混淆或伤害)。

核心结论

  • AI生成的肌肉骨骼MR报告外行摘要的准确性和完整性总体评分较高,仅少数可能造成混淆或不准确;本研究展示了利用生成式AI(如ChatGPT-4)自动化生成患者友好摘要的潜力。

研究设计与样本

  • 可行性研究,通过PACS获取60份放射学报告(20份肩MR、20份膝MR、20份腰椎MR),去标识化后提交ChatGPT-4生成外行摘要;由三名独立读者(两名主要读者和一名后续加入用于验证的读者)评估摘要的完整性和准确性。

诊断标准与关键数值

  • 摘要评分采用1-3分制:1分表示摘要不正确或不完整,可能提供有害或混淆信息;2分表示摘要大部分正确和完整,不太可能造成混淆或伤害;3分表示摘要完全正确和完整。准确性平均评分:读者1为2.58,读者2为2.71,读者3为2.77;完整性平均评分:读者1为2.87,读者2为2.73,读者3为2.87。准确性评分为1分的摘要数量:读者1识别3份,读者2识别1份,读者3识别0份。读者间一致性:准确性kappa 0.33,完整性kappa 0.29;纳入第三位读者评分后读者间一致性无统计学显著变化。

临床应用与阅片要点

  • 作者认为,本研究展示了利用生成式AI(如ChatGPT-4)自动化生成肌肉骨骼MR影像患者友好摘要的潜力。

局限性

  • 两名主要读者间对准确性和完整性的评分一致性较低(准确性kappa 0.33,完整性kappa 0.29);仅少数摘要被评分为1分(不正确或不完整,可能造成混淆或伤害)。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

英文原文(摘要)· 点击展开

OBJECTIVE: To assess the feasibility of using large language models (LLMs), specifically ChatGPT-4, to generate concise and accurate layperson summaries of musculoskeletal radiology reports.

METHODS: Sixty radiology reports, comprising 20 MR shoulder, 20 MR knee, and 20 MR lumbar spine reports, were obtained via PACS. The reports were deidentified and then submitted to ChatGPT-4, with the prompt "Produce an organized and concise layperson summary of the findings of the following radiology report. Target a reading level of 8-9th grade and word count <300 words." Three (two primary and one later added for validation) independent readers evaluated the summaries for completeness and accuracy compared to the original reports. Summaries were rated on a scale of 1 to 3: 1) summaries that were incorrect or incomplete, potentially providing harmful or confusing information; 2) summaries that were mostly correct and complete, unlikely to cause confusion or harm; and 3) summaries that were entirely correct and complete.

RESULTS: All 60 responses met the criteria for word count and readability. Mean ratings for accuracy were 2.58 for reader 1, 2.71 for reader 2, and 2.77 for reader 3. Mean ratings for completeness were 2.87 for reader 1 and 2.73 for reader 2 and 2.87 for reader 3. For accuracy, reader 1 identified three summaries as a 1, reader 2 identified one, and reader 3 identified none. For the two primary readers, inter-reader agreement was low for accuracy (kappa 0.33) and completeness (kappa 0.29). There were no statistically significant changes in inter-reader agreement when the third reader's ratings were included in analysis.

CONCLUSION: Overall ratings for accuracy and completeness of the AI-generated layperson report summaries were high with only a small minority likely to be confusing or inaccurate. This study illustrates the potential for leveraging generative AI, such as ChatGPT-4, to automate the production of patient-friendly summaries for musculoskeletal MR imaging.

来源:Skeletal Radiol,查看原文。版权归原作者所有。

引用

  1. Kuckelman IJ, Wetley K, Yi PH, et al. Translating musculoskeletal radiology reports into patient-friendly summaries using ChatGPT-4. Skeletal Radiol. 2024. PMID: 38270616. doi:10.1007/s00256-024-04599-2

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。