文献速递乳腺US

利用大语言模型进行乳腺超声结构化报告:Open AI(GPT-4.0)与Microsoft Bing(GPT-4)的比较研究

Harnessing Large Language Models for Structured Reporting in Breast Ultrasound: A Comparative Study of Open AI (GPT-4.0) and Microsoft Bing (GPT-4).
PubMed 文献 · Ultrasound Med Biol · 2024-08-12

本研究回顾性收集2023年1月至5月间接受手术的100份自由文本乳腺超声报告,评估Open AI(GPT-4.0)和Microsoft Bing(GPT-4)生成结构化报告、BI-RADS分类和管理建议的能力。GPT-4.0在结构化报告质量(88% vs. 55%)、正确BI-RADS分类(54% vs. 47%)和合理管理建议(81% vs. 63%)方面均优于Bing,且预测良恶性特征的AUC更高(0.9317 vs. 0.8177),但两者均显著低于高年资放射科医生(AUC 0.9763)。

要点

  • GPT-4.0生成结构化报告的质量优于Bing(88% vs. 55%,p<0.001)。
  • GPT-4.0提供正确BI-RADS分类的比例为54%,Bing为47%(p=0.013)。
  • 两者预测良恶性的AUC均低于高年资放射科医生(0.9317/0.8177 vs. 0.9763)。

核心结论

  • 大语言模型(尤其是GPT-4.0)有潜力将非结构化乳腺超声报告转换为结构化报告,并提供准确诊断和合理建议。

研究设计与样本

  • 回顾性研究,纳入100份2023年1月至5月间接受手术患者的自由文本乳腺超声报告,比较Open AI(GPT-4.0)与Microsoft Bing(GPT-4)生成结构化报告、BI-RADS分类和管理建议的能力,由高年资放射科医生依据指南评估。

诊断标准与关键数值

  • 结构化报告质量:GPT-4.0 88% vs. Bing 55%(p<0.001)。
  • 正确BI-RADS分类:GPT-4.0 54% vs. Bing 47%(p=0.013)。
  • 合理管理建议:GPT-4.0 81% vs. Bing 63%(p<0.001)。
  • 预测良恶性特征的AUC:GPT-4.0 0.9317 vs. Bing 0.8177(p<0.001);两者均显著低于高年资放射科医生(AUC 0.9763,均p<0.001)。

临床应用与阅片要点

  • 高年资放射科医生依据指南评估结构化报告、BI-RADS分类和管理建议的质量。
  • GPT-4.0在结构化报告、BI-RADS分类、管理建议及良恶性预测方面均优于Bing。
  • 两种大语言模型的良恶性预测能力均显著低于高年资放射科医生。

局限性

  • 原文未报告局限性。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

英文原文(摘要)· 点击展开

OBJECTIVES: To assess the capabilities of large language models (LLMs), including Open AI (GPT-4.0) and Microsoft Bing (GPT-4), in generating structured reports, the Breast Imaging Reporting and Data System (BI-RADS) categories, and management recommendations from free-text breast ultrasound reports.

MATERIALS AND METHODS: In this retrospective study, 100 free-text breast ultrasound reports from patients who underwent surgery between January and May 2023 were gathered. The capabilities of Open AI (GPT-4.0) and Microsoft Bing (GPT-4) to convert these unstructured reports into structured ultrasound reports were studied. The quality of structured reports, BI-RADS categories, and management recommendations generated by GPT-4.0 and Bing were evaluated by senior radiologists based on the guidelines.

RESULTS: Open AI (GPT-4.0) was better than Microsoft Bing (GPT-4) in terms of performance in generating structured reports (88% vs. 55%; p < 0.001), giving correct BI-RADS categories (54% vs. 47%; p = 0.013) and providing reasonable management recommendations (81% vs. 63%; p < 0.001). As the ability to predict benign and malignant characteristics, GPT-4.0 performed significantly better than Bing (AUC, 0.9317 vs. 0.8177; p < 0.001), while both performed significantly inferior to senior radiologists (AUC, 0.9763; both p < 0.001).

CONCLUSION: This study highlights the potential of LLMs, specifically Open AI (GPT-4.0), in converting unstructured breast ultrasound reports into structured ones, offering accurate diagnoses and providing reasonable recommendations.

来源:Ultrasound Med Biol,查看原文。版权归原作者所有。

引用

  1. Liu C, Wei M, Qin Y, et al. Harnessing Large Language Models for Structured Reporting in Breast Ultrasound: A Comparative Study of Open AI (GPT-4.0) and Microsoft Bing (GPT-4). Ultrasound Med Biol. 2024. PMID: 39138026. doi:10.1016/j.ultrasmedbio.2024.07.007

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。