文献速递乳腺技术与安全乳腺X线USMR

评估GPT作为放射学决策辅助工具:乳腺影像试点中GPT-4与GPT-3.5的比较

Evaluating GPT as an Adjunct for Radiologic Decision Making: GPT-4 Versus GPT-3.5 in a Breast Imaging Pilot.
PubMed 文献 · J Am Coll Radiol · 2023-06-21

本研究比较ChatGPT-3.5与GPT-4在乳腺癌筛查和乳腺疼痛两种临床表现中推荐合适影像检查的能力,以ACR适宜性标准为参考。开放式提问下两者筛查平均得分均为1.830(满分2);选择所有适用项格式下GPT-3.5平均正确率88.9%、GPT-4为98.4%。乳腺疼痛方面GPT-3.5开放式平均得分1.125、选择所有适用项平均正确率58.3%,GPT-4分别为1.666和77.7%。

要点

  • 乳腺癌筛查提示下GPT-4选择所有适用项平均正确率98.4%,GPT-3.5为88.9%。
  • 乳腺疼痛提示下GPT-4开放式平均得分1.666、选择所有适用项平均正确率77.7%,均高于GPT-3.5的1.125和58.3%。
  • 作者认为结果展示了大语言模型用于放射学决策的最终可行性,但需更多用例和更高准确性来评估和实施。

核心结论

  • 结果展示了大语言模型如ChatGPT用于放射学决策的最终可行性,可能改善临床工作流程和放射学服务的合理使用,但评估和实施此类工具需要更多用例和更高准确性。

研究设计与样本

  • 研究比较ChatGPT-3.5和GPT-4对乳腺癌筛查与乳腺疼痛的应答与ACR适宜性标准的一致性,提示格式包括开放式和选择所有适用项两种,每种提示重复三次取平均值确定最终得分。

诊断标准与关键数值

  • 评分标准评估所提出的影像检查方式是否符合ACR指南;乳腺癌筛查提示下ChatGPT-3.5与ChatGPT-4开放式平均得分均为1.830(满分2),GPT-3.5选择所有适用项平均正确率88.9% vs GPT-4 98.4%;乳腺疼痛提示下GPT-3.5开放式平均得分1.125、选择所有适用项平均正确率58.3%,GPT-4开放式平均得分1.666、选择所有适用项平均正确率77.7%。

临床应用与阅片要点

  • 作者认为结果展示了大语言模型用于放射学决策的最终可行性,可能改善临床工作流程和放射学服务的合理使用,但评估和实施此类工具需要更多用例和更高准确性。

局限性

  • 作者指出需要更多用例和更高准确性来评估和实施此类工具。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

英文原文(摘要)· 点击展开

OBJECTIVE: Despite rising popularity and performance, studies evaluating the use of large language models for clinical decision support are lacking. Here, we evaluate ChatGPT (Generative Pre-trained Transformer)-3.5 and GPT-4's (OpenAI, San Francisco, California) capacity for clinical decision support in radiology via the identification of appropriate imaging services for two important clinical presentations: breast cancer screening and breast pain.

METHODS: We compared ChatGPT's responses to the ACR Appropriateness Criteria for breast pain and breast cancer screening. Our prompt formats included an open-ended (OE) and a select all that apply (SATA) format. Scoring criteria evaluated whether proposed imaging modalities were in accordance with ACR guidelines. Three replicate entries were conducted for each prompt, and the average of these was used to determine final scores.

RESULTS: Both ChatGPT-3.5 and ChatGPT-4 achieved an average OE score of 1.830 (out of 2) for breast cancer screening prompts. ChatGPT-3.5 achieved a SATA average percentage correct of 88.9%, compared with ChatGPT-4's average percentage correct of 98.4% for breast cancer screening prompts. For breast pain, ChatGPT-3.5 achieved an average OE score of 1.125 (out of 2) and a SATA average percentage correct of 58.3%, as compared with an average OE score of 1.666 (out of 2) and a SATA average percentage correct of 77.7%.

DISCUSSION: Our results demonstrate the eventual feasibility of using large language models like ChatGPT for radiologic decision making, with the potential to improve clinical workflow and responsible use of radiology services. More use cases and greater accuracy are necessary to evaluate and implement such tools.

来源:J Am Coll Radiol,查看原文。版权归原作者所有。

引用

  1. Rao, Kim, Kamineni, et al. Evaluating GPT as an Adjunct for Radiologic Decision Making: GPT-4 Versus GPT-3.5 in a Breast Imaging Pilot. J Am Coll Radiol. 2023. PMID: 37356806. doi:10.1016/j.jacr.2023.05.003

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。