文献速递乳房整形与再造美容整形

当前人工智能大语言模型作为美容整形术后医疗支持聊天机器人的效用与比较性能

Utility and Comparative Performance of Current Artificial Intelligence Large Language Models as Postoperative Medical Support Chatbots in Aesthetic Surgery.
PubMed 文献 · Aesthet Surg J · 2024-01-01

研究比较了ChatGPT与Bard在美容乳房整形术后医疗支持聊天机器人中的表现。ChatGPT在整体准确率上显著优于Bard(66% vs 55%,P<.05),在提供最可能诊断(91% vs 68%,P<.01)和红色警示症状(79% vs 54%,P<.01)方面准确性更高。结论认为ChatGPT是更优的大语言模型,但其性能不完美,需改进以促进应用。

要点

  • ChatGPT整体准确率66%,显著优于Bard的55%(P<.05)。
  • ChatGPT最可能诊断准确率91%,高于Bard的68%(P<.01)。
  • 红色警示症状识别准确率ChatGPT 79% vs Bard 54%(P<.01)。

核心结论

  • ChatGPT是用于术后医疗支持聊天机器人潜在应用的更优大语言模型。
  • 其不完美的性能和讨论的局限性可为促进应用所需的必要改进提供指导。

研究设计与样本

  • 研究设计了22个模拟的美容乳房整形术后患者表现,并经专家验证。
  • 并发症在术后潜伏期和所需医疗关注紧迫性上各不相同。
  • 评估Open AI的ChatGPT和Google的Bard未修改且免费可用版本在生成适当鉴别诊断、最可能诊断、建议医疗处置、居家开始的治疗或干预、和/或指示恶化的红色警示体征/症状方面的比较准确性。

主要发现

  • ChatGPT在所有检查的客观评估指标上累积且显著优于Bard(66% vs 55%,P<.05)。
  • 生成适当鉴别诊断的准确性ChatGPT为61%,Bard为57%(P=.45)。
  • ChatGPT平均询问9.2个病史问题,Bard为6.8个(P<.001)。
  • 报告最可能诊断的准确性ChatGPT为91%,Bard为68%(P<.01)。
  • 建议适当医疗处置的准确性ChatGPT为50%,Bard为41%(P=.40);适当居家干预/治疗的准确性分别为59% vs 55%(P=.94);红色警示体征/症状的准确性分别为79% vs 54%(P<.01)。

临床怎么用

  • 原文未据此给出推荐。

局限性

  • 原文未报告局限性。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

BACKGROUND: Large language models (LLMs) have revolutionized the way plastic surgeons and their patients can access and leverage artificial intelligence (AI). OBJECTIVES: The present study aims to compare the performance of 2 current publicly available and patient-accessible LLMs in the potential application of AI as postoperative medical support chatbots in an aesthetic surgeon's practice. METHODS: Twenty-two simulated postoperative patient presentations following aesthetic breast plastic surgery were devised and expert-validated. Complications varied in their latency within the postoperative period, as well as urgency of required medical attention. In response to each patient-reported presentation, Open AI's ChatGPT and Google's Bard, in their unmodified and freely available versions, were objectively assessed for their comparative accuracy in generating an appropriate differential diagnosis, most-likely diagnosis, suggested medical disposition, treatments or interventions to begin from home, and/or red flag signs/symptoms indicating deterioration. RESULTS: ChatGPT cumulatively and significantly outperformed Bard across all objective assessment metrics examined (66% vs 55%, respectively; P < .05). Accuracy in generating an appropriate differential diagnosis was 61% for ChatGPT vs 57% for Bard (P = .45). ChatGPT asked an average of 9.2 questions on history vs Bard's 6.8 questions (P < .001), with accuracies of 91% vs 68% reporting the most-likely diagnosis, respectively (P < .01). Appropriate medical dispositions were suggested with accuracies of 50% by ChatGPT vs 41% by Bard (P = .40); appropriate home interventions/treatments with accuracies of 59% vs 55% (P = .94), and red flag signs/symptoms with accuracies of 79% vs 54% (P < .01), respectively. Detailed and comparative performance breakdowns according to complication latency and urgency are presented. CONCLUSIONS: ChatGPT represents the superior LLM for the potential application of AI technology in postoperative medical support chatbots. Imperfect performance and limitations discussed may guide the necessary refinement to facilitate adoption.

来源:Aesthet Surg J,查看原文。版权归原作者所有。

引用

  1. Abi-Rafeh J, Henry N, Xu HH, et al. Utility and Comparative Performance of Current Artificial Intelligence Large Language Models as Postoperative Medical Support Chatbots in Aesthetic Surgery. Aesthet Surg J. 2024. PMID: 38318684. doi:10.1093/asj/sjae025

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。