评估基于人工智能的聊天机器人作为脑肿瘤患者咨询代理的适用性:一项综合性调查分析
研究使用ChatGPT-3.5、Bing Chat和Google Bard回答脑肿瘤患者常见问题,由脑肿瘤委员会成员盲法评判。三种聊天机器人在准确性和沟通能力上无显著差异(P=0.253,0.090),但在共情方面Bing Chat和Google Bard优于ChatGPT(P=0.004,0.002)。研究旨在识别AI聊天机器人用于患者医疗目的时所需的改进。
要点
- 三种聊天机器人在准确性和沟通能力上无显著差异。
- Bing Chat和Google Bard在共情方面优于ChatGPT。
- 医学专业人员须确保聊天机器人提供的信息准确安全。
核心结论
- 基于人工智能的聊天机器人是患者和公众获取医疗信息的便捷方式,医学专业人员必须确保提供给聊天机器人用户的信息准确且安全。
研究设计与样本
- 使用三种代表性聊天机器人(ChatGPT-3.5、Bing Chat、Google Bard)回答由脑肿瘤委员会选定和模拟的脑肿瘤患者常见问题,所有回答未经修改收集,由委员会成员在不知聊天机器人类型的情况下盲法评判。
主要发现
- 三种聊天机器人在准确性和沟通能力方面无显著差异(P=0.253,P=0.090)。
- 在共情方面,Bing Chat和Google Bard优于ChatGPT(P=0.004,P=0.002)。
临床怎么用
- 原文未据此给出推荐。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
OBJECTIVE: The internet, particularly social media, has become a popular resource for learning about health and investigating one's own health conditions. The development of artificial intelligence (AI) chatbots has been fueled by the increasing availability of digital health data and advances in natural language processing techniques. While these chatbots are more accessible than before, they sometimes fail to provide accurate information. METHODS: We used representative chatbots currently available (Chat Generative Pretrained Transformer-3.5, Bing Chat, and Google Bard) to answer questions commonly asked by brain tumor patients. The simulated situations with questions were made and selected by the brain tumor committee. These questions are commonly asked by brain tumor patients. The goal of the study was introduced to each chatbot, the situation was explained, and questions were asked. All responses were collected without modification. The answers were shown to the committee members, and they were asked to judge the responses while blinded to the type of chatbot. RESULTS: There was no significant difference in accuracy and communication ability among the 3 groups (P = 0.253, 0.090, respectively). For empathy, Bing Chat and Google Bard were superior to Chat Generative Pretrained Transformer (P = 0.004, 0.002, respectively). The purpose of this study was not to assess or verify the relative superiority of each chatbot. Instead, the aim was to identify the shortcomings and changes needed if AI chatbots are to be used for patient medical purposes. CONCLUSION: AI-based chatbots are a convenient way for patients and the general public to access medical information. Under such circumstances, medical professionals must ensure that the information provided to chatbot users is accurate and safe.
引用
- Kim YI, Kim KH, Oh HJ, et al. Assessing the Suitability of Artificial Intelligence-Based Chatbots as Counseling Agents for Patients with Brain Tumor: A Comprehensive Survey Analysis. World Neurosurg. 2024. PMID: 38735564. doi:10.1016/j.wneu.2024.05.023
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。