文献速递系统性红斑狼疮

ChatGPT4回答系统性红斑狼疮患者问题的能力:与专科医生的盲法比较研究

ChatGPT4's proficiency in addressing patients' questions on systemic lupus erythematosus: a blinded comparative study with specialists.
PubMed 文献 · Rheumatology (Oxford) · 2024-01-01

这项横断面研究纳入95个系统性红斑狼疮患者常见问题(55个中文、40个英文),由风湿科医生小组和狼疮患者分别对ChatGPT4与5名风湿科医生的回答在六个维度上评分。风湿科医生评分显示ChatGPT4在满意度和共情方面优于医生,患者评分显示两者在全部六个维度相当;亚组分析显示ChatGPT4在中文问题上的逻辑一致性、完整性、可理解性、满意度和共情更优,但英文问题的可理解性较差。

要点

  • 风湿科医生评分:ChatGPT4满意度平均差0.537(95%CI 0.252-0.823,P<0.01)、共情平均差0.460(95%CI 0.227-0.693,P<0.01),均优于医生回答。
  • 狼疮患者评分:ChatGPT4与风湿科医生回答在全部六个维度上相当。
  • 亚组分析:ChatGPT4在中文问题上的逻辑一致性、完整性、可理解性、满意度和共情更优,英文问题可理解性较差。

核心结论

  • 与专科医生相比,ChatGPT4在回答系统性红斑狼疮患者常见问题时表现相当,在某些维度可能更好。
  • 本研究显示ChatGPT4有潜力用于改善系统性红斑狼疮患者的咨询。

研究设计与样本

  • 横断面研究,纳入95个常见问题,包括55个中文问题和40个英文问题。
  • 由风湿科医生小组和一组系统性红斑狼疮患者分别对ChatGPT4和5名风湿科医生的回答进行评分。
  • 评分覆盖科学有效性、逻辑一致性、可理解性、完整性、满意度和共情六个维度,采用0-10分制,0分表示回答完全错误,10分表示回答准确且全面。

主要发现

  • 风湿科医生评分显示,ChatGPT4生成回答在满意度上优于风湿科医生回答,平均差0.537(95%CI 0.252-0.823;P<0.01)。
  • 风湿科医生评分显示,ChatGPT4生成回答在共情上优于风湿科医生回答,平均差0.460(95%CI 0.227-0.693;P<0.01)。
  • 从系统性红斑狼疮患者角度看,ChatGPT4生成回答与风湿科医生回答在全部六个维度上相当。
  • 亚组分析显示,无论语言,ChatGPT4回答的逻辑一致性和完整性更优;中文问题中可理解性、满意度和共情更优。
  • 亚组分析显示,ChatGPT4回答在英文常见问题上的可理解性较差。

关键数值与推荐

  • 评分采用0-10分制,0分表示回答完全错误,10分表示回答准确且全面。
  • 满意度平均差0.537(95%CI 0.252-0.823;P<0.01),ChatGPT4优于风湿科医生。
  • 共情平均差0.460(95%CI 0.227-0.693;P<0.01),ChatGPT4优于风湿科医生。

临床怎么用

  • 原文结论指出ChatGPT4在回答系统性红斑狼疮患者常见问题方面与专科医生相当、在某些维度可能更好,并显示其改善患者咨询的潜力。

局限性

  • 原文未报告局限性

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

OBJECTIVES: The efficacy of artificial intelligence (AI)-driven chatbots like ChatGPT4 in specialized medical consultations, particularly in rheumatology, remains underexplored. This study compares the proficiency of ChatGPT4' responses with practicing rheumatologists to inquiries from patients with SLE. METHODS: In this cross-sectional study, we curated 95 frequently asked questions (FAQs), including 55 in Chinese and 40 in English. Responses for FAQs from ChatGPT4 and five rheumatologists were scored separately by a panel of rheumatologists and a group of patients with SLE across six domains (scientific validity, logical consistency, comprehensibility, completeness, satisfaction level and empathy) on a 0-10 scale (a score of 0 indicates entirely incorrect responses, while 10 indicates accurate and comprehensive answers). RESULTS: Rheumatologists' scoring revealed that ChatGPT4-generated responses outperformed those from rheumatologists in satisfaction level and empathy, with mean differences of 0.537 (95% CI, 0.252-0.823; P < 0.01) and 0.460 (95% CI, 0.227-0.693; P < 0.01), respectively. From the SLE patients' perspective, ChatGPT4-generated responses were comparable to the rheumatologist-provided answers in all six domains. Subgroup analysis revealed ChatGPT4 responses were more logically consistent and complete regardless of language and exhibited greater comprehensibility, satisfaction and empathy in Chinese. However, ChatGPT4 responses were inferior in comprehensibility for English FAQs. CONCLUSION: ChatGPT4 demonstrated comparable, possibly better in certain domains, to address FAQs from patients with SLE, when compared with the answers provided by specialists. This study showed the potential of applying ChatGPT4 to improve consultation in SLE patients.

来源:Rheumatology (Oxford),查看原文。版权归原作者所有。

引用

  1. Xu D, Zhao J, Liu R, et al. ChatGPT4's proficiency in addressing patients' questions on systemic lupus erythematosus: a blinded comparative study with specialists. Rheumatology (Oxford). 2024. PMID: 38648756. doi:10.1093/rheumatology/keae238

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。