文献速递脊柱关节炎

ChatGPT与风湿科医生对银屑病关节炎患者信息的准确性与感知对比研究

ChatGPT vs rheumatologists: cross-sectional study on accuracy and patient perception of AI-generated information for psoriatic arthritis.
PubMed 文献 · Ann Rheum Dis · 2025-12-12

该横断面研究让76例银屑病关节炎患者提出32个问题(共296次选择),由ChatGPT-4和12名意大利PsA专家分别回答,14名临床医生对准确性和完整性进行评分,另67例患者对16对随机选取的答案进行偏好选择。结果显示ChatGPT准确性似与专家相当但完整性较低,患者对ChatGPT答案的偏好比例为49.2%,高于医生答案的34.4%,且ChatGPT答案更易读。

要点

  • ChatGPT-4对PsA问题的准确性与专家相当,但完整性评分较低。
  • 患者对ChatGPT答案偏好率为49.2%,高于医生答案的34.4%。
  • ChatGPT答案平均可读性优于医生答案,但妊娠/生育领域准确性较低。

核心结论

  • 在这项探索性研究中,ChatGPT-4似能生成准确且易读的PsA相关回答,并常被患者偏好。

研究设计与样本

  • 探索性、描述性横断面比较研究,76例PsA患者提出32个问题(296次选择),由ChatGPT-4和12名意大利PsA专家回答,14名临床医生评分,67例患者参与偏好评价。
  • 未进行正式样本量计算,P值为描述性,并结合效应量和95%CI解读。

主要发现

  • 患者最常寻求的信息领域为预后/合并症(54/76,71.1%)、治疗策略(48/76,63.2%)和治疗风险(38/76,50.0%)。
  • ChatGPT与专家的准确性似乎相当,但ChatGPT的完整性评分较低。
  • 妊娠/生育领域的准确性较低,其他领域无明确相关差异。
  • ChatGPT答案被选择491/998次(49.2%),医生答案343/998次(34.4%),无偏好164/998次(16.4%,P < .001),在预后和治疗方面相对更偏好ChatGPT回答。
  • ChatGPT回答平均在各项可读性指标上更易读。

关键数值与推荐

  • 准确性采用1-5 Likert量表评分,完整性采用1-3评分。
  • 偏好选择:ChatGPT 491/998次(49.2%),医生343/998次(34.4%),无偏好164/998次(16.4%,P < .001)。

临床怎么用

  • ChatGPT-4可为PsA患者提供准确且易读的信息,但完整性不足,临床中应补充关键内容。
  • 患者对AI回答偏好较高,医生需引导患者批判性使用AI信息,尤其在妊娠/生育领域。

局限性

  • 探索性研究,未进行正式样本量计算,P值为描述性。
  • 仅涉及意大利PsA专家和患者,结果外推需谨慎。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

OBJECTIVES: Patients with rheumatic diseases frequently turn to online sources for medical information. Large language models, such as ChatGPT, may offer an accessible alternative to conventional patient‑education resources; however, their reliability remains poorly explored. We conducted an exploratory, descriptive comparison to examine whether ChatGPT-4 might provide responses comparable to those of experts. METHODS: Seventy-six psoriatic arthritis (PsA) patients generated 32 questions (296 selections) grouped into 6 themes. Each question was answered by ChatGPT-4 and by 12 Italian PsA specialists (each drafted 2-3 answers). Fourteen clinicians, The 14 clinicians scored the accuracy and completeness of AI and human-generated answers, rated accuracy (1-5 Likert scale) and completeness (1-3). Interrater reliability was calculated, and mixed-effects ordinal logistic models were used to compare sources. In a separate arm, 67 PsA patients reviewed 16 randomly selected answer pairs and indicated their preference. Readability was assessed. No formal sample size calculation was performed; P values were descriptive and interpreted alongside effect sizes and 95% CIs. RESULTS: Patients most frequently sought information on prognosis/comorbidities (54/76, 71.1%), therapy strategy (48/76, 63.2%), and treatment risks (38/76, 50.0%). Accuracy appeared comparable between ChatGPT and experts, but ChatGPT scored lower in completeness. Accuracy was lower for pregnancy/fertility, with no clear relevant differences in other domains. ChatGPT answers were chosen 491/998 times (49.2%), clinician answers 343/998 times (34.4%), and no preference 164/998 times (16.4%, P < .001), with a relative preference for ChatGPT responses in prognosis and therapy. ChatGPT responses were, on average, more readable across indices. CONCLUSIONS: In this exploratory study, ChatGPT-4 appeared able to generate accurate and readable responses to PsA-related questions and was often preferred by patients.

来源:Ann Rheum Dis,查看原文。版权归原作者所有。

引用

  1. Forte G, Mauro D, Raimondi M, et al. ChatGPT vs rheumatologists: cross-sectional study on accuracy and patient perception of AI-generated information for psoriatic arthritis. Ann Rheum Dis. 2025. PMID: 41390304. doi:10.1016/j.ard.2025.11.012

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。