ChatGPT与风湿科医生对银屑病关节炎患者信息的准确性与感知对比研究
该横断面研究让76例银屑病关节炎患者提出32个问题(共296次选择),由ChatGPT-4和12名意大利PsA专家分别回答,14名临床医生对准确性和完整性进行评分,另67例患者对16对随机选取的答案进行偏好选择。结果显示ChatGPT准确性似与专家相当但完整性较低,患者对ChatGPT答案的偏好比例为49.2%,高于医生答案的34.4%,且ChatGPT答案更易读。
要点
- ChatGPT-4对PsA问题的准确性与专家相当,但完整性评分较低。
- 患者对ChatGPT答案偏好率为49.2%,高于医生答案的34.4%。
- ChatGPT答案平均可读性优于医生答案,但妊娠/生育领域准确性较低。
核心结论
- 在这项探索性研究中,ChatGPT-4似能生成准确且易读的PsA相关回答,并常被患者偏好。
研究设计与样本
- 探索性、描述性横断面比较研究,76例PsA患者提出32个问题(296次选择),由ChatGPT-4和12名意大利PsA专家回答,14名临床医生评分,67例患者参与偏好评价。
- 未进行正式样本量计算,P值为描述性,并结合效应量和95%CI解读。
主要发现
- 患者最常寻求的信息领域为预后/合并症(54/76,71.1%)、治疗策略(48/76,63.2%)和治疗风险(38/76,50.0%)。
- ChatGPT与专家的准确性似乎相当,但ChatGPT的完整性评分较低。
- 妊娠/生育领域的准确性较低,其他领域无明确相关差异。
- ChatGPT答案被选择491/998次(49.2%),医生答案343/998次(34.4%),无偏好164/998次(16.4%,P < .001),在预后和治疗方面相对更偏好ChatGPT回答。
- ChatGPT回答平均在各项可读性指标上更易读。
关键数值与推荐
- 准确性采用1-5 Likert量表评分,完整性采用1-3评分。
- 偏好选择:ChatGPT 491/998次(49.2%),医生343/998次(34.4%),无偏好164/998次(16.4%,P < .001)。
临床怎么用
- ChatGPT-4可为PsA患者提供准确且易读的信息,但完整性不足,临床中应补充关键内容。
- 患者对AI回答偏好较高,医生需引导患者批判性使用AI信息,尤其在妊娠/生育领域。
局限性
- 探索性研究,未进行正式样本量计算,P值为描述性。
- 仅涉及意大利PsA专家和患者,结果外推需谨慎。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
OBJECTIVES: Patients with rheumatic diseases frequently turn to online sources for medical information. Large language models, such as ChatGPT, may offer an accessible alternative to conventional patient‑education resources; however, their reliability remains poorly explored. We conducted an exploratory, descriptive comparison to examine whether ChatGPT-4 might provide responses comparable to those of experts. METHODS: Seventy-six psoriatic arthritis (PsA) patients generated 32 questions (296 selections) grouped into 6 themes. Each question was answered by ChatGPT-4 and by 12 Italian PsA specialists (each drafted 2-3 answers). Fourteen clinicians, The 14 clinicians scored the accuracy and completeness of AI and human-generated answers, rated accuracy (1-5 Likert scale) and completeness (1-3). Interrater reliability was calculated, and mixed-effects ordinal logistic models were used to compare sources. In a separate arm, 67 PsA patients reviewed 16 randomly selected answer pairs and indicated their preference. Readability was assessed. No formal sample size calculation was performed; P values were descriptive and interpreted alongside effect sizes and 95% CIs. RESULTS: Patients most frequently sought information on prognosis/comorbidities (54/76, 71.1%), therapy strategy (48/76, 63.2%), and treatment risks (38/76, 50.0%). Accuracy appeared comparable between ChatGPT and experts, but ChatGPT scored lower in completeness. Accuracy was lower for pregnancy/fertility, with no clear relevant differences in other domains. ChatGPT answers were chosen 491/998 times (49.2%), clinician answers 343/998 times (34.4%), and no preference 164/998 times (16.4%, P < .001), with a relative preference for ChatGPT responses in prognosis and therapy. ChatGPT responses were, on average, more readable across indices. CONCLUSIONS: In this exploratory study, ChatGPT-4 appeared able to generate accurate and readable responses to PsA-related questions and was often preferred by patients.
引用
- Forte G, Mauro D, Raimondi M, et al. ChatGPT vs rheumatologists: cross-sectional study on accuracy and patient perception of AI-generated information for psoriatic arthritis. Ann Rheum Dis. 2025. PMID: 41390304. doi:10.1016/j.ard.2025.11.012
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。