测试ChatGPT回答公众关于心脏骤停与心肺复苏问题的能力
该研究由Sudden Cardiac Arrest UK成员共同制作40个涵盖心脏骤停与心肺复苏各方面的问题,由14名专业人员和16名非专业人员对ChatGPT的回答进行评分(1分差至5分优)。总体评分为4.3 ± 0.7,清晰度4.4 ± 0.6、相关性4.3 ± 0.6、准确性4.0 ± 0.6、全面性4.2 ± 0.7均较高;但专业人员对总体价值(4.0 ± 0.5 vs 4.6 ± 0.7;p = 0.02)和全面性(3.9 ± 0.6 vs 4.5 ± 0.7;p = 0.02)的评分低于非专业人员。心肺复苏相关回答在所有参数上均得分较低,可读性为「困难」(Flesch阅读易读性评分中位数34 [IQR 26-42])。
要点
- ChatGPT对心脏骤停与心肺复苏公众问题的总体评分为4.3 ± 0.7(14名专业人员和16名非专业人员,1-5分制)。
- 清晰度4.4 ± 0.6、相关性4.3 ± 0.6、准确性4.0 ± 0.6、全面性4.2 ± 0.7均获高分。
- 专业人员对总体价值(4.0 ± 0.5 vs 4.6 ± 0.7;p = 0.02)和全面性(3.9 ± 0.6 vs 4.5 ± 0.7;p = 0.02)的评分低于非专业人员。
- 心肺复苏相关回答在所有参数上得分一致较低。
- 回答可读性为「困难」(Flesch阅读易读性评分中位数34 [IQR 26-42])。
适用人群与场景
- 心脏骤停目击者、幸存者及其亲属以及非专业施救者提出的关于心脏骤停与心肺复苏的问题。
研究设计与关键结果
- 研究由Sudden Cardiac Arrest UK成员共同制作40个涵盖心脏骤停与心肺复苏各方面的问题,由14名专业人员和16名非专业人员对ChatGPT的回答进行评分。
- 总体评分为4.3 ± 0.7;清晰度4.4 ± 0.6、相关性4.3 ± 0.6、准确性4.0 ± 0.6、全面性4.2 ± 0.7。
- 专业人员对总体价值评分低于非专业人员(4.0 ± 0.5 vs 4.6 ± 0.7;p = 0.02)。
- 专业人员对全面性评分低于非专业人员(3.9 ± 0.6 vs 4.5 ± 0.7;p = 0.02)。
- 心肺复苏相关回答在所有参数上得分一致较低。
- 可读性为「困难」(Flesch阅读易读性评分中位数34 [IQR 26-42])。
证据类型、等级与局限
- 证据类型:横断面评估研究;局限:原文未报告局限性。
对床旁实践的改变
- ChatGPT对心脏骤停与心肺复苏的公众问题提供了大体准确、相关且全面的回答,但心肺复苏相关回答得分最低,且回答可读性为「困难」。
- 作者认为,大型语言模型将在未来发挥重要作用,对其生成的医疗相关内容应进行监测。
提炼说明
- 本条依据论文摘要整理;摘要未涉及「诊断标准与关键阈值」「处置流程与药物剂量」「红旗征与不能漏的情况」「指南或共识出处」,故未列出这些栏目;全文请点「阅读原文」。
英文原文(摘要)· 点击展开
INTRODUCTION: Cardiac arrest leaves witnesses, survivors, and their relatives with a multitude of questions. When a young or a public figure is affected, interest around cardiac arrest and cardiopulmonary resuscitation (CPR) increases. ChatGPT allows everyone to obtain human-like responses on any topic. Due to the risks of accessing incorrect information, we assessed ChatGPT accuracy in answering laypeople questions about cardiac arrest and CPR.
METHODS: We co-produced a list of 40 questions with members of Sudden Cardiac Arrest UK covering all aspects of cardiac arrest and CPR. Answers provided by ChatGPT to each question were evaluated by professionals for their accuracy, by professionals and laypeople for their relevance, clarity, comprehensiveness, and overall value on a scale from 1 (poor) to 5 (excellent), and for readability.
RESULTS: ChatGPT answers received an overall positive evaluation (4.3 ± 0.7) by 14 professionals and 16 laypeople. Also, clarity (4.4 ± 0.6), relevance (4.3 ± 0.6), accuracy (4.0 ± 0.6), and comprehensiveness (4.2 ± 0.7) of answers was rated high. Professionals, however, rated overall value (4.0 ± 0.5 vs 4.6 ± 0.7; p = 0.02) and comprehensiveness (3.9 ± 0.6 vs 4.5 ± 0.7; p = 0.02) lower compared to laypeople. CPR-related answers consistently received a lower score across all parameters by professionals and laypeople. Readability was 'difficult' (median Flesch reading ease score of 34 [IQR 26-42]).
CONCLUSIONS: ChatGPT provided largely accurate, relevant, and comprehensive answers to questions about cardiac arrest commonly asked by survivors, their relatives, and lay rescuers, except CPR-related answers that received the lowest scores. Large language model will play a significant role in the future and healthcare-related content generated should be monitored.
参考来源
- Scquizzato T, Semeraro F, Swindell P, et al. Testing ChatGPT ability to answer laypeople questions about cardiac arrest and cardiopulmonary resuscitation. Resuscitation. 2023. PMID: 38081504. doi:10.1016/j.resuscitation.2023.110077
供执业医师抢救与诊疗参考,不替代临床判断;剂量请结合体重、肝肾功能和本院药品规格复核,以指南原文为准。