评估ChatGPT对全髋关节置换术常见患者问题的回答
研究向ChatGPT连续提出10个全髋关节置换术常见问题,采用循证方法评估回答准确性。仅1条回答被评为「不满意」,2条无需澄清,多数需少量(10条中4条)或中等(10条中3条)澄清。结论认为该聊天机器人能有效提供循证回答,未来或可作为骨科咨询前患者教育的临床工具。
要点
- 10个全髋关节置换术常见问题中,仅1条回答被评为「不满意,需大量澄清」。
- 2条回答无需任何纠正,4条需少量澄清,3条需中等澄清。
- 作者认为该聊天机器人可有效提供循证回答,未来或可作为患者教育工具。
核心结论
- 该聊天机器人有效提供了全髋关节置换术前患者常问问题的循证回答,并以多数患者能理解的方式呈现信息。
- 作者认为该资源未来或可作为骨科咨询前患者教育与理解的宝贵临床工具。
研究设计与样本
- 研究向ChatGPT在全髋关节置换术相关对话线程中提出10个常见问题,不设追问或重复。
- 采用循证方法分析每条回答的准确性,并按四级评级:优秀无需澄清、满意需少量澄清、满意需中等澄清、不满意需大量澄清。
主要发现
- 聊天机器人的回答中仅1条被评为「不满意」。
- 10条回答中2条无需任何纠正,4条需少量澄清,3条需中等澄清。
- 尽管若干回答需要细微澄清,聊天机器人的回答总体无偏倚且基于证据,即使对争议性话题也是如此。
临床怎么用
- 作者认为该聊天机器人未来或可作为骨科咨询前患者教育与理解的宝贵临床工具。
局限性
- 原文未报告局限性
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
BACKGROUND: The contemporary patient has access to numerous resources on common orthopaedic procedures before ever presenting for a clinical evaluation. Recently, artificial intelligence (AI)-driven chatbots have become mainstream, allowing patients to engage with interfaces that supply convincing, human-like responses to prompts. ChatGPT (OpenAI), a recently developed AI-based chat technology, is one such application that has garnered rapid growth in popularity. Given the likelihood that patients may soon call on this technology for preoperative education, we sought to determine whether ChatGPT could appropriately answer frequently asked questions regarding total hip arthroplasty (THA). METHODS: Ten frequently asked questions regarding total hip arthroplasty were posed to the chatbot during a conversation thread, with no follow-up questions or repetition. Each response was analyzed for accuracy with use of an evidence-based approach. Responses were rated as "excellent response not requiring clarification," "satisfactory requiring minimal clarification," "satisfactory requiring moderate clarification," or "unsatisfactory requiring substantial clarification." RESULTS: Of the responses given by the chatbot, only 1 received an "unsatisfactory" rating; 2 did not require any correction, and the majority required either minimal (4 of 10) or moderate (3 of 10) clarification. Although several responses required nuanced clarification, the chatbot's responses were generally unbiased and evidence-based, even for controversial topics. CONCLUSIONS: The chatbot effectively provided evidence-based responses to questions commonly asked by patients prior to THA. The chatbot presented information in a way that most patients would be able to understand. This resource may serve as a valuable clinical tool for patient education and understanding prior to orthopaedic consultation in the future.
引用
- Mika AP, Martin JR, Engstrom SM, et al. Assessing ChatGPT Responses to Common Patient Questions Regarding Total Hip Arthroplasty. J Bone Joint Surg Am. 2023. PMID: 37459402. doi:10.2106/JBJS.23.00209
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。