聊天机器人在头颈部黏膜恶性肿瘤治疗推荐中的应用
本研究将40个头颈部黏膜恶性肿瘤的临床情景提示输入ChatGPT-3.5,评估其分期与治疗推荐的准确性。结果显示,基于2021年NCCN指南的一线治疗推荐准确率为95%,但55%的回答分期不准确,50%的病例不恰当地遗漏了颈清扫,40%的治疗推荐被认为不必要。作者认为ChatGPT在头颈癌患者教育方面有潜力,但需在专家指导下使用。
要点
- ChatGPT基于2021年NCCN指南的一线治疗推荐准确率达95%。
- 55%的回答分期不准确,50%的病例不恰当地遗漏颈清扫。
- 40%的治疗推荐被认为不必要,需专家指导使用。
核心结论
- ChatGPT在头颈癌患者教育方面有潜力,与NCCN指南对黏膜恶性肿瘤的推荐一致,但观察到的肿瘤、淋巴结、转移分期不准确、手术选项不完整及不恰当治疗推荐,凸显了持续改进和审查的重要性。
- 耳鼻咽喉科医生可利用此信息提醒患者、家属和受训者,在没有专家指导的情况下使用ChatGPT进行头颈癌教育存在风险。
研究设计与样本
- 研究将40个不同的临床情景提示输入ChatGPT,询问头颈部黏膜恶性肿瘤的分期和治疗推荐。
- 提示基于头颈癌疾病描述(癌症位置、肿瘤大小、淋巴结受累和症状)创建。
- 分期和治疗推荐根据2021年美国国家综合癌症网络(NCCN)指南,由来自两家独立三级医疗机构的3名经过专科培训的头颈癌外科医生进行评分。
- 头颈癌外科医生评估了分期和治疗推荐的准确性,如手术的完整性和治疗方式的适当性。
主要发现
- ChatGPT的回答基于2021年NCCN指南推荐正确一线治疗的准确率为95%。
- 55%的回答包含不准确的分期。
- 在50%的病例中,颈清扫被错误地从治疗推荐中省略。
- 此外,40%的ChatGPT治疗推荐被认为不必要。
临床怎么用
- 作者认为ChatGPT在头颈癌患者教育方面有潜力,但强调需要持续改进和审查。
- 作者指出耳鼻咽喉科医生可利用此信息提醒患者、家属和受训者,在没有专家指导的情况下使用ChatGPT进行头颈癌教育存在风险。
局限性
- 原文未报告局限性
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
OBJECTIVE: As cancer patients increasingly use chatbots, it is crucial to recognize ChatGPT's potential in enhancing health literacy while ensuring validation to prevent misinformation. This study aims to assess ChatGPT-3.5's capability to provide appropriate staging and treatment recommendations for head and neck mucosal malignancies for vulnerable populations. STUDY DESIGN AND SETTING: Forty distinct clinical vignettes were introduced into ChatGPT to inquire about staging and treatment recommendations for head and neck mucosal malignancies. METHODS: Prompts were created based on head and neck cancer (HNC) disease descriptions (cancer location, tumor size, lymph node involvement, and symptoms). Staging and treatment recommendations according to the 2021 National Comprehensive Cancer Network (NCCN) guidelines were scored by three fellowship-trained HNC surgeons from two separate tertiary care institutions. HNC surgeons assessed the accuracy of staging and treatment recommendations, such as the completeness of surgery and the appropriateness of treatment modality. RESULTS: Whereas ChatGPT's responses were 95% accurate at recommending the correct first-line treatment based on the 2021 NCCN guidelines, 55% of the responses contained inaccurate staging. Neck dissection was incorrectly omitted from treatment recommendations in 50% of the cases. Moreover, 40% of ChatGPT's treatment recommendations were deemed unnecessary. CONCLUSION: This study emphasizes ChatGPT's potential in HNC patient education, aligning with NCCN guidelines for mucosal malignancies, but highlights the importance of ongoing refinement and scrutiny due to observed inaccuracies in tumor, nodal, metastasis staging, incomplete surgery options, and inappropriate treatment recommendations. Otolaryngologists can use this information to caution patients, families, and trainees regarding the use of ChatGPT for HNC education without expert guidance.
引用
- Washington CJ, Abouyared M, Karanth S, et al. The Use of Chatbots in Head and Neck Mucosal Malignancy Treatment Recommendations. Otolaryngol Head Neck Surg. 2024. PMID: 38769872. doi:10.1002/ohn.818
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。