Gemini与GPTs基于自由文本18F-FDG PET/CT乳腺癌报告生成结构化报告的潜力
该研究回顾性纳入2005年7月至2023年10月连续行18F-FDG PET/CT随访的乳腺癌患者(平均年龄50岁±11,均为女性),用20份报告训练Gemini和GPTs的用户自定义文本提示,为131例患者生成结构化PET/CT报告。GPTs在原发灶大小(89.6% vs 53.8%,p<0.001)和转移灶(96.3% vs 89.6%,p<0.001)数据挖掘上优于Gemini,在进展决策(p<0.001)和语义相似性(F1 0.930 vs 0.907,p<0.001)上也优于Gemini。结论认为GPTs在基于自由文本PET/CT报告生成结构化报告方面优于Gemini,可能应用于临床实践。
要点
- GPTs在原发灶大小提取准确率89.6%,高于Gemini的53.8%(p<0.001)。
- GPTs在转移灶提取准确率96.3%,高于Gemini的89.6%(p<0.001)。
- GPTs在进展决策(p<0.001)和语义相似性(F1 0.930 vs 0.907,p<0.001)上优于Gemini。
核心结论
- GPTs在基于自由文本PET/CT报告生成结构化报告方面优于Gemini,可能应用于临床实践。
研究设计与样本
- 回顾性纳入2005年7月至2023年10月连续行18F-FDG PET/CT随访的乳腺癌患者(平均年龄50岁±11,均为女性)。
- 用10例患者的20份报告训练Gemini和GPTs的用户自定义文本提示,为131例患者生成结构化PET/CT报告。
- 将自然语言处理生成的结构化报告与核医学医师标注的结构化报告在数据提取准确性和进展决策能力方面进行比较。
- 统计方法包括卡方检验、McNemar检验和配对样本t检验。
诊断标准与关键数值
- 原发灶大小数据挖掘准确率:GPTs 89.6% vs Gemini 53.8%(p<0.001)。
- 转移灶数据挖掘准确率:GPTs 96.3% vs Gemini 89.6%(p<0.001)。
- 进展决策:GPTs优于Gemini(p<0.001)。
- 语义相似性F1评分:GPTs 0.930 vs Gemini 0.907(p<0.001)。
临床应用与阅片要点
- 原文结论认为GPTs在生成结构化PET/CT报告方面优于Gemini,可能应用于临床实践。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
RATIONALE AND OBJECTIVE: To compare the performance of large language model (LLM) based Gemini and Generative Pre-trained Transformers (GPTs) in data mining and generating structured reports based on free-text PET/CT reports for breast cancer after user-defined tasks.
MATERIALS AND METHODS: Breast cancer patients (mean age, 50 years ± 11 [SD]; all female) who underwent consecutive 18F-FDG PET/CT for follow-up between July 2005 and October 2023 were retrospectively included in the study. A total of twenty reports from 10 patients were used to train user-defined text prompts for Gemini and GPTs, by which structured PET/CT reports were generated. The natural language processing (NLP) generated structured reports and the structured reports annotated by nuclear medicine physicians were compared in terms of data extraction accuracy and capacity of progress decision-making. Statistical methods, including chi-square test, McNemar test and paired samples t-test, were employed in the study.
RESULTS: The structured PET/CT reports for 131 patients were generated by using the two NLP techniques, including Gemini and GPTs. In general, GPTs exhibited superiority over Gemini in data mining in terms of primary lesion size (89.6% vs. 53.8%, p < 0.001) and metastatic lesions (96.3% vs 89.6%, p < 0.001). Moreover, GPTs outperformed Gemini in making decision for progress (p < 0.001) and semantic similarity (F1 score 0.930 vs 0.907, p < 0.001) for reports.
CONCLUSION: GPTs outperformed Gemini in generating structured reports based on free-text PET/CT reports, which is potentially applied in clinical practice.
DATA AVAILABILITY: The data used and/or analyzed during the current study are available from the corresponding author on reasonable request.
引用
- Chen K, Xu W, Li X, et al. The Potential of Gemini and GPTs for Structured Report Generation based on Free-Text 18F-FDG PET/CT Breast Cancer Reports. Acad Radiol. 2024. PMID: 39245597. doi:10.1016/j.acra.2024.08.052
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。