文献速递技术与安全

ChatGPT作为影像组学研究质量评估的有效工具

ChatGPT as an effective tool for quality evaluation of radiomics research.
PubMed 文献 · Eur Radiol · 2024-10-15

本研究评估ChatGPT-4o使用影像组学质量评分(RQS)评估影像组学研究方法学质量的有效性,并与人类专家比较。纳入52篇开放获取同行评议论文,ChatGPT-4o与人类专家的RQS中位数均为14.5(40.3%),差异无统计学意义(p > 0.05)。ChatGPT-4o评估时间更短(每篇2.9-3.5分钟 vs 人类专家每篇13.9分钟),且信度良好至优秀。

要点

  • ChatGPT-4o与人类专家RQS评分中位数均为14.5(40.3%),差异无统计学意义。
  • ChatGPT-4o评估每篇耗时2.9-3.5分钟,人类专家每篇13.9分钟。
  • ChatGPT-4o组内信度ICC为0.905,与人类专家组间信度ICC为0.859和0.914。

核心结论

  • ChatGPT-4o能可靠且高效地评估影像组学研究质量,其评估与人类专家高度一致并减少评估时间。

研究设计与样本

  • 研究纳入2023至2024年发表于European Radiology、European Radiology Experimental和Insights into Imaging的开放获取同行评议影像组学论文,以及MedRxiv预印本,共52篇开放获取同行评议论文;每项研究由ChatGPT-4o独立评估两次,并由两名放射科医生共识评估,使用RQS评分。

诊断标准与关键数值

  • ChatGPT-4o两次评估平均值与人类专家的RQS中位数均为14.5(百分比得分40.3%),p > 0.05;ChatGPT-4o组内信度ICC为0.905(95% CI: 0.840-0.944),与人类专家两次评估的组间信度ICC分别为0.859(95% CI: 0.756-0.919)和0.914(95% CI: 0.855-0.949)。

临床应用与阅片要点

  • 作者认为ChatGPT-4o为评估影像组学研究质量提供了一种快速可靠的自动化替代方案,未来有潜力大规模评估影像组学研究。

局限性

  • 原文未报告局限性。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

英文原文(摘要)· 点击展开

OBJECTIVES: This study aimed to evaluate the effectiveness of ChatGPT-4o in assessing the methodological quality of radiomics research using the radiomics quality score (RQS) compared to human experts.

METHODS: Published in European Radiology, European Radiology Experimental, and Insights into Imaging between 2023 and 2024, open-access and peer-reviewed radiomics research articles with creative commons attribution license (CC-BY) were included in this study. Pre-prints from MedRxiv were also included to evaluate potential peer-review bias. Using the RQS, each study was independently assessed twice by ChatGPT-4o and by two radiologists with consensus.

RESULTS: In total, 52 open-access and peer-reviewed articles were included in this study. Both ChatGPT-4o evaluation (average of two readings) and human experts had a median RQS of 14.5 (40.3% percentage score) (p > 0.05). Pairwise comparisons revealed no statistically significant difference between the readings of ChatGPT and human experts (corrected p > 0.05). The intraclass correlation coefficient for intra-rater reliability of ChatGPT-4o was 0.905 (95% CI: 0.840-0.944), and those for inter-rater reliability with human experts for each evaluation of ChatGPT-4o were 0.859 (95% CI: 0.756-0.919) and 0.914 (95% CI: 0.855-0.949), corresponding to good to excellent reliability for all. The evaluation by ChatGPT-4o took less time (2.9-3.5 min per article) compared to human experts (13.9 min per article by one reader). Item-wise reliability analysis showed ChatGPT-4o maintained consistently high reliability across almost all RQS items.

CONCLUSION: ChatGPT-4o provides reliable and efficient assessments of radiomics research quality. Its evaluations closely align with those of human experts and reduce evaluation time.

KEY POINTS: Question Is ChatGPT effective and reliable in evaluating radiomics research quality based on RQS? Findings ChatGPT-4o showed high reliability and efficiency, with evaluations closely matching human experts. It can considerably reduce the time required for radiomics research quality assessment. Clinical relevance ChatGPT-4o offers a quick and reliable automated alternative for evaluating the quality of radiomics research, with the potential to assess radiomics research at a large scale in the future.

来源:Eur Radiol,查看原文。版权归原作者所有。

引用

  1. Mese I, Kocak B. ChatGPT as an effective tool for quality evaluation of radiomics research. Eur Radiol. 2024. PMID: 39406959. doi:10.1007/s00330-024-11122-7

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。