文献速递胸部技术与安全DR

隐藏在明处的宝库:GPT-4在胸部X线片评估中的效用

The Treasure Trove Hidden in Plain Sight: The Utility of GPT-4 in Chest Radiograph Evaluation.
PubMed 文献 · Radiology · 2024-01-01

本回顾性研究评估了GPT-4高级数据分析(ADA)自主分析多来源临床数据集的能力,数据集包含2009年1月至2019年12月重症监护患者的床旁胸部X线片报告、人口学数据及炎症实验室标志物。GPT-4 ADA能生成可视化、描述性统计、逻辑回归关联及梯度提升机预测模型(AUC 0.75),其机器学习策略与人类专家开发的参考模型性能相当(AUC 0.80 vs 0.80,P=0.51),但存在一些统计错误和不准确之处。

要点

  • GPT-4 ADA可自主完成从描述性统计到机器学习预测建模的放射学数据分析。
  • 其构建的机器学习模型性能与人类专家模型相当(AUC 0.80 vs 0.80,P=0.51)。
  • 分析过程中出现了一些统计错误和不准确之处,需人工验证。

核心结论

  • 大语言模型可能促进放射学中的数据分析,从基础统计到基于机器学习的先进预测建模。

研究设计与样本

  • 回顾性研究,纳入2009年1月至2019年12月德国亚琛工业大学医院重症监护患者的43 788份床旁胸部X线片报告及实验室值,共43 788例患者(平均年龄66岁±15[标准差];26 804例男性);由三名具有6-10年机器学习经验的科学家验证GPT-4 ADA(2023年12月至2024年1月访问)自主分析输出。

诊断标准与关键数值

  • GPT-4 ADA构建的梯度提升机预测模型AUC为0.75;与人类专家模型相比,AUC 0.80(95%CI:0.80,0.81)vs 0.80(95%CI:0.80,0.81),P=0.51;准确率79%(6910/8758例)vs 78%(6875/8758例),P=0.27。

临床应用与阅片要点

  • 作者结论认为大语言模型可能促进放射学数据分析,从基础统计到先进机器学习预测建模;验证过程包括核实方法学、评估编码质量、重新执行代码及与人类模型头对头比较。

局限性

  • 分析过程中遇到了一些统计错误和不准确之处。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

英文原文(摘要)· 点击展开

Background Limited statistical knowledge can slow critical engagement with and adoption of artificial intelligence (AI) tools for radiologists. Large language models (LLMs) such as OpenAI's GPT-4, and notably its Advanced Data Analysis (ADA) extension, may improve the adoption of AI in radiology. Purpose To validate GPT-4 ADA outputs when autonomously conducting analyses of varying complexity on a multisource clinical dataset. Materials and Methods In this retrospective study, unique itemized radiologic reports of bedside chest radiographs, associated demographic data, and laboratory markers of inflammation from patients in intensive care from January 2009 to December 2019 were evaluated. GPT-4 ADA, accessed between December 2023 and January 2024, was tasked with autonomously analyzing this dataset by plotting radiography usage rates, providing descriptive statistics measures, quantifying factors of pulmonary opacities, and setting up machine learning (ML) models to predict their presence. Three scientists with 6-10 years of ML experience validated the outputs by verifying the methodology, assessing coding quality, re-executing the provided code, and comparing ML models head-to-head with their human-developed counterparts (based on the area under the receiver operating characteristic curve [AUC], accuracy, sensitivity, and specificity). Statistical significance was evaluated using bootstrapping. Results A total of 43 788 radiograph reports, with their laboratory values, from University Hospital RWTH Aachen were evaluated from 43 788 patients (mean age, 66 years ± 15 [SD]; 26 804 male). While GPT-4 ADA provided largely appropriate visualizations, descriptive statistical measures, quantitative statistical associations based on logistic regression, and gradient boosting machines for the predictive task (AUC, 0.75), some statistical errors and inaccuracies were encountered. ML strategies were valid and based on consistent coding routines, resulting in valid outputs on par with human specialist-developed reference models (AUC, 0.80 [95% CI: 0.80, 0.81] vs 0.80 [95% CI: 0.80, 0.81]; P = .51) (accuracy, 79% [6910 of 8758 patients] vs 78% [6875 of 8758 patients], respectively; P = .27). Conclusion LLMs may facilitate data analysis in radiology, from basic statistics to advanced ML-based predictive modeling. © RSNA, 2024 Supplemental material is available for this article.

来源:Radiology,查看原文。版权归原作者所有。

引用

  1. Tayebi Arasteh S, Siepmann R, Huppertz M, et al. The Treasure Trove Hidden in Plain Sight: The Utility of GPT-4 in Chest Radiograph Evaluation. Radiology. 2024. PMID: 39530893. doi:10.1148/radiol.233441

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。