文献速递医学人工智能与信息学临床试验与 GCP医学教育与科研方法

医学大语言模型干预临床试验报告完整性系统评价

Reporting completeness of clinical trials evaluating large language model interventions in medicine: systematic review.
PubMed 文献 · BMJ · 2026-10-13

采用TRIPOD-LLM、CONSORT 2025、CONSORT-AI、SPIRIT 2025、SPIRIT-AI、CHART及WHO试验注册数据集评估报告完整性,发现总体依从性不高,LLM特异方法学细节、可复现材料与危害报告普遍不足。

要点

  • LLM干预临床试验对既定报告指南的依从性总体不高,方法学细节报告不足。
  • 数据与干预来源、输入预处理、模型版本与访问方式、推理参数、输出后处理等报告不一致。
  • 危害的计划、监测与报告常不完整,限制了对LLM干预安全性的评价。

核心结论

  • LLM干预临床试验对既定报告指南依从性不高,LLM特异方法学细节、可复现材料与危害报告持续不足,难以支持可复现性和安全性风险的可靠评价。

研究设计与样本

  • 纳入前瞻性将人类受试者分配至LLM干预并评估健康相关生物医学或行为结局的试验报告、方案和注册。
  • 共纳入461条记录,包括68篇试验报告、15份方案和378项注册;两名评价者独立筛选与提取,三名校准评价者评估条目级报告完整性。

主要发现

  • 试验报告多为小样本、单中心随机研究,发表于2024至2025年,覆盖多个临床专科,主要面向患者教育与管理建议。
  • 平均报告完整性:试验报告TRIPOD-LLM 40%、CONSORT 2025 44%、CONSORT-AI 59%、CHART 52%;方案SPIRIT 2025 51%、SPIRIT-AI 57%;注册WHO试验注册数据集61%。
  • 试验报告偏重叙述性框架与评论,而非可复制、偏倚风险评估及模型性能随时间治理所需的方法学细节。
  • 数据与干预来源、输入预处理、模型版本与访问方式、推理参数、输出后处理及方案、数据、代码获取等报告不一致。
  • 危害的计划、监测与报告常不完整,限制了对LLM干预安全性的批判性评价。

关键数值与推荐

  • 报告完整性采用与设计及目的匹配的指南评估:TRIPOD-LLM、CONSORT 2025、CONSORT-AI、SPIRIT 2025、SPIRIT-AI、CHART及WHO试验注册数据集(版本1.3.1)。
  • 纳入标准要求前瞻性分配人类受试者并评估健康相关生物医学或行为结局。
  • LLM干预须用于临床试验场景中的医疗功能,包括患者教育、临床文书、沟通支持、分诊、就医导航、管理建议或临床决策支持。

临床怎么用

  • 阅读或开展LLM干预临床试验时,应参照TRIPOD-LLM、CONSORT-AI、SPIRIT-AI、CHART等指南核查报告完整性。
  • 关注模型版本、推理参数、输入预处理与输出后处理等LLM特异细节,以及方案、数据和代码的可获取性。
  • 在临床转化前应重视危害计划、监测与报告,以支持对安全风险的可靠评价。

局限性

  • 原文未报告纳入研究的偏倚风险评价结果及报告完整性的置信区间。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

OBJECTIVE: To characterise the reporting completeness of clinical trials involving large language model (LLM) interventions using established reporting guidelines. DESIGN: Systematic review and meta-research analysis. DATA SOURCES: Bibliographic databases (Embase, Medline, and CENTRAL) were searched for trial reports and protocols, and trial registries (ClinicalTrials.gov and World Health Organization International Clinical Trials Registry Platform) were searched for trial registrations published from 1 January 2017 to 6 December 2025. ELIGIBILITY CRITERIA: Trial reports, trial protocols, and trial registrations describing clinical trials of LLM interventions in medicine. LLM interventions used for a healthcare function in a clinical trial setting included patient education, clinical documentation, communication support, triage, care navigation, management recommendations, or clinical decision support. Eligible studies prospectively assigned human participants to the intervention and evaluated a health related biomedical or behavioural outcome. MAIN OUTCOME MEASURES: Overall study level and item level reporting completeness assessed using reporting guidelines matched to study design and purpose, including TRIPOD-LLM (Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis-LLM), CONSORT (Consolidated Standards of Reporting Trials) 2025 and CONSORT-AI, SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) 2025 and SPIRIT-AI, CHART (Chatbot Assessment Reporting Tool), and the WHO Trial Registration Dataset (version 1.3.1). Two reviewers independently screened and extracted trial and intervention characteristics, and three calibrated raters assessed item level reporting completeness. RESULTS: The systematic search yielded 461 records, including 68 trial reports, 15 protocols, and 378 registrations. Trial reports were predominantly small sample, single site randomised studies published between 2024 and 2025, spanning diverse clinical specialties and primarily targeting patient facing education and management recommendations. Mean reporting completeness was modest across eligible trial reports (TRIPOD-LLM 40%, CONSORT 2025 44%, CONSORT-AI 59%, CHART 52%), trial protocols (SPIRIT 2025 51%, SPIRIT-AI 57%), and trial registrations (WHO Trial Registration Dataset 61%). Clinical trial reports of LLM interventions prioritise narrative framing and commentary over the methodological detail needed for replication, risk-of-bias assessment, and governance of model performance over time. Critical methodological elements for reproducibility, including data and intervention provenance, input preprocessing, model versioning and access modality, inference parameters, output postprocessing, as well as protocol, data, and code access, were inconsistently reported. Harms planning, monitoring, and reporting were often incompletely reported, limiting the critical appraisal of the safety of LLM interventions in healthcare. CONCLUSION: Clinical trials of LLM interventions show modest adherence to established reporting guidelines, with persistent underreporting of LLM specific methodological details, reproducibility artefacts (protocols, data, code), and harms reporting required to support reproducibility and confident appraisal of safety risks before their clinical translation.

来源:BMJ,查看原文。版权归原作者所有。

引用

  1. Chen D, Blayne A, Ning T, et al. Reporting completeness of clinical trials evaluating large language model interventions in medicine: systematic review. BMJ. 2026. PMID: 42869550. doi:10.1136/bmj-2026-100855

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。