肿瘤预测模型外部验证研究的报告质量需改进:一项系统综述
该研究通过检索 MEDLINE 2023 年 6 月至 7 月的文献,评估肿瘤预测模型研究中外部验证的比例及报告完整性。在 287 项 eligible 研究中,仅 89 项(31%)包含外部验证,且关键方法学和性能细节报告常不完整,尽管 TRIPOD 声明已发布。外部验证常缺乏明确目标或模型性能的情境化,提示需更透明地报告外部验证研究。
要点
- 仅 31% 的肿瘤预测模型研究包含外部验证,且多数为同一研究内新开发模型的验证。
- 仅 64% 和 46% 的研究完整报告结局和预测因子定义,仅 1 项报告样本量计算。
- 校准评估占 64%,临床效用评估占 37%,开放科学实践(注册、方案、代码共享)均不常见。
核心结论
- 2023 年 2 个月横断面快照显示,肿瘤预测模型外部验证研究远少于模型开发研究,且关键方法学和性能细节报告常不完整。
研究设计与样本
- 横断面系统综述,检索 MEDLINE(via Ovid)2023 年 6 月 1 日至 7 月 31 日发表的原始研究。
- 纳入评估肿瘤多变量预测模型并使用非建模数据(含时间或地理分割样本)的研究,共 287 项 eligible。
- 报告完整性采用 TRIPOD 声明评估,筛选和数据提取双人重复进行。
主要发现
- 287 项 eligible 肿瘤预测模型研究中,89 项(31%)包含外部验证,仅 3 项(1%)仅做外部验证而不开发模型。
- 78/89(88%)的验证针对同一研究内新开发的模型,32/89(36%)使用时间或地理分割样本方法。
- 结局和预测因子定义完整的研究分别占 64% 和 46%,仅 1 项研究报告样本量计算。
- 性能测量报告置信区间者 52/89(58%),评估校准者 57/89(64%),评估临床效用者 33/89(37%)。
- 仅 22/89(25%)明确描述验证数据集中预测值的计算方式;31/89 项研究数据存在聚集,但从未检验中心或亚组间异质性。
关键数值与推荐
- 外部验证研究比例:89/287(31%);仅外部验证无模型开发:3/287(1%)。
- 同研究内新开发模型验证:78/89(88%);时间或地理分割样本:32/89(36%)。
- 结局定义完整:64%;预测因子定义完整:46%;样本量计算报告:仅 1 项。
- 性能测量含置信区间:52/89(58%);校准评估:57/89(64%);临床效用评估:33/89(37%)。
- 预测计算明确描述:22/89(25%);开放科学实践:注册 2.2%、方案 1.1%、代码共享 4.5%。
临床怎么用
- 开展肿瘤预测模型外部验证时,应遵循 TRIPOD 声明完整报告方法学、性能测量及开放科学实践。
- 外部验证应明确目标并情境化模型性能,而非仅作为程序性步骤。
局限性
- 仅检索 2023 年 6 月至 7 月 MEDLINE 文献,为 2 个月横断面快照,可能不代表更广泛时段或其它数据库。
- 评估报告完整性基于 TRIPOD 声明,未评估研究质量或偏倚风险。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
BACKGROUND AND OBJECTIVE: External validation is essential for assessing the generalizability and transportability of a clinical prediction model. A previous review of studies published in 2010 identified substantial deficiencies in the reporting of external validations. In 2015, the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement was introduced, though its impact is unclear. Although external validation is widely recommended, its prevalence in recent oncology prediction model studies is uncertain, despite the large number of models developed in this field. We aimed to examine the proportion of oncology prediction model studies including an external validation and the reporting completeness of these studies, providing a cross-sectional overview of current practice. METHODS: We searched MEDLINE (via Ovid) for primary studies published between June 1, 2023 and July 31, 2023. Eligible studies evaluated multivariable prediction models in oncology using data not used for model development, including temporal or geographical split-sample approaches. Reporting completeness was assessed using the TRIPOD statement. Screening and data extraction were performed in duplicate. Findings were summarized using counts and percentages. RESULTS: Of 287 eligible oncology-based prediction model studies, 89 (31%) included an external validation component, with only 3 studies (1%) performing external validation without model development. Most validations (78/89, 88%) were conducted on newly developed models within the same study, and 32/89 (36%) used temporal or geographical split-sample approaches. Study design and participant characteristics were frequently reported, but outcome and predictor definitions were complete in only 64% and 46% of studies, respectively. Only one study reported a sample size calculation. Performance measures were reported with confidence intervals in 52/89 (58%). Calibration was assessed in 57/89 (64%), and clinical utility in 33/89 (37%). Only 22/89 (25%) explicitly described how predictions were calculated in the validation dataset. Despite validation data being clustered in 31/89 studies, heterogeneity across centers or subgroups was never examined. Open science practices were uncommon, including study registration (2.2%), protocol availability (1.1%), and code sharing (4.5%). CONCLUSION: In this 2-month cross-sectional snapshot in 2023, external validation studies were substantially less common than prediction model development studies and reporting of key methodological and performance details were frequently incomplete despite the availability of TRIPOD. External validation was often reported without clear articulation of its objectives or contextualization of model performance, suggesting it may sometimes be treated as a procedural step rather than a study designed to evaluate model transportability. Greater emphasis on transparently reporting external validation studies is required to enable reliable evaluation and comparison of prediction models.
引用
- Whittle R, Legha A, Tsegaye B, et al. Reporting of external validation studies needs improvement: a systematic review of reporting in oncology prediction model studies. J Clin Epidemiol. 2026. PMID: 42580368. doi:10.1016/j.jclinepi.2026.112460
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。