基于生成式预训练变换器的PEDro量表自动化评估工具的开发与性能验证
结果显示总分的评分者内信度ICC=1.00,各条目PABAK=0.94-1.00;同时效度总分ICC=0.83、0.83,各条目PABAK=0.68-0.97。该工具可能有助于减少人工评估工作量。
要点
- GPT-PEDro ET在PEDro量表总分上评分者内信度ICC=1.00,各条目PABAK=0.94-1.00。
- 同时效度总分ICC=0.83、0.83,各条目PABAK=0.68-0.97,呈中至高度。
核心结论
- GPT-PEDro ET可能是一种有用的自动化工具,用于评估随机对照试验的方法学质量。
- 该工具在减少人工评估工作量和支持临床与研究应用方面具有潜力。
研究设计与样本
- 心理测量学验证研究,采用重复测量设计,在研究实验室进行。
主要发现
- 评分者内信度:总分ICC=1.00,各条目PABAK=0.94-1.00,达到几乎完美水平。
- 同时效度:总分ICC=0.83、0.83,各条目PABAK=0.68-0.97,为中至高度。
关键数值与推荐
- 评分者内信度采用组内相关系数(ICC)评估总分,采用患病率调整和偏倚调整的κ(PABAK)评估各条目。
- 同时效度采用ICC评估总分、PABAK评估各条目,并进行Bland-Altman分析(95%一致性界限)及异方差性检验。
临床怎么用
- 该工具可用于自动化评估随机对照试验的PEDro方法学质量,减少人工评估负担。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
OBJECTIVES: To develop an automated generative pretrained transformer (GPT)- physiotherapy evidence database (PEDro) evaluation tool (GPT-PEDro ET) and validate its performance (including intrarater reliability and concurrent validity) for evaluating the methodological quality of RCTs using the PEDro scale. DESIGN: A psychometric validation study with a repeated measurements design. SETTING: Research laboratory. PARTICIPANTS: One hundred and twenty-five RCTs on neurofacilitation interventions in stroke rehabilitation were retrieved from the PEDro database. INTERVENTIONS: Not applicable. MAIN OUTCOMES AND MEASURES: The primary outcome was RCT quality scores evaluated using the GPT-PEDro ET tool at both total score and individual item levels across 2 rounds of evaluation. Intrarater reliability was determined using the intraclass correlation coefficient (ICC) for total score and prevalence-adjusted and bias-adjusted κ (PABAK) for individual items. Concurrent validity was assessed with ICC (total score) and PABAK (individual item), Bland-Altman analysis with 95% limits of agreement, and heteroscedasticity testing. RESULTS: The GPT-PEDro ET tool achieved almost perfect intrarater reliability (total score ICC=1.00; individual item PABAK=0.94-1.00). The concurrent validity was moderate to high at both the total score level (ICCs=0.83, 0.83) and the individual item level for all the items (PABAKs=0.68-0.97). CONCLUSIONS: The results suggest that the GPT-PEDro ET tool may be a useful automated tool for evaluating the methodological quality of RCTs. It shows potential for reducing manual evaluation workload and supporting clinical and research applications.
引用
- Ju YJ, Wang YC, Chou F, et al. Development and Performance Validation of an Automated Generative Pretrained Transformer-Based Evaluation Tool for the Physiotherapy Evidence Database Scale. Arch Phys Med Rehabil. 2026. PMID: 41771466. doi:10.1016/j.apmr.2026.02.011
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。