一项心理测量学研究显示 AMSTAR 2 是有效且中等信度的评价工具
该心理测量学研究在重度抑郁症药物或心理干预系统综述中评估 AMSTAR 2 的评分者间信度并与 AMSTAR、ROBIS 比较,结果 AMSTAR 2 信度中等,效度与 AMSTAR、ROBIS 呈正相关。
要点
- AMSTAR 2 评分者间信度中等,中位 kappa 值为 0.51。
- AMSTAR 2 与 AMSTAR 效度正相关(r=0.91),与 ROBIS 正相关(r=0.84)。
- 总体评级上 AMSTAR 2 与 ROBIS 一致性高,与 AMSTAR 一致性较低。
核心结论
- AMSTAR 2 的评分者间信度略低于 AMSTAR,高于 ROBIS。
- 效度指标表明 AMSTAR 2 与 ROBIS 和 AMSTAR 均密切相关。
研究设计与样本
- 研究设置两组评分者,每组四人,每组各自分配两个样本之一,每个样本含 30 篇系统综述。
- 八名评分者均对自己的样本应用 AMSTAR 2,每组还分别应用 AMSTAR 或 ROBIS。
- 计算 Fleiss kappa 和 Gwet AC1,并评估工具间的一致性。
主要发现
- 作为评分者间信度指标的中位 kappa 值显示:AMSTAR 2 为中等一致(中位=0.51),AMSTAR 为高度一致(中位=0.62),ROBIS 为一般一致(中位=0.27)。
- 效度结果显示 AMSTAR 与 AMSTAR 2 呈正相关(r=0.91),ROBIS 与 AMSTAR 2 也呈正相关(r=0.84)。
- 对于总体评级,AMSTAR 2 与 ROBIS 显示高一致性,与 AMSTAR 一致性较低。
关键数值与推荐
- AMSTAR 2 评分者间信度中位 kappa=0.51。
- AMSTAR 评分者间信度中位 kappa=0.62。
- ROBIS 评分者间信度中位 kappa=0.27。
- AMSTAR 与 AMSTAR 2 效度相关系数 r=0.91。
- ROBIS 与 AMSTAR 2 效度相关系数 r=0.84。
临床怎么用
- 原文未据此给出推荐。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
OBJECTIVES: The objectives of this study were to determine the interrater reliability (IRR) of assessment of multiple systematic reviews (AMSTAR) 2 for reviews of pharmacological or psychological interventions for the treatment of major depression, to compare it to that of AMSTAR and risk of bias in systematic reviews (ROBIS), and to assess the convergent validity between the appraisal tools. STUDY DESIGN AND SETTING: Two groups of four raters were each assigned one of two samples of 30 systematic reviews. All eight raters applied AMSTAR 2 to their sample. Each group also applied either AMSTAR or ROBIS. Fleiss' kappa and Gwet's AC1 were calculated, and agreement between the tools was assessed. RESULTS: The median kappa values as a measure of IRR indicated a moderate agreement for AMSTAR 2 (median = 0.51), a substantial agreement for AMSTAR (median = 0.62), and a fair agreement for ROBIS (median = 0.27). Validity results showed a positive association for AMSTAR and AMSTAR 2 (r = 0.91) as well as ROBIS and AMSTAR 2 (r = 0.84). For the overall rating, AMSTAR 2 showed a high concordance with ROBIS and a lower concordance with AMSTAR. CONCLUSION: The IRR of AMSTAR 2 was found to be slightly lower than the IRR of AMSTAR and higher than the IRR of ROBIS. Validity measurements indicate that AMSTAR 2 is closely related to both ROBIS and AMSTAR.
引用
- Lorenz RC, Matthias K, Pieper D, et al. A psychometric study found AMSTAR 2 to be a valid and moderately reliable appraisal tool. J Clin Epidemiol. 2019. PMID: 31152864. doi:10.1016/j.jclinepi.2019.05.028
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。