文献速递消化、代谢与肾脏急症

基于CMS Hospital Compare指标的医院排名可重复性:指标信度的作用

Reproducibility of Hospital Rankings Based on Centers for Medicare & Medicaid Services Hospital Compare Measures as a Function of Measure Reliability.
PubMed 文献 · JAMA Netw Open · 2021-12-01
一句话结论指标信度提高并未降低医院排名重分类率,作者认为不应以重测信度评估指标信度。

该横断面研究分析2017与2018报告期CMS Hospital Compare的28项指标(死亡率、再入院、手术并发症等),计算医院在三分位、四分位、十分位及统计离群值分类下的重分类率,并用线性回归检验重分类率与组内相关系数的关联。结果显示中位组内相关系数0.78,中位重分类率十分位70%、四分位43%、三分位34%、离群值3.8%。组内相关系数每增加0.1,十分位、四分位及离群值分类的重分类率反而分别增加6.80、4.15、3.70个百分点。作者结论认为更可靠的指标并未降低医院重分类率,提示不应以重测信度评估指标信度。

要点

  • 28项CMS指标中位组内相关系数0.78(IQR 0.72-0.81),范围0.50-0.85;十分位排名中位重分类率70%。
  • 组内相关系数每增加0.1,十分位排名重分类率增加6.80个百分点(95%CI 2.28-11.30;P=0.005)。
  • 四分位排名重分类率增加4.15个百分点(95%CI 1.16-7.14;P=0.008),离群值分类增加3.70个百分点(95%CI 1.30-6.09;P=0.004)。
  • 三分位排名重分类率增加1.47个百分点(95%CI -1.84至4.77;P=0.37),未达统计学显著。
  • 作者结论认为不应以重测信度评估指标信度。

适用人群与场景

  • 2017报告期(基于2014-2017年数据)与2018报告期(基于2015-2018年数据)CMS Hospital Compare数据。
  • 纳入28项指标,包括死亡率(急性心肌梗死、充血性心力衰竭、肺炎、冠状动脉旁路移植术)、再入院(急性心肌梗死、充血性心力衰竭、肺炎、冠状动脉旁路移植术)及手术并发症(术后急性肾衰竭、术后呼吸衰竭、术后脓毒症、抢救失败)。
  • 分析队列含4452家医院,每项指标贡献数据医院中位数2927家(IQR 2378-3160)。
  • 参与住院预付费系统(Inpatient Prospective Payment System)的医院n=3195,中位床位数141(IQR 69-261),平均每日住院患者数70(IQR 24-155),中位弱势患者医院百分比38.2%(IQR 18.7%-36.6%)。

研究设计与关键结果

  • 横断面设计,分析2017与2018报告期28项CMS指标,研究时间2020年12月13日至2021年9月30日。
  • 医院按三分位、四分位、十分位及统计离群值分类,计算重分类率;用线性回归检验重分类率与组内相关系数的关联。
  • 中位组内相关系数0.78(IQR 0.72-0.81),范围0.50-0.85。
  • 中位重分类率:十分位排名70%(IQR 62%-71%),四分位43%(IQR 39%-45%),三分位34%(IQR 31%-36%),离群值分类3.8%(IQR 2.0%-6.2%)。
  • 组内相关系数每增加0.1,重分类率变化:十分位增加6.80个百分点(95%CI 2.28-11.30;P=0.005);四分位增加4.15个百分点(95%CI 1.16-7.14;P=0.008);三分位增加1.47个百分点(95%CI -1.84至4.77;P=0.37);离群值分类增加3.70个百分点(95%CI 1.30-6.09;P=0.004)。

证据类型、等级与局限

  • 证据类型:横断面研究。
  • 作者结论:更可靠的指标并未降低医院重分类率,提示不应以重测信度评估指标信度。
  • 局限:原文未报告局限性。

对床旁实践的改变

  • 作者认为,指标信度不应以重测信度来评估。

提炼说明

  • 本条依据论文摘要整理;摘要未涉及「诊断标准与关键阈值」「处置流程与药物剂量」「红旗征与不能漏的情况」「指南或共识出处」,故未列出这些栏目;全文请点「阅读原文」。
英文原文(摘要)· 点击展开

IMPORTANCE: Unreliable performance measures can mask poor-quality care and distort financial incentives in value-based purchasing.

OBJECTIVE: To examine the association between test-retest reliability and the reproducibility of hospital rankings.

DESIGN, SETTING, AND PARTICIPANTS: In a cross-sectional design, Centers for Medicare & Medicaid Services Hospital Compare data were analyzed for the 2017 (based on 2014-2017 data) and 2018 (based on 2015-2018 data) reporting periods. The study was conducted from December 13, 2020, to September 30, 2021. This analysis was based on 28 measures, including mortality (acute myocardial infarction, congestive heart failure, pneumonia, and coronary artery bypass grafting), readmissions (acute myocardial infarction, congestive heart failure, pneumonia, and coronary artery bypass grafting), and surgical complications (postoperative acute kidney failure, postoperative respiratory failure, postoperative sepsis, and failure to rescue).

EXPOSURES: Measure reliability based on test-retest reliability testing.

MAIN OUTCOMES AND MEASURES: The reproducibility of hospital rankings was quantified by calculating the reclassification rate across the 2017 and 2018 reporting periods after categorizing the hospitals into terciles, quartiles, deciles, and statistical outliers. Linear regression analysis was used to examine the association between the reclassification rate and the intraclass correlation coefficient for each of the classification systems.

RESULTS: The analytic cohort consisted of 28 measures from 4452 hospitals with a median of 2927 (IQR, 2378-3160) hospitals contributing data for each measure. The hospitals participating in the Inpatient Prospective Payment System (n = 3195) had a median bed size of 141 (IQR, 69-261), average daily census of 70 (IQR, 24-155) patients, and a median disproportionate share hospital percentage of 38.2% (IQR, 18.7%-36.6%). The median intraclass correlation coefficient was 0.78 (IQR, 0.72-0.81), ranging between 0.50 and 0.85. The median reclassification rate was 70% (IQR, 62%-71%) when hospitals were ranked by deciles, 43% (IQR, 39%-45%) when ranked by quartiles, 34% (IQR, 31%-36%) when ranked by terciles, and 3.8% (IQR, 2.0%-6.2%) when ranked by outlier status. Increases in measure reliability were not associated with decreases in the reclassification rate. Each 0.1-point increase in the intraclass correlation coefficient was associated with a 6.80 (95% CI, 2.28-11.30; P = .005) percentage-point increase in the reclassification rate when hospitals were ranked into performance deciles, 4.15 (95% CI, 1.16-7.14; P = .008) when ranked into performance quartiles, 1.47 (95% CI, 1.84, 4.77; P = .37) when ranked into performance terciles, and 3.70 (95% CI, 1.30-6.09; P = .004) when ranked by outlier status.

CONCLUSIONS AND RELEVANCE: In this study, more reliable measures were not associated with lower rates of reclassifying hospitals using test-retest reliability testing. These findings suggest that measure reliability should not be assessed with test-retest reliability testing.

来源:JAMA Netw Open,查看原文。版权归原作者所有。

参考来源

  1. Glance LG, Nerenz DR, Joynt Maddox KE, et al. Reproducibility of Hospital Rankings Based on Centers for Medicare & Medicaid Services Hospital Compare Measures as a Function of Measure Reliability. JAMA Netw Open. 2021. PMID: 34874402. doi:10.1001/jamanetworkopen.2021.37647

供执业医师抢救与诊疗参考,不替代临床判断;剂量请结合体重、肝肾功能和本院药品规格复核,以指南原文为准。