文献速递重症综合管理与镇静镇痛

评分者间信度及分歧对急性生理与慢性健康评分IV死亡预测的影响

Inter-Rater Reliability and Impact of Disagreements on Acute Physiology and Chronic Health Evaluation IV Mortality Predictions.
PubMed 文献 · Crit Care Explor · 2019-10-30
一句话结论在急性生理与慢性健康评分IV中,评分者对格拉斯哥昏迷量表新设的「无法评分」选项使用不一致,是导致死亡预测信度下降的主要原因。

本研究为横断面研究,在学术医疗中心5个成人ICU中,由两名经急性生理与慢性健康评分IV方法学培训的临床评分者,对41份病历的手动录入元素进行独立提取。结果显示,格拉斯哥昏迷量表(一致率63.8%,组内相关系数0.83)、急性生理与慢性健康评分诊断(一致率68.3%,kappa 0.67)和入院来源(一致率90.2%,kappa 0.85)是分歧的主要来源。因格拉斯哥昏迷量表分歧导致的预测死亡率差异显著(评分者1的观察与预期死亡率比为1.009,评分者2为1.134,p<0.05)。新设的「无法评分」选项仅用于18%的格拉斯哥昏迷量表测量,却占63%的「重大」格拉斯哥昏迷量表分歧和50%的总体预测死亡率差异。作者在 study 后开发了一套格拉斯哥昏迷量表算法以提高与该选项使用相关的信度。

要点

  • 格拉斯哥昏迷量表一致率仅63.8%(组内相关系数0.83),是急性生理与慢性健康评分IV手动录入元素中信度最低的一项。
  • 急性生理与慢性健康评分诊断一致率68.3%(kappa 0.67),入院来源一致率90.2%(kappa 0.85),分歧相对较小。
  • 因格拉斯哥昏迷量表分歧导致两名评分者的观察与预期死亡率比显著不同(1.009 vs 1.134,p<0.05)。
  • 「无法评分」选项仅用于18%的格拉斯哥昏迷量表测量,却占63%的重大格拉斯哥昏迷量表分歧和50%的总体预测死亡率差异。
  • 作者在 study 后开发了一套格拉斯哥昏迷量表算法,以改善与该「无法评分」选项使用相关的信度。

适用人群与场景

  • 入住5个成人ICU的患者(病历抽样n=41),用于评估急性生理与慢性健康评分IV手动录入元素的评分者间信度。

研究设计与关键结果

  • 横断面研究,在学术医疗中心5个成人ICU中抽取41份病历,由两名经急性生理与慢性健康评分IV方法学培训的临床评分者独立提取手动录入元素。
  • 手动录入元素包括急性生理与慢性健康评分诊断、格拉斯哥昏迷量表评分、入院来源、慢性病、择期/急诊手术和呼吸机使用。
  • 格拉斯哥昏迷量表一致率63.8%(组内相关系数0.83),急性生理与慢性健康评分诊断一致率68.3%(kappa 0.67),入院来源一致率90.2%(kappa 0.85)。
  • 因格拉斯哥昏迷量表分歧导致的预测死亡率差异显著:评分者1的观察与预期死亡率比为1.009,评分者2为1.134(p<0.05)。
  • 因急性生理与慢性健康评分诊断或入院来源分歧导致的预测死亡率差异可忽略不计。
  • 「无法评分」选项用于18%的格拉斯哥昏迷量表测量,却占63%的「重大」格拉斯哥昏迷量表分歧和50%的总体急性生理与慢性健康评分预测死亡率差异。

证据类型、等级与局限

  • 证据类型:横断面研究。
  • 局限:原文未报告局限性。

对床旁实践的改变

  • 作者认为,评分者对格拉斯哥昏迷量表新设「无法评分」选项使用不一致,是导致格拉斯哥昏迷量表信度和急性生理与慢性健康评分IV死亡预测信度下降的重要原因。
  • 作者在 study 后开发了一套格拉斯哥昏迷量表算法,以改善与该「无法评分」选项使用相关的信度,并予以展示。

提炼说明

  • 本条依据论文摘要整理;摘要未涉及「诊断标准与关键阈值」「处置流程与药物剂量」「红旗征与不能漏的情况」「指南或共识出处」,故未列出这些栏目;全文请点「阅读原文」。
英文原文(摘要)· 点击展开

UNLABELLED: Acute Physiology and Chronic Health Evaluation is a well-validated method to risk-adjust ICU patient outcomes. However, predictions may be affected by inter-rater reliability for manually entered elements. We evaluated inter-rater reliability for Acute Physiology and Chronic Health Evaluation IV manually entered elements among clinician abstractors and assessed the impacts of disagreements on mortality predictions.

DESIGN: Cross-sectional.

SETTING: Academic medical center.

SUBJECTS: Patients admitted to five adult ICUs.

INTERVENTIONS: None.

MEASUREMENTS AND MAIN RESULTS: Acute Physiology and Chronic Health Evaluation IV manually entered elements were abstracted from a selection of charts (n = 41) by two clinician "raters" trained in Acute Physiology and Chronic Health Evaluation IV methodology. Rater agreement (%) was determined for each manually entered element, including Acute Physiology and Chronic Health Evaluation diagnosis, Glasgow Coma Scale score, admission source, chronic conditions, elective/emergency surgery, and ventilator use. Cohen's kappa (K) or intraclass correlation coefficient was calculated for nominal and continuous manually entered elements, respectively. The impacts of manually entered element choices on Acute Physiology and Chronic Health Evaluation IV mortality predictions were computed using published Acute Physiology and Chronic Health Evaluation IV equations, and observed to expected hospital mortality ratios were compared between rater groups. The majority of manually entered element inconsistency was due to disagreement in choice of Glasgow Coma Scale (63.8% agreement, 0.83 intraclass correlation coefficient), Acute Physiology and Chronic Health Evaluation diagnosis (68.3% agreement, 0.67 kappa), and admission source (90.2% agreement, 0.85 kappa). The difference in predicted mortality between raters related to Glasgow Coma Scale disagreements was significant (observed to expected mortality ratios for Rater 1 [1.009] vs Rater 2 [1.134]; p < 0.05). Differences related to Acute Physiology and Chronic Health Evaluation diagnosis or admission source disagreements were negligible. The new "unable to score" choice for Glasgow Coma Scale was used for 18% of Glasgow Coma Scale measurements but accounted for 63% of "major" Glasgow Coma Scale disagreements, and 50% of the overall difference in Acute Physiology and Chronic Health Evaluation-predicted mortality between raters.

CONCLUSIONS: Inconsistent use among raters of the new "unable to score" choice for Glasgow Coma Scale introduced in Acute Physiology and Chronic Health Evaluation IV was responsible for important decreases in both Glasgow Coma Scale and Acute Physiology and Chronic Health Evaluation IV mortality prediction reliability in our study. A Glasgow Coma Scale algorithm we developed after the study to improve reliability related to use of this new "unable to score" choice is presented.

来源:Crit Care Explor,查看原文。版权归原作者所有。

参考来源

  1. Simkins M, Iqbal A, Gronemeyer A, et al. Inter-Rater Reliability and Impact of Disagreements on Acute Physiology and Chronic Health Evaluation IV Mortality Predictions. Crit Care Explor. 2019. PMID: 32166239. doi:10.1097/CCE.0000000000000059

供执业医师抢救与诊疗参考,不替代临床判断;剂量请结合体重、肝肾功能和本院药品规格复核,以指南原文为准。