通过学习已发表示例数据库可估计系统综述更新中结论改变的风险
该研究应用分类树(一种机器学习方法),以系统综述及其更新配对为样本,建模系统综述更新中结论改变的风险。共识别63对系统综述与更新,其中20对(32%)在更新中结论发生改变。使用新增试验信息的分类器性能最高(AUC 0.71;召回率0.75;精确率0.43),而使用较少特征的分类器AUC为0.65、召回率0.75、精确率0.39。结论:估计更新中结论改变风险时,新增试验的样本量信息最有用。
要点
- 63对系统综述与更新中,20对(32%)更新后结论发生改变。
- 使用新增试验信息的分类器AUC 0.71、召回率0.75、精确率0.43。
- 新增试验的样本量信息对估计结论改变风险最有用。
核心结论
- 估计系统综述更新中结论改变的风险时,关于更新中将新增的试验规模的信息最有用。
- 未来用于警示结论改变风险的工具将受益于自动筛选相关试验的补充工具。
研究设计与样本
- 应用分类树(一种机器学习方法)建模系统综述更新中结论改变的风险,以系统综述及其更新的配对为样本。
- 分类器使用从系统综述及已发表更新中新增的相关试验提取的一组特征构建。
- 模型性能通过召回率、精确率和受试者工作特征曲线下面积(AUC)衡量。
主要发现
- 共识别63对系统综述与更新,其中20对(32%)在更新中结论发生改变。
- 使用新增试验信息的分类器性能最高:AUC 0.71;召回率0.75;精确率0.43。
- 使用较少特征的分类器:AUC 0.65;召回率0.75;精确率0.39。
局限性
- 原文未报告局限性
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
OBJECTIVES: To determine which systematic review characteristics are needed to estimate the risk of conclusion change in systematic review updates. STUDY DESIGN AND SETTING: We applied classification trees (a machine learning method) to model the risk of conclusion change in systematic review updates, using pairs of systematic reviews and their updates as samples. The classifiers were constructed using a set of features extracted from systematic reviews and the relevant trials added in published updates. Model performance was measured by recall, precision, and area under the receiver operating characteristic curve (AUC). RESULTS: We identified 63 pairs of systematic reviews and updates, of which 20 (32%) exhibited a change in conclusion in their updates. A classifier using information about new trials exhibited the highest performance (AUC: 0.71; recall: 0.75; precision: 0.43) compared to a classifier that used fewer features (AUC: 0.65; recall: 0.75; precision: 0.39). CONCLUSION: When estimating the risk of conclusion change in systematic review updates, information about the sizes of trials that will be added in an update are most useful. Future tools aimed at signaling conclusion change risks would benefit from complementary tools that automate screening of relevant trials.
引用
- Bashir R, Surian D, Dunn AG, et al. The risk of conclusion change in systematic review updates can be estimated by learning from a database of published examples. J Clin Epidemiol. 2019. PMID: 30849512. doi:10.1016/j.jclinepi.2019.02.015
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。