用户反馈对证据合成评估工具开发的重要性:以TACIT为例
本文评论比较了四种随机试验评估工具的用户测试程序,并详述了作者开发的新工具TACIT在开发早期和后期分别收集非正式与正式用户反馈的过程,强调充分用户反馈对工具开发的重要性,建议采用迭代修订并由不同预期用户在不同情境下以多种形式提供反馈。
要点
- 评估工具开发应纳入不同预期用户在不同情境下的迭代反馈。
- TACIT在开发早期收集非正式反馈,后期对草案收集正式反馈。
- 工具公开发布后、最终发表前的用户反馈可覆盖大量用户和广泛试验。
核心结论
- 充分的用户反馈对证据合成评估工具的开发至关重要。
- 建议由不同预期用户在不同情境下应用工具并以多种形式反馈,迭代修订工具。
- 工具公开发布后至最终发表前的用户反馈有潜力覆盖大量用户和广泛试验。
研究设计与样本
- 本文为评论文章,概述了四种用于证据合成中随机试验的评估工具的用户测试程序。
- 描述并讨论了作者开发的新工具TACIT的用户测试程序。
主要发现
- 所选评估工具的用户反馈程序在用户类型、涉及试验、用户测试场景和反馈收集方式上存在差异。
- TACIT在开发早期收集非正式用户反馈,并对晚期草案收集正式反馈。
- TACIT的正式反馈包括研讨班参与者对预选试验集使用工具,以及系统评价者在自己的评价中使用工具。
- 反馈形式包括半结构化访谈和问卷。
临床怎么用
- 系统评价者应应用公开发布版的TACIT并向作者提供反馈。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
BACKGROUND AND OBJECTIVE: The validity of randomized trials included in evidence syntheses is typically assessed using appraisal tools. Such tools are often developed by a group of experts and usually involve some feedback from intended tool users. STUDY DESIGN AND SETTING: In this commentary, we provide an overview of user testing procedures in the development of four selected appraisal tools for evidence synthesis of randomized trials and describe and discuss the user testing procedures in TACIT, a new Tool for Addressing Conflicts of Interest in Trials (TACIT) that we have developed. RESULTS: The procedures for user feedback varied between the selected appraisal tools in relation to the type of users and trials involved, in which setting user testing was done, and how feedback was collected. In TACIT, informal user feedback was provided in the early phase of tool development and formalized feedback was provided to a late draft of the tool. The formalized feedback involved workshop participants using the tool on a preselected set of trials and systematic reviewers using the tool in their own review. Feedback formats included both semistructured interviews and a questionnaire. CONCLUSIONS: Adequate user feedback is important for the development of appraisal tools, and we suggest striving for an iterative process of tool revisions informed by different groups of intended users applying the tool in different contexts and providing feedback using diverse formats. Feedback from users after public release of the tool, but before final publication, has the potential to reach a large number of users applying the tool on a broad set of trials. We encourage systematic reviewers to apply the publicly released version of TACIT and contact us with feedback.
引用
- Lundh A, Boutron I, Stewart LA, et al. The importance of adequate user feedback for developing appraisal tools for evidence synthesis-the case of TACIT (Tool for Addressing Conflicts of Interest in Trials). J Clin Epidemiol. 2026. PMID: 42431474. doi:10.1016/j.jclinepi.2026.112415
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。