人工智能支持的证据综合：智能输注泵互操作性的案例研究。
全文（开放获取） · 中文译文
摘要 背景/目标：人工智能工具已成为系统评价和卫生技术评估（HTA）有前景的方法学支持。智能输注泵互操作性因其对用药安全、护理工作流程和医院质量改进的意义，是一个相关的案例研究。目的是评估人工智能作为方法学支持工具在整个系统评价中的表现，以智能输注泵-电子健康档案互操作性的证据合成为案例。 方法：按照PRISMA 2020指南进行了一项系统评价。在MEDLINE、Embase和Cochrane Library数据库中进行了检索。在严格人工监督下，将人工智能辅助工具（ChatGPT GPT-4和Open Science Reviewer）纳入数据提取、基于STROBE标准的报告评估以及方法学局限性的探索性识别。采用描述性方法评估人工智能辅助提取与人工提取之间的一致性。 结果：在评估的变量中，AI辅助数据提取与人工数据提取的总体一致性为82.5%（99/120）。结构化变量的一致性最高，包括研究设计（10/10；100.0%）、研究识别变量（19/20；95.0%）和参与者特征（36/40；90.0%）。研究内容变量（18/30；60.0%）和方法学评价（16/20；80.0%）的一致性较低。在21处差异中，误分类错误最常见（13/21；61.9%），其次是遗漏（3/21；14.3%）、数据不完整（3/21；14.3%）和幻觉（2/21；9.5%）。AI辅助识别方法学局限性与人类评估相比显示出实质性的描述一致性，但在判断性解释方面能力有限。 结论：人工智能在数据提取和报告评价等结构化审评任务中显示出实用性，但在需要解释性和方法学判断的任务中表现出局限性。因此，在整个审评过程中，人工监督仍然至关重要。这些发现来源于单一案例研究，不应推广到所评估情境和AI工具之外。 智能输液泵代表一种护理关键的健康技术，对用药安全、工作流程组织和护理质量有直接影响。这些系统支持减少用药错误，而用药错误仍然是医院实践中最为持久的安全挑战之一[,]，其实施对临床决策、工作量和患者结局具有重大影响[,]。 尽管实证研究越来越多，但由于文献中报告的研究设计、结局和实施背景的多样性，关于有效性、背景因素和非预期后果的证据基础仍然存在异质性[]。 卫生技术迅速融入临床实践，加剧了对稳健且及时的证据的需求，以支持医疗卫生系统的决策。卫生技术评估（HTA）和系统评价是为政策提供信息的基础方法学。这些研究在护理领域尤其具有影响力，因为技术介导的护理服务、用药安全和工作流程优化直接影响患者结局和专业实践[,]。系统评价仍然是综合卫生技术有效性、安全性和实施证据的金标准。然而，它们资源密集、耗时，并且容易因延迟而降低其在快速变化领域中的相关性[]。由于生物医学文献数量不断增长、当代干预措施的复杂性、原始研究的方法学局限、日益复杂的统计学方法，以及提供及时证据以支持医疗卫生决策的需要，系统评价已变得日益严苛。包括PRISMA 2020在内的国际报告标准强调透明性和方法学严谨性，但本身并不能解决手动证据综合所固有的结构性低效问题[]。 人工智能（AI）和机器学习方法已成为应对这些挑战的有前景的方法学支持工具。在过去十年中，基于人工智能的方法已越来越多地应用于系统评价过程的关键阶段，包括文献筛选、偏倚风险评估、数据提取和证据图谱[,]。基于人类决策训练的主动学习算法可以在保持高敏感性的同时大幅减少筛选工作量，使评价团队能够更有效地优先处理临床相关证据[,]。诸如RobotReviewer之类的工具已证明在随机试验中自动化结构化偏倚风险评估的可行性，凸显了人工智能在结合评价者判断的同时增强方法学一致性的潜力[]。 从卫生服务角度来看，人工智能辅助的系统评价的相关性不仅限于技术效率。更快且更具可扩展性的证据综合可以增强护理领导者和政策制定者对新兴安全问题的响应能力，以及对新技术相关的人力影响和实施障碍的应对能力 []。此外，将人工智能整合到证据综合过程中，与护理领域更广泛的数字化转型议程相一致，该议程强调使用高级分析来支持临床推理、质量改进和以患者为中心的护理 [,]。重要的是，人工智能工具并非旨在取代临床医生和护理研究者的批判性评估，而是为了增强方法论能力，并使人们能够更多地关注在复杂护理情境中对研究结果的解释和应用。 将AI支持的方法应用于智能输液泵的评估，为探索这些工具是否能在不损害方法学质量或可解释性的前提下有意义地支持证据综合过程提供了具体机会。因此，本案例研究能够在临床相关领域中考察将AI整合到系统评价工作流程中的潜在益处和实践局限性。本研究的一个显著特点是，在系统评价的多个阶段整合AI辅助支持，同时保留人类评审员作为最终决策者。这种多阶段、人在回路的方法能够更全面地评估AI辅助工具在何时达到可靠的一致性，以及何时人类判断仍然必不可少。 本研究的主要目的是评估人工智能作为方法学支持工具在人类主导的系统评价中的表现，以智能输液泵–电子健康记录互操作性作为案例研究，考察人工智能辅助流程如何整合到传统综述工作流中，评估其对效率与严谨性的贡献，并讨论这些方法对医疗保健领域证据综合与决策制定的启示。主要目标是评估人工智能辅助数据提取与人工数据提取在预设数据元素上的一致性，以百分比一致性和差异分类作为一致性的主要衡量指标。次要目标是考察在报告质量评估和方法学评价中的一致性。我们假设，对于结构化且明确定义的任务，人工智能辅助流程与人工评审者之间将表现出高度一致性，而对于需要背景或方法学解释的复杂叙述性变量，则一致性较低。 本研究是作为一项嵌入系统评价中的方法学个案研究进行的。该系统评价为在证据综合过程的选定阶段评估人工智能（AI）辅助工具的性能提供了一个真实世界框架。该系统评价重点关注医院环境中智能输液泵与电子健康记录之间的互操作性。该评价遵循PRISMA 2020指南。本系统评价按照预先在PROSPERO（CRD420261327073）注册的方案进行，方案可在 https://www.crd.york.ac.uk/PROSPERO/view/CRD420261327073（访问日期：2026年8月20日）获取。 AI被概念化为一种嵌入在常规、由人主导的系统评价工作流程中的方法学辅助工具。关于研究纳入、数据解读和质量评估的所有最终决定，均完全由人工评价者做出。因此，本稿的主要焦点是评估AI辅助的评价过程，而非所评价卫生技术的临床有效性。 2.2. 评价问题与纳入标准 该系统评价围绕以下问题展开：在医院护理中，智能输液泵与电子健康记录系统之间互操作性的有效性、安全性和实施影响是什么？ 入选标准采用“人群–干预–对照–结局”框架预先定义。人群包括通过输液泵接受静脉用药的住院患者。干预措施是智能输液泵与电子健康记录之间的互操作性，定义为自动编程、自动记录或双向数据交换。 对照包括互操作性实施前阶段或非互操作性工作流程。 结局根据已注册的综述方案分为主要结局和额外结局。主要结局关注用药安全，包括用药错误、患者安全指标、警报相关结局以及与智能输液泵和电子健康记录互操作性相关的安全性结局。额外结局包括经济、组织和实施相关指标，如记录质量、工作流程指标、培训要求、可用性问题及其他过程相关结局。 纳入在真实医院环境中开展的观察性研究。模拟研究、会议摘要、社论以及无互操作性的研究被排除。 2.3 信息来源与检索策略 检索在MEDLINE（PubMed）、Embase和Cochrane图书馆中进行，并辅以参考文献筛查和选定的灰色文献来源。最终检索更新于2024年7月进行。 检索策略结合了与智能泵、电子健康记录、互操作性和静脉用药安全相关的受控词汇和自由文本检索词。检索限定为英文和西班牙文。完整的检索策略见。 2.4 研究选择 去除重复文献后，由两名评审员在标题/摘要和全文层面独立筛选记录。分歧通过共识解决。所有研究选择决定完全由人工评审员根据预先确定的资格标准做出。 2.5 使用人工智能作为方法学支持 上述系统评价过程按照常规证据综合标准进行。随后将人工智能辅助工具纳入选定的评价阶段，以评估其潜在的方法学贡献。具体而言，ChatGPT（OpenAI）和GPT Open Science Reviewer被用于支持数据提取、基于STROBE标准的报告评价，以及方法学局限性的探索性识别。所有人工智能生成的输出均由研究者独立审查和验证。 本研究通过研究时可用的 GPT-4 模型使用 ChatGPT。AI 辅助分析由基于预定义数据提取模板的结构化提示引导，重点识别研究特征、干预特征、结局指标、方法学局限性和报告要素。未主动修改任何模型参数，整个综述过程均使用默认平台设置。两种工具生成的输出均被视为初步建议，并在纳入综述前与原始全文出版物进行系统性核对。 在数据提取期间，采用了一种基于大语言模型的工具作为结构化阅读辅助工具。该系统被提示根据评价者的提取模板识别预定义的数据元素，如研究设计、干预特征、结局指标和报告的局限性。AI 辅助提取独立于人工评价者提取进行，随后对两种输出进行比较。在数据提取和方法学评价期间，采用了一种基于大语言模型的工具作为结构化阅读辅助工具。该系统被提示根据评价者的提取模板识别预定义的数据元素，如研究设计、干预特征、结局指标和报告的局限性。 同时，所有数据由第二位评审员手动提取。人工智能（AI）工具生成的输出与手动提取的数据进行了系统比较，任何差异均在全文层面进行审查。AI生成输出与手动提取数据之间的任何差异，均对照原始全文出版物进行审查，并通过评审员共识解决。数据准确性的最终责任仍由人工评审员承担。 在报告质量和方法学评价方面，AI输出仅用于识别潜在的报告缺陷和方法学局限性，以供评审员进一步考虑。这些输出不影响最终的方法学判断，最终判断仍基于评审员的评估和共识。 AI工具在任何阶段均未就研究质量、相关性或研究结果的解释生成独立判断。关于纳入、数据准确性、偏倚风险和综合的最终决定完全由人工评审员作出。 AI辅助数据提取和评价过程中使用的提示词由具有卫生技术评估和系统评价方法学专长的评审团队成员制定。没有计算机科学家或AI专家参与提示词的制定。该研究未正式评估学习曲线或以往使用AI辅助评审工具的经验。 2.6. 数据提取 提取框架包括120个预定义数据项，分为五个领域：研究识别变量（包括作者、发表年份、国家）、研究设计特征、参与者特征、研究内容变量（干预措施、结局和主要发现）以及方法学评价项目。这些预定义字段同时用于AI辅助和手动数据提取。 数据由两名评审者使用一个包括研究设计、环境、人群、互操作性特征和结局的标准化模板独立提取。AI辅助输出已与原始出版物逐行核对。数据准确性的最终责任仍由人工评审者承担。 2.7. 偏倚风险与报告质量评估 两名评审者使用ROBINS-I独立评估偏倚风险。报告质量使用STROBE进行描述性评价。纳入该评估是因为本研究旨在评估AI辅助工具在支持数据提取之外识别报告优势和局限性的能力。 一种基于AI的评审工具被用作探索性辅助手段，以识别潜在的报告缺失和方法学局限性。AI生成的观察结果被记录下来，随后由评审者对照原始研究报告进行审查。这些输出用于支持讨论和批判性反思，但并未直接修改ROBINS-I评估、STROBE评价或最终的方法学判断。最终的方法学评估完全由人工共识决定。 2.8. 数据综合 由于研究设计和结局存在异质性，荟萃分析并不适用。研究结果采用结构化表格进行叙述性综合。结局被分为安全相关、过程相关和警报相关领域。解释侧重于结果的一致性、方法学局限性以及对临床实践的意义。 2.9. 统计分析 采用描述性和基于一致性的分析来评估AI与人工评审者相比的贡献。 研究筛选过程中的一致性通过百分比一致性和科恩卡帕系数（κ）及其95%置信区间进行评估。差异被分类为过度纳入或纳入不足，并通过共识解决。 数据提取的一致性使用预设数据项的百分比一致性进行总结。分析单位是单个提取项，定义为从每项纳入研究中提取的每个预设数据字段。这五项纳入研究共产生120个评估单位。差异根据其类型（遗漏、不完整提取、错误分类或幻觉）进行分类。AI辅助工作流程的效率进行了描述性评估。 所有分析旨在评估一致性和方法学支持，而非诊断准确性，以人工审查员作为参考决策者。 3. 结果 3.1. 研究选择 通过数据库检索共识别出495条记录。去重后，309条记录由两名独立审查员在标题和摘要层面进行筛选。 经过标题和摘要筛选，10篇文章被选入全文评估，其中5篇符合纳入标准，并被纳入最终综合（文中未提供图示或引用）。 纳入的研究一致评估了智能泵互操作性对药物管理流程和安全性相关结局的影响。报告的获益包括选定用药错误的减少、文件记录完整性的改善、药物库依从性的提高以及高风险警报覆盖的减少。然而，证据仍存在异质性，且主要基于观察性前后对照设计，限制了对临床有效性的因果推断。 3.2. AI辅助数据提取中的一致性 在纳入的五项研究中，共评估了120个数据元素。AI辅助提取与人工数据提取之间的总体一致性为82.5%（99/120）（）。 结构化变量的一致性最高，包括研究设计（10/10；100.0%）和研究识别变量（19/20；95.0%）。受试者特征也显示出较高的一致性（36/40；90.0%）。研究内容变量的一致性较低（18/30；60.0%），这些变量需要对叙述性数据进行更复杂的解释。方法学评价变量显示中等一致性（16/20；80.0%）。 一致性按单个提取条目进行评估。从每项纳入研究中提取的每个预定数据字段构成一个评估单元，在五项纳入研究中总共产生120个项目。 3.3. 数据提取中的差异分析 在AI辅助输出与人工数据提取之间共识别出21处差异（）。大多数差异对应错误分类，即AI对提取信息的解释不准确。 典型的错误分类涉及对叙述性部分中报告的研究特征进行过度简化或错误分类。例如，AI生成的输出有时未能区分互操作性相关结果与更广泛的实施结果，导致研究结果的分类不准确。幻觉错误较少见，但通常涉及归因。
全文（开放获取） · 英文原文
【Abstract】 BACKGROUND/OBJECTIVES: Artificial intelligence tools have emerged as promising methodological support for systematic reviews and health technology assessment (HTA). Smart infusion pump interoperability represents a relevant case study due to its implications for medication safety, nursing workflow, and hospital quality improvement. The aim was to evaluate the performance of artificial intelligence as a methodological support tool across a systematic review, using the evidence synthesis on smart infusion pump-electronic health record interoperability as a case study. METHODS: A systematic review following PRISMA 2020 guidelines was conducted. Searches were performed in MEDLINE, Embase, and Cochrane Library databases. AI-assisted tools (ChatGPT GPT-4 and Open Science Reviewer) were incorporated into data extraction, reporting appraisal based on STROBE criteria, and exploratory identification of methodological limitations under strict human supervision. Concordance between AI-assisted and manual extraction was evaluated descriptively. RESULTS: Overall concordance between AI-assisted and manual data extraction was 82.5% (99/120) across assessed variables. Agreement was highest for structured variables, including study design (10/10; 100.0%), study identification variables (19/20; 95.0%), and participant characteristics (36/40; 90.0%). Agreement was lower for study content variables (18/30; 60.0%) and methodological appraisal (16/20; 80.0%). Among the 21 discrepancies, misclassification errors were most common (13/21; 61.9%), followed by omissions (3/21; 14.3%), incomplete data (3/21; 14.3%), and hallucinations (2/21; 9.5%). AI-assisted identification of methodological limitations showed substantial descriptive agreement with human assessments but demonstrated limited capacity for judgmental interpretations. CONCLUSIONS: Artificial intelligence demonstrated utility for structured review tasks such as data extraction and reporting appraisal, but showed limitations in tasks requiring interpretative and methodological judgement. Human oversight therefore remains essential throughout the review process. These findings derive from a single case study and should not be generalised beyond the evaluated context and AI tools. 【1. Introduction】 Smart infusion pumps represent a nursing-critical health technology with direct implications for medication safety, workflow organisation and quality of care. These systems support medication error reduction, which remains among the most persistent safety challenges in hospital practice [,], and their implementation has significant consequences for clinical decision-making, workload and patient outcomes [,]. Despite a growing body of empirical studies, the evidence base on effectiveness, contextual factors and unintended consequences remains heterogeneous, given the diversity of study designs, outcomes and implementation contexts reported in the literature []. The rapid integration of health technologies into clinical practice has intensified the demand for robust and timely evidence to support decision-making in healthcare systems. Health technology assessment (HTA) and systematic reviews are cornerstone methodologies for informing policy. These studies are particularly impactful in nursing, where technology-mediated care delivery, medication safety and workflow optimisation directly affect patient outcomes and professional practice [,]. Systematic reviews remain the gold standard for synthesising evidence on effectiveness, safety and implementation of health technologies. However, they are resource-intensive, time-consuming, and vulnerable to delays that may reduce their relevance in fast-evolving areas []. They have become increasingly demanding due to the growing volume of biomedical literature, the complexity of contemporary interventions, methodological limitations in primary studies, increasingly sophisticated statistical approaches, and the need to deliver timely evidence to support healthcare decision-making. International reporting standards, including PRISMA 2020, emphasise transparency and methodological rigour, but do not in themselves resolve the structural inefficiencies inherent to manual evidence synthesis []. Artificial intelligence (AI) and machine learning methods have emerged as promising methodological support tools to address these challenges. Over the last decade, AI-based approaches have been increasingly applied to key stages of the systematic review process, including citation screening, risk of bias assessment, data extraction, and evidence mapping [,]. Active learning algorithms trained on human decisions can substantially reduce screening workload while maintaining high sensitivity, enabling review teams to prioritise clinically relevant evidence more efficiently [,]. Tools such as RobotReviewer have demonstrated the feasibility of automating structured risk of bias assessment in randomised trials, highlighting the potential of AI to enhance methodological consistency alongside reviewer judgement []. From a health services perspective, the relevance of AI-supported systematic reviews extends beyond technical efficiency. Faster and more scalable evidence synthesis can strengthen the responsiveness of nursing leadership and policymakers to emerging safety concerns, workforce implications and implementation barriers associated with new technologies []. Moreover, the integration of AI into evidence synthesis processes aligns with broader digital transformation agendas in nursing, which emphasise the use of advanced analytics to support clinical reasoning, quality improvement and patient-centred care [,]. Importantly, AI tools are not intended to replace critical appraisal by clinicians and nurse researchers, but to augment methodological capacity and allow greater focus on interpretation and application of findings within complex care contexts. Applying AI-supported methodologies to the evaluation of smart infusion pumps provides a concrete opportunity to explore whether these tools can meaningfully support evidence synthesis processes without compromising methodological quality or interpretability. This case study therefore allows for examining both the potential benefits and the practical limitations of integrating AI into systematic review workflows in a clinically relevant domain. A distinctive feature of this study is the integration of AI-assisted support across multiple stages of a systematic review while retaining human reviewers as the final decision-makers. This multi-stage, human-in-the-loop approach provides a more comprehensive assessment of when AI-assisted tools achieve reliable agreement and when human judgement remains essential. The present study aimed primarily to evaluate the performance of AI as a methodological support tool within a human-led systematic review, using smart infusion pump–electronic health record interoperability as a case study to examine how AI-assisted processes can be integrated into conventional review workflows, assess their contribution to efficiency and rigour, and discuss the implications of these approaches for evidence synthesis and decision-making in healthcare. The primary objective was to assess the concordance between AI-assisted and human data extraction across predefined data elements, using percentage concordance and the classification of discrepancies as the main measures of agreement. Secondary objectives were to examine concordance in reporting-quality assessment and methodological appraisal. We hypothesised that AI-assisted processes would show substantial agreement with human reviewers for structured and explicitly defined tasks, but lower agreement for complex narrative variables requiring contextual or methodological interpretation. 【2. Materials and Methods】 【2.1. Study Design】 This study was conducted as a methodological case study embedded within a systematic review. The systematic review provided a real-world framework for evaluating the performance of AI-assisted tools during selected stages of the evidence synthesis process. The systematic review was focused on evaluating interoperability between smart infusion pumps and electronic health records in hospital settings. The review followed PRISMA 2020 guidelines. This systematic review was conducted in accordance with a prospectively registered protocol in PROSPERO (CRD420261327073), available at https://www.crd.york.ac.uk/PROSPERO/view/CRD420261327073 (accessed on 20 August 2026). AI was conceptualised as a methodological aid embedded within a conventional, human-led systematic review workflow. All final decisions regarding study inclusion, data interpretation and quality assessment were made exclusively by human reviewers. Accordingly, the principal focus of the present manuscript is the evaluation of AI-assisted review processes rather than the clinical effectiveness of the health technology under review. 【2.2. Review Question and Eligibility Criteria】 The review addressed the following question: What is the effectiveness, safety and implementation impact of interoperability between smart infusion pumps and electronic health record systems in hospital care? Eligibility criteria were defined a priori using the Population–Intervention–Comparator–Outcome framework. The population included hospitalised patients receiving intravenous medication via infusion pumps. The intervention was interoperability between smart infusion pumps and electronic health records, defined as automated programming, automated documentation or bidirectional data exchange. Comparators included pre-interoperability periods or non-interoperable workflows. Outcomes were categorised as primary and additional outcomes according to the registered review protocol. Primary outcomes focused on medication administration safety, including medication errors, patient safety indicators, alert-related outcomes, and safety outcomes associated with interoperability between smart infusion pumps and electronic health records. Additional outcomes included economic, organisational and implementation-related measures, such as documentation quality, workflow indicators, training requirements, usability issues and other process-related outcomes. Observational studies conducted in real-world hospital settings were included. Simulation studies, conference abstracts, editorials and studies without interoperability were excluded. 【2.3. Information Sources and Search Strategy】 Searches were conducted in MEDLINE (PubMed), Embase and the Cochrane Library, supplemented by reference screening and selected grey literature sources. The final search update was performed in July 2024. Search strategies combined controlled vocabulary and free-text terms related to smart pumps, electronic health records, interoperability and intravenous medication safety. Searches were limited to English and Spanish. Full strategies are provided in the . 【2.4. Study Selection】 After duplicate removal, records were screened independently by two reviewers at title/abstract and full-text levels. Disagreements were resolved by consensus. All study selection decisions were made exclusively by human reviewers according to the predefined eligibility criteria. 【2.5. Use of AI as Methodological Support】 The systematic review procedures described above were conducted according to conventional evidence synthesis standards. AI-assisted tools were subsequently incorporated into selected review stages to evaluate their potential methodological contribution. Specifically, ChatGPT (OpenAI) and GPT Open Science Reviewer were used to support data extraction, reporting appraisal based on STROBE criteria, and exploratory identification of methodological limitations. All AI-generated outputs were independently reviewed and validated by the investigators. ChatGPT was used through the GPT-4 model available at the time of the study. AI-assisted analyses were guided by structured prompts derived from the predefined data extraction template and focused on identifying study characteristics, intervention features, outcome measures, methodological limitations and reporting elements. No model parameters were actively modified, and the default platform settings were used throughout the review process. Outputs generated by both tools were treated as preliminary suggestions and were systematically checked against the original full-text publications before incorporation into the review. During data extraction, a large-language-model–based tool was employed as a structured reading assistant. The system was prompted to identify predefined data elements, such as study design, intervention characteristics, outcome measures and reported limitations, based on the reviewers’ extraction template. AI-assisted extraction was conducted independently of manual reviewer extraction, and both outputs were subsequently compared. During data extraction and methodological appraisal, a large-language-model–based tool was employed as a structured reading assistant. The system was prompted to identify predefined data elements, such as study design, intervention characteristics, outcome measures and reported limitations, based on the reviewers’ extraction template. In parallel, all data were extracted manually by a second reviewer. Outputs generated by the AI tool were systematically compared against the manually extracted data, and any discrepancies were reviewed at the full-text level. Any discrepancies between AI-generated outputs and manually extracted data were reviewed against the original full-text publications and resolved by reviewer consensus. Final responsibility for data accuracy remained with the human reviewers. For reporting quality and methodological appraisal, AI outputs were used exclusively to identify potential reporting gaps and methodological limitations for further reviewer consideration. These outputs did not influence final methodological judgements, which remained based on reviewer assessment and consensus. At no stage did the AI tool generate independent judgements regarding study quality, relevance or interpretation of findings. Final decisions regarding inclusion, data accuracy, risk of bias and synthesis were made exclusively by the human reviewers. The prompts used during AI-assisted data extraction and appraisal were developed by members of the review team with expertise in health technology assessment and systematic review methodology. No computer scientists or AI specialists were involved in prompt development. The study did not formally assess learning curves or previous experience with AI-assisted review tools. 【2.6. Data Extraction】 The extraction framework comprised 120 predefined data items grouped into five domains: study identification variables (including authorship, publication year, country), study design characteristics, participant characteristics, study content variables (intervention, outcomes and key findings), and methodological appraisal items. These predefined fields were used for both AI-assisted and manual data extraction. Data were extracted independently by two reviewers using a standardised template including study design, setting, population, interoperability characteristics and outcomes. AI-assisted outputs were validated line-by-line against original publications. Final responsibility for data accuracy remained with human reviewers. 【2.7. Risk of Bias and Reporting Quality Assessment】 Two reviewers independently assessed risk of bias using ROBINS-I. Reporting quality was evaluated descriptively using STROBE. This assessment was included because the study sought to evaluate the ability of AI-assisted tools to identify reporting strengths and limitations in addition to supporting data extraction. An AI-based reviewer tool was used as an exploratory aid to identify potential reporting gaps and methodological limitations. AI-generated observations were recorded and subsequently reviewed against the original study reports by the reviewers. These outputs were used to support discussion and critical reflection but did not directly modify ROBINS-I assessments, STROBE evaluations, or final methodological judgements. Final methodological assessments were determined exclusively by human consensus. 【2.8. Data Synthesis】 Due to heterogeneity in study designs and outcomes, meta-analysis was not appropriate. Findings were synthesised narratively using structured tables. Outcomes were grouped into safety-related, process-related and alert-related domains. Interpretation focused on consistency of findings, methodological limitations and implications for clinical practice. 【2.9. Statistical Analysis】 Descriptive and agreement-based analyses were used to evaluate the contribution of AI compared with human reviewers. Agreement during study selection was assessed using percentage agreement and Cohen’s kappa (κ) with 95% confidence intervals. Discrepancies were classified as over- or under-inclusion and resolved by consensus. Agreement in data extraction was summarised using percentage concordance across predefined data items. The unit of analysis was the individual extraction item, defined as each predefined data field extracted from each included study. This yielded a total of 120 assessment units across the five included studies. Discrepancies were classified according to their type (omission, incomplete extraction, misclassification, or hallucination). Efficiency of the AI-assisted workflow was evaluated descriptively. All analyses were designed to assess concordance and methodological support rather than diagnostic accuracy, with the human reviewer as the reference decision-maker. 【3. Results】 【3.1. Study Selection】 A total of 495 records were identified through database searching. After removal of duplicates, 309 records were screened at the title and abstract level by two independent reviewers. Following title and abstract screening, 10 articles were selected for full-text assessment, of which five met the eligibility criteria and were included in the final synthesis (). The included studies consistently evaluated the impact of smart pump interoperability on medication administration processes and safety-related outcomes. Reported benefits included reductions in selected medication errors, improvements in documentation completeness, increased compliance with drug libraries, and reductions in high-risk alert overrides. However, the evidence remained heterogeneous and was largely based on observational before–after designs, limiting causal inference regarding clinical effectiveness. 【3.2. Concordance in AI-Assisted Data Extraction】 A total of 120 data elements were assessed across the five included studies. Overall concordance between AI-assisted and manual data extraction was 82.5% (99/120) (). Agreement was highest for structured variables, including study design (10/10; 100.0%) and study identification variables (19/20; 95.0%). Participant characteristics also showed high concordance (36/40; 90.0%). Lower agreement was observed for study content variables (18/30; 60.0%), which required more complex interpretation of narrative data. Methodological appraisal variables showed moderate agreement (16/20; 80.0%). Agreement was assessed at the level of individual extraction items. Each predefined data field extracted from each included study constituted one assessment unit, yielding a total of 120 items across the five included studies. 【3.3. Analysis of Discrepancies in Data Extraction】 A total of 21 discrepancies were identified between AI-assisted outputs and manual data extraction (). Most discrepancies corresponded to misclassification errors, in which the AI inaccurately interpreted extracted information. Typical misclassification errors involved the oversimplification or incorrect categorisation of study characteristics reported in narrative sections. For example, AI-generated outputs occasionally failed to distinguish between interoperability-related outcomes and broader implementation outcomes, resulting in inaccurate classification of study findings. Hallucination errors were less frequent but typically involved the attribution