决策支持还是自主人工智能?以错误采血试管误差为例
本研究通过计算机模拟以50%比例注入错误采血试管(WBIT)误差,让实验室人员审核人工神经网络对电解质、尿素和肌酐(EUC)结果的预测。人工监督流程识别WBIT的准确度81.2%、灵敏度73.7%、特异度88.6%,而AI模型自主分类各项指标均更优(P值<0.05),准确度92.5%、灵敏度90.6%、特异度94.5%。
要点
- 人工监督AI识别WBIT的准确度81.2%、灵敏度73.7%、特异度88.6%。
- AI自主分类WBIT准确度92.5%、灵敏度90.6%、特异度94.5%,均优于人工监督(P<0.05)。
- 对于计算复杂任务如WBIT识别,AI自主运行可能获得最佳性能。
核心结论
- 人工与AI模型交互可显著改变其性能,对于WBIT错误识别这类计算复杂任务,AI模型自主运行可能达到最佳性能。
研究设计与样本
- 研究使用去标识患者当前及前7天内EUC结果,以50%比例计算机模拟WBIT错误,实验室人员志愿者审核AI模型预测后做最终判断,并与AI自主运行性能比较。
主要发现
- 实验室人员监督分类510套EUC结果,识别WBIT准确度81.2%、灵敏度73.7%、特异度88.6%。
- AI模型自主分类这些样本各项指标均优于人工监督(P值<0.05),准确度92.5%、灵敏度90.6%、特异度94.5%。
关键数值与推荐
- 人工监督流程:准确度81.2%、灵敏度73.7%、特异度88.6%。
- AI自主分类:准确度92.5%、灵敏度90.6%、特异度94.5%,与人工监督比较P值<0.05。
临床怎么用
- 对于WBIT错误识别等计算复杂任务,可考虑让AI模型自主运行以获最佳性能。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
OBJECTIVES: Artificial intelligence (AI) models are increasingly being developed for clinical chemistry applications, however, it is not understood whether human interaction with the models, which may occur once they are implemented, improves or worsens their performance. This study examined the effect of human supervision on an artificial neural network trained to identify wrong blood in tube (WBIT) errors. METHODS: De-identified patient data for current and previous (within seven days) electrolytes, urea and creatinine (EUC) results were used in the computer simulation of WBIT errors at a rate of 50%. Laboratory staff volunteers reviewed the AI model's predictions, and the EUC results on which they were based, before making a final decision regarding the presence or absence of a WBIT error. The performance of this approach was compared to the performance of the AI model operating without human supervision. RESULTS: Laboratory staff supervised the classification of 510 sets of EUC results. This workflow identified WBIT errors with an accuracy of 81.2%, sensitivity of 73.7% and specificity of 88.6%. However, the AI model classifying these samples autonomously was superior on all metrics (p-values<0.05), including accuracy (92.5%), sensitivity (90.6%) and specificity (94.5%). CONCLUSIONS: Human interaction with AI models can significantly alter their performance. For computationally complex tasks such as WBIT error identification, best performance may be achieved by autonomously functioning AI models.
引用
- Farrell CL. Decision support or autonomous artificial intelligence? The case of wrong blood in tube errors. Clin Chem Lab Med. 2021. PMID: 34717051. doi:10.1515/cclm-2021-0873
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。