Automation complacency: risks of abdicating medical decision making

Abstract

<p>This work investigates automation complacency in relation to decision support systems used in healthcare contexts, especially their impact on clinicians, patients, and the quality of care. While AI and decision support systems can enhance efficiency and outcomes in healthcare, the potential for automation bias risks clinical perils. These include eroded vigilance, impoverished therapeutic relationships, and potentially poorer outcomes regarding overall well-being. This work highlights these concerns to urge actors in the health sector to effectively integrate technology in a way that spares cognitive resources without compromising the essential role of human experts in making medical decisions. To ensure decision support improves patient care, it is crucial to balance computational processing of information with embodied local expertise; we provide a possible starting point for mindful integration. The implementation of systems in the clinical context should encourage vigilance and guard against fatigue and complacency. There is reason to be excited about increasingly efficient and available care. If the risks of automation complacency are avoided, shared time and resources can be used to preserve and promote valuable interactions, insights, and holistic aspects of care.<br></p>