ARAISE - Augmented Reality (AR) and Artificial Intelligence (AI) Enhanced Simulation Education
We aim to enhance medical simulations with AR and AI elements to provide realistic visual and auditory cues for more immersive and interactive training.
Duration: September 2025 – September 2026
Status: Ongoing
Educational Level: Tertiary Level
Topic: Artificial Intelligence AI, Digital Tools
Keywords: Simulation, AI, Extended Reality, AR
Initial Situation
AR simulations can provide immersive and interactive learning environments within a safe and controlled setting. While high-fidelity simulations, such as those at the University of Bern, offer effective medical training, not all institutions have access to such setups. At institutions like the Bern University of Applied Sciences (BFH), simulation often relies on peer-to-peer practice based on a cognitive apprenticeship approach. Regardless of format, key visual and auditory cues essential for accurate clinical assessment are often missing. For example, recognizing skin color changes in anaphylaxis or hypoxia is critical, but difficult to replicate on mannequins or peers. Abnormal breathing sounds and background noise, typical in real clinical settings, are also rarely integrated. With AR, visual and auditory cues can be projected onto mannequins or peers, thereby enhancing a more realistic representation of the ‘normal’ working environment through the enhanced immersive quality of the simulation. In addition, the integration of Artificial Intelligence (AI) offers the potential for autonomous communication and real-time feedback in simulations involving mannequins, whether high- or low-fidelity.
Objectives
The goal of this project is to develop a platform that enhances existing simulations (peer and mannequin based) through the integration of AR and AI. The improvements will target various parameters (such as situation awareness, learners satisfaction), with the ultimate aim of benefiting students in their learning process and, by extension, improving outcomes for the patients they will care for in the future.
Method
We will create clinical cases for medical and nursing simulations to enhance existing curricula and address institutional and end-user needs. These will include AR/AI-enhanced scenarios with visual and auditory cues. Afca will then develop a web-based interface to integrate these AR/AI features, and an application for extended reality devices. We will pilot-test the features with students and educators to evaluate usability, feasibility, clinical accuracy, and AI performance, using the feedback to guide refinements with afca. We will then implement the platform in simulation courses at Uni Bern and BFH, conducting a randomized controlled study to assess different parameters, e.g., situation awareness, usability, and satisfaction, across AR/AI-enhanced and standard simulations.
Planned Translation
In our project, the needs of clinical practice are identified and used as the basis for designing the platform as teaching material. We, as an interdisciplinary team from the Uni Bern and BFH will develop the content and the platform will be developed by experts from afca. Additionally, simulation instructors and students are actively involved in the process, making it a co-design approach. The integration takes place directly by embedding the simulations into existing courses at Uni Bern and BFH. There is rapid development in various educational technologies, e.g., extended reality and AI. Our goal is to engage with these developments in a way that provides added value for students as well as their learning experience / -outcomes. To achieve this, we aim to take the current limitations and unmet needs, initially from the two institutions (Uni Bern, BFH), and use them as the foundation for developing cases that meaningfully enhances simulation-based education. The underlying goal is to support students in their learning and better prepare them for real-life clinical situations, which could ultimately contribute to improved patient care in the long run. Furthermore, this approach could reduce the instructors’ workload or enable them to focus on other critical aspects of the training.
