Extended reality and computer-based simulation for teaching situational awareness in undergraduate health professions education: a scoping review

Abstract

Introduction

With the rapid evolution of healthcare environments, effective and accessible experiential learning has become an integral part of health education. Virtual reality (VR) poses the advantage of providing users with a virtual, immersive experience, to allow them to interact with elements of a simulated environment. This scoping review aims to evaluate the use of virtual reality (VR)-based simulation for the training of situational awareness (SA) and decision-making (DM) for undergraduate healthcare education.

Methods

A search was carried out across eight databases, namely: MEDLINE, Embase + Embase Classic, Scopus, Google Scholar, PubMed, CINAHL, ERIC, and PsycINFO. Studies evaluating the use of VR and its extended interfaces (i.e., augmented reality (AR) and mixed reality (MR)) for training SA and DM in undergraduate healthcare education were included.

Results

Of 3932 studies retrieved from the database search, 35 studies were included within the review. VR-based interventions were used across a range of healthcare profession trainees, including nursing, medical, paramedical, midwifery, and healthcare assistant students. Seventeen studies used screen-based VR interventions, and 16 studies used head-mounted devices (HMD). One study used both screen-based and HMD interventions and one further augmented reality. Twenty-nine studies assessed the role of the intervention in DM training, and 6 studies assessed its role in SA training. Eighteen studies used validated assessment tools, and 17 studies used educational theories to underpin their learning techniques.

Conclusions

The role of VR in training of SA and DM for healthcare professions has been well recognized, as demonstrated by the increasing number of studies. There is need for consensus of reporting for such studies to ensure a high-quality body of evidence with standardized outcome assessment.

Introduction

Continued growing demand for trained healthcare professionals alongside pressured health systems globally has made the need for further development of effective and accessible experiential educational opportunities apparent [1]. Evolving healthcare environments, societal expectations, and rapidly evolving medical research, along with a growing focus on holistic, patient-centred care, has posed significant challenges to traditional healthcare education curricula. The use of technology, including simulation-based methods, in response to these challenges has proved promising for improved knowledge acquisition, decision-making, skill coordination, and preparation for competent practice of healthcare professionals in training [2].

Simulation-based learning is defined as “an array of structured activities that represent actual or potential situations in education and practice” [3]. The concept of simulation can be perceived to be complex, with multiple definitions and categorizations. While simulation has been described as a “technique not a technology” [4], there are several diverse modes of delivery by which simulation-based methods can be classified: human patient actors, mannequin-based simulators, computer-based simulations, or virtual reality simulators (VR), for example [5]. Virtual simulation has been described as a broad term referring to computer-based programs which use varying types of technology to present scenarios; however, previous authors note that terminology in this field is often confusing [6, 7]. Computer-based simulations allow users to interact with patients usually via a screen-based interface [3, 5] which may include a virtual environment as well as virtual patient, whereas VR is a technology which provides the user with an illusion of immersion and presence [7] within a computer-generated environment while allowing them to interact with its elements. Over the past decade, further definitions in the spectrum of VR interfaces have been described with the latest version of the Healthcare Simulation Dictionary [3] including augmented reality (AR), a type of VR where real-world elements are supplemented with computer-generated factors such as audio, graphics used to enhance the learning process [3] and mixed reality (MR), and the merging of virtual and real-world elements along the “virtuality continuum” [3] to facilitate the real-time interaction of digital and physical elements [8]. Often, these technologies are grouped under an umbrella term as extended reality (XR). VR environments have often been described based on the technology being used, for example head-mounted display (HMD-VR) or other technologies such as computer-based virtual reality environments controlled by mouse, keyboard, voice, or haptic devices [8]. More recent descriptions of VR attempt to categorize VR into immersive VR where the user wears a display (such as a HMD) or non-immersive where a combination of screens surrounds the user presenting virtual information [3].

Simulation-based education has been widely used for both practical skills as and non-technical/behavioural skills (NTS) education encompassing the social and cognitive capabilities required within healthcare environments [9]. NTS include but are not limited to decision-making (DM) and situational awareness (SA), which will be the focus of our article. Clinical DM is a complex process which involves combining critical thinking, and clinical knowledge with awareness of the situation, and patient’s wishes [10]. According to the Endsley model, SA is a combination of the perception and comprehension of elements of a situation and surrounding environment, along with the ability to project the future status of the situation [11]. SA is known to contribute to the process of dynamic decision-making, making it a core training skill [11].

While skills of SA and DM are traditionally learnt via experience on clinical placements, research into the development of such skills in an undergraduate population is crucial due to their limited exposure to clinical practice. With increasing numbers of health professions students and the difficulties of assessment of these skills on clinical placements, it is important for clinical educators to adopt new technologies to address this gap [1, 12, 13].

Despite a multitude of review articles examining the use of simulation training for non-technical skills, there seems to be a lack of reviews focussing on the use of VR for the development of SA and DM skills [14,15,16]. Hence, we sought to examine the use of VR across the spectrum of undergraduate health professionals’ education specifically focussing on the key behavioural skills of SA and DM [17, 18]. In this review, we will use the term “behavioural skills” instead of the more traditional term “non-technical skills” as this better reflects modern understanding of both simulation and real-world practice terminology [17, 18]. Behavioural skills are defined as the cognitive, social, and personal skills that complement technical skills, contributing to safe and efficient task performance [19].

This scoping review aims to evaluate the breadth and depth of the evidence of the use of VR in the teaching and assessment of SA and DM in undergraduate healthcare professions education. Specific questions this review aims to answer are as follows:

• 1) In what contexts are VR, AR, and MR used within simulation training for SA and DM in healthcare professions education?

• 2) What outcome measures are used to examine the impacts of VR, AR, and MR use within simulation training for SA and DM?

• 3) What educational theories underpin VR, AR, and MR use within simulation training in healthcare professions education?

Methods

The study methodology, rationale, protocol, and search strategy have been previously described [20]. In summary, the scoping review utilized the framework described by Arksey and O’Malley [21]. This consisted of identification of relevant literature, selection of studies, mapping out the data, and synthesizing and reporting of results.

In this review, we used the definition of VR as described by Abbas et al.: “VR is a three-dimensional computer-generated simulated environment, which attempts to replicate real world or imaginary environments and interactions, thereby supporting work, education, recreation, and health” [22]. The principles of a three-dimensional computer-generated simulation of reality to support education were taken to include studies with all extended realities as well as patients represented in virtual environments where participants could interact with both the patient and wider environment including computer-based simulation.

Databases and search strategy

The scoping review is reported in accordance with the Preferred Reporting Items for Systematic review and Meta-Analysis Scoping Review extension (PRISMA-ScR) (Appendix 2) [23]. A systematic search was conducted to identify relevant studies looking into the use of VR, AR, MR, and computer-based simulation modalities for the training of SA and DM in health professions’ education on 9th July 2023 using Ovid MEDLINE, Embase, PubMed, Google Scholar, CINAHL, Scopus, PsycINFO, and ERIC databases. Relevant synonyms and Medical Subject Headings (MeSH) were used to carry out the search using the keywords as follows: (“Virtual Reality” OR “Mixed Reality” OR “Augmented Reality”) AND “Education” AND “Healthcare Professionals” AND “Non-technical skills”. A detailed description of the search strings used on the databases can be found in Appendix 3.

Eligibility criteria

Included studies were (1) peer-reviewed empirical primary research studies or published academic work (2) of quantitative or qualitative study designs, (3) looking into VR, AR, MR, or computer-based simulation techniques (4) for the training and assessment of DM and SA (5) in undergraduate healthcare students (6) in clinical workplace or education environments. Studies were excluded from review if as follows: (1) There was no available English language translated text, (2) review articles, and (3) studies which did not include extractable information for outcomes for SA or DM. The reference list for all excluded review articles was screened for texts that met the inclusion criteria.

Study selection

The results from the database search were screened simultaneously by three independent reviewers (each result being screened by two reviewers) in two stages: (1) title and abstract screening against pre-defined inclusion and exclusion criteria, followed by (2) full-text screening. Reviewers were blinded to each other’s decisions during the screening process. Reviewers were unblinded at the end of each stage, and discrepancies were resolved in the presence of a third adjudicator through discussion and to support calibration for future rounds.

Data charting

A data extraction form was designed (Appendix 3) to ensure standardized data extraction. The data was extracted under the following headings: Study characteristics (study title, author, journal, year of publication, country of origin, type of study), Participant characteristics (total population selected, number of controls, number of cases), and Outcomes (outcomes measured, mode of measurement, conclusion(s) drawn, educational theories outlined). Relevant data was extracted from all included studies by one reviewer. Where appropriate, data has been displayed in the form of tables supplemented with narrative review.

Results

Selection process and study characteristics

The process of selection of the included studies has been detailed in Fig. 1. Of 3932 articles retrieved from the database search, 3423 papers were deemed eligible for screening by title and abstract after de-duplication. Two-hundred eighty-one articles were screened by full text for inclusion within the study, of which 33 were included [24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56]. While reviews were excluded from the study, the reference lists of reviews were screened for relevant texts. Two studies were selected for inclusion within the study from reference list screening, resulting in a total of 35 papers deemed eligible for inclusion [57, 58]. Of the 35 studies included for review, 9 studies were from the United States of America (USA) [24, 29, 31, 32, 34, 36, 38, 41, 45], 6 were from South Korea [39, 42, 43, 49, 50, 56], 3 were from the United Kingdom (UK) [47, 57, 58], 3 were from France [26, 27, 46], 2 were from Japan [40, 54], 2 from Australia [48, 51], 2 from Canada [25, 55], 2 from China [33, 35], and 1 each from Switzerland [28], Taiwan [30], Turkey [37], Germany [44], Ireland [52], and Finland [53]. All study characteristics are detailed in Table 1.

Fig. 1

PRISMA flow diagram summary of search and screening process

Table 1 Study characteristics

Use of VR interventions within simulation training for SA and DM

Seventeen studies used computer-based VR interventions [25, 26, 28, 29, 34, 35, 37, 41, 45,46,47, 49,50,51, 54, 55, 58], while 16 studies used HMD-VR [24, 27, 31,32,33, 36, 38,39,40, 42,43,44, 48, 52, 56, 57]. The computer-based interventions were predominantly screen based; however, three studies employed tablet- or mobile-based VR [35, 37, 45]. Sara et al. [53] used both computer-based and HMD VR interventions in their study [53]. Chen and Liou classified their application viewed from a smartphone as AR [30]. Nine studies compared VR-based interventions to traditional teaching or mannequin-based simulation methods [26, 35, 37, 39, 42, 43, 47, 49, 56] (Table 2) .

Table 2 Intervention

Participants and outcomes measured

Twenty-four articles looked at the role of VR for behavioural skills in nursing education [24,25,26,27, 29,30,31, 34, 35, 37,38,39, 41,42,43, 49,50,51,52,53, 55,56,57,58]. Ten articles looked at the role of the intervention for medical students [24, 28, 32, 33, 36, 40, 44, 45, 47, 54], 2 for paramedical education [40, 48], 1 for midwifery education [46], and 1 for healthcare assistant education [55]. Three papers studied the intervention in multiple groups of undergraduate healthcare professionals [24, 40, 55]. Twenty-nine articles studied the role of the intervention in training clinical judgement and related skills (i.e., problem-solving, decision-making, clinical reasoning) [25, 26, 30,31,32,33,34,35,36,37,38,39,40,41,42,43,44, 46,47,48,49,50,51,52,53,54, 56,57,58], and 6 articles looked at SA [24, 27, 28, 45, 46, 55]. One article looked at the effect of asynchronous debriefing on the virtual simulation experience [29]. The results and intervention-outcome effects of each study can be found in Table 3.

Table 3 Participants and outcomes

Outcome measures used to assess impact of intervention on SA and DM

Eighteen studies used a validated method of assessment for outcome measurement [25, 27, 29, 31, 32, 34, 37,38,39, 41, 43, 46, 48,49,50, 53, 56, 57]. Of these, four studies used the Nursing Anxiety and Self Confidence with Clinical Decision-Making Scale (NASC-CDM) [25, 37, 39, 57], and three studies used the Lasater Clinical Judgement Rubric (LCJR) [34, 38, 50]. Details of other tools and methods of assessment can be found in Table 4.. The remaining 17 studies used a combination of project-developed Likert scale questionnaires, semi-structured interviews, one-to-one interviews, and focus groups to assess the outcomes [24, 26, 28, 30, 33, 35, 36, 40, 42, 44, 45, 47, 51, 52, 54, 55, 58].

Table 4 Educational theories and methods of assessment

Educational theories underpinning VR simulation training for SA and DM

Seventeen articles described educational learning theories [59] which would guide the technique of learning with the intervention [25,26,27, 29, 37, 40,41,42,43,44, 50, 51, 53, 55,56,57, 60]. Eight papers used experiential learning theory [25, 26, 29, 37, 40, 42, 44, 50]; two papers used constructivist learning theory [41, 51], two papers used gamification [64] [53, 57] — a theory used to describe the use of game design elements in non-game contexts, rooted in behaviourism (see Wang et al. for further description [61]); two papers used the 3D model of debriefing [27, 29]; two papers used Jeffries Simulation Theory [55, 60]; one paper used the 5E circular learning model [43]; and one paper used Keller’s attention, relevance, confidence, and satisfaction (ARCS) model [56]. Eighteen articles did not report the use of educational theories to guide the learning techniques using the intervention [24, 27, 28, 30,31,32,33,34,35,36, 38, 39, 45,46,47,48,49, 52,53,54,55, 58]. Details of the learning theories used in each study can be found in Table 4. (for further information on learning theories, see Taylor and Hamdy [59]).

Discussion

To the best of our knowledge, this is the first scoping review that provides a comprehensive synthesis of evidence surrounding the use of VR for the training of SA and DM skills for healthcare education. VR is a revolutionary technology that has the potential to remodel the landscape of healthcare education. As a result, it has attracted significant academic interest, with educators and academics exploring optimal contexts and methods for the use of VR in healthcare education. VR poses the advantages of accessible training “on demand” either in its immersive or non-immersive forms [5, 62].

VR-based terminology

Within this review, VR was defined as technology that provides the user with a three-dimensional (3D) digital environment while allowing them to interact with its elements [22]. VR can be broadly categorized as immersive (consisting of HMDs, allowing complete involvement within the task environment) and non-immersive (consisting of screen-based displays which can be manually navigated by the user) [63, 64].

While studies were excluded if they had interventions that were not considered 3D or interactive, included studies were found to have high levels of disparity within their definitions of VR interventions. When categorizing studies based on use of immersive and non-immersive interventions, it was found that 16 studies employed the use of HMD-based VR, while the other 17 used screen-based non-immersive modalities. This disparity in terms of definitions of what constitutes VR has been described previously with Kardong-Edgren et al., calling for unifying definitions of VR, suggesting elements such as level of immersion considering characteristics of presence, based on the senses of the learners which various technologies are designed to deceive, are reported [7].

A recent review found that subjectively, learners report enjoying training with HMD, and advances in technology now result in fewer adverse effects such as “cybersickness” [65]. It is likely that the higher numbers of non-immersive VR studies in our review relate to the fact that SA and DM can be represented easily using non-immersive technologies, requiring less specialized equipment and are cheaper than immersive-VR options [66]. Cant and Ryan’s recent mapping review describes the range of virtual environments and technologies available for use within nursing education, either paid-for subscription options or open-access options using various technologies [67].

Winn [68] described learning in artificial environments to be influenced by both the adaptation of the learner to the environment, and the environment to the learner [68], highlighting the role of immersion in the learning experience. It is imperative that the right form of simulator is utilized to match the desired learning outcomes; however, it is noted that this is a complex and challenging process. Goodwin and Nestel describe the affordances and limitations of different simulation modalities, in particular associated cost benefits and flexibility/accessibility associated with computer-based VR simulations and the potential of improved psychological safety with solo use HMD-VR simulations [69]. This highlights the need for educators to consider the advantages and disadvantages posed by the level of immersion within given contexts, particularly with the increasing availability of XR/AR technologies where the user can interact with parts of a real-world environment with virtual overlays. Shin et al. recently conducted an integrative review into the use of virtual simulation in nursing which described virtual-specific characteristics that educators need to consider alongside general simulation characteristics when designing VR educational strategies [70]. These included instructor competency, mode of representation, participant role, interaction, type of platform, theoretical framework, and virtual ethics [70]. Without having clear descriptions and definitions within the included studies, it is challenging to draw firm robust conclusions. Our findings support those of Nuha et al. who reviewed the distance simulation literature which found an absence of standardization of terminology within that field which made the review challenging [71].

Outcome measures to assess SA and DM within VR simulation

It was encouraging to see the use of validated outcome measures for SA and DM across 18 of the 35 included studies. The Kirkpatrick training model is a standardized framework to objectively evaluate the outcomes of training interventions at different levels [72]. This considers four levels of outcome evaluation, namely reaction of the learner, learning or skills acquired, behaviour or performance measures, and organizational outcomes [72]. While the included studies reported outcomes for learner reaction and acquired skills, there were no reports of behaviour or organizational outcomes.

Evaluating higher levels of the Kirkpatrick model in the context of SA and DM would pose the challenges of (1) reliable outcome measures to assess the changes in learner behaviour after SA and DM skill acquisition and (2) evaluating the effect of such skills on patient care at an organizational level. However, Kardong-Edgren [72] highlighted the importance and need for the evaluation of such long-term outcomes in simulation research, to better justify the introduction of such modalities into healthcare education [73]. Our findings align with a 2024 umbrella review of the in-person simulation literature where 83% of studies focussed on level 2 outcomes [74]. Our findings, which is most of the studies are at the lower end of the Kirkpatrick framework, are not surprising given the relative infancy of the introduction of these technologies within this particular area of healthcare education and identify future research opportunities focussing on how this can be transferred into higher level outcomes [71]. A recent meta-analysis of VR in nursing education suggested that VR was more effective than control conditions in knowledge improvement however found no difference when assessing skills, satisfaction, or confidence [75].

Educational theories underpinning the use of VR in healthcare education

Only 13 of the included studies reference specific learning theories; the majority is describing experiential learning while the others describing constructivist, gamification, Keller’s ARCS model, and the 5E circular learning model (Table 4.). This meant 22 of the included studies did not describe the utilization of learning theories when delivering the VR simulation training intervention.

All educational interventions are complex interventions [76], and therefore, a key aspect of their design process should be the use of an underpinning theoretical framework, meaning at the design stage there should be theoretically driven hypothesis of why the intervention should work in a particular context; this understanding should be evidenced in any publication describing the use or development of a teaching intervention. To ensure instructional design for novel educational tools such as VR, the underpinning learning theories should be well described to better suit the desired learning outcomes of the simulation exercise [77]. In accordance with instructional design models, it is imperative that methods of instruction are clearly researched and outlined during the development phase of an intervention to ensure effective design and implementation [78]. Application of education theories not only enhance the efficacy of the training but also provide a framework for evaluation to enhance understanding of how such interventions work and how they could be revised to improve their efficacy further [59].

Strengths and limitations

Scoping reviews are a useful tool to determine the scope and coverage of published literature on a given topic, particularly in emerging disciplines where the number of studies that are available restricts the possibility of systematic reviews [79]. A significant strength of our review is its adherence to the robust framework described by Arksey and O’Malley and reporting as per the PRISMA-ScR guidelines [21, 23] identifying 35 studies for inclusion.

Our review is not without limitations. Our review was limited to English language articles, and our search therefore may have missed articles from other languages, introducing potential bias into the search results. A significant limitation to this review was the variability in definitions of our intervention of interest, VR, within included studies. Confusion regarding terminology within this field has been noted previously, with the understanding of the components of VR being variable [22]; this may have impacted the ability to identify relevant studies via our search terms. However, to standardize the intervention for the purpose of this review, we used the definition by Abbas et al., to include studies assessing SA and DM training using a wide range of “virtual” interventions encompassed by VR [22]. The broad definition of the virtual environment in use allowed us to evaluate the various virtual simulation techniques currently used to train SA and DM skills.

Additionally, there was a lack of standardization of the definitions of SA and DM within the included studies, with some studies using “clinical judgement” and “decision-making” interchangeably. While the reviewers attempted to be systematic in the process, this may have led to the potential exclusion of some relevant research. Another potential limitation was the extraction of data by a single researcher. We also note that our inclusion criteria focussed on undergraduate programmes only and recognize that many healthcare professions programmes across the world are classified as graduate education programmes limiting the generalizability of this review in these settings. Scoping reviews are designed to map the territory on a given subject, and as such do not routinely assess the risk of bias or methodological rigour of the included studies which may limit the applicability of the results [80].

Future work

Our scoping review was able to identify multiple future research priorities within the field of simulation-based education research. Firstly, the lack of standardization of VR-based intervention terminology and outcomes such as SA and DM call for consensus on standard definitions and reporting measures for educational research in this field. This would allow for more homogenous reporting of simulation-based studies.

Secondly, we recommend additional research comparing immersive and non-immersive VR interventions for training of behavioural skills, to ascertain the most optimal approach. Lastly, we recommend research to evaluate long-term impacts of VR-based training on learner behaviour and patient experience to strengthen the evidence surrounding VR-based modalities in the training of behavioural skills.

Conclusion

This scoping review has mapped how VR is being used across the spectrum of health professionals’ education to teach and assess SA + DM. We recommend the development of standardized reporting guidelines for VR studies within healthcare education to ensure increased quality of studies within this area and for clear descriptions of level of immersion and presence to be described. We would encourage authors to adhere to the definitions and terminology as per the latest edition of the Healthcare Simulation Dictionary to enhance the standardization of reporting allowing readers to fully understand what technologies were used and how they are implemented [3]. Included studies reported a variety of outcome measures ranging from validated SA + DM scales to Likert scale questionnaires and interviews. Around half of included studies reported clear educational theories when describing their VR interventions. We would encourage future VR researchers to describe the educational theories which apply when implementing VR interventions.