Metaverse, a 3-dimensional (3D) virtual environment in which participants interact as avatars, has been useful in health professions education, especially during the COVID-19 pandemic.1,2 Originally used for entertainment and gaming purposes, the concept of the metaverse as a nexus of the virtual and physical world3 has evolved to enable immersive virtual experiences that can seem real to participants.4 Metaverse describes an environment enabled by technology in which participants can have interactive experiences,5 which employs immersive virtual reality (VR) (an interaction with a 3D computer-generated simulation using a headset device),6,7 augmented reality (AR) (the overlay of computer-generated elements on top of the real physical environment),6,8,9 virtual worlds (VW) (controlled web-based environments that utilize avatars to communicate via speech or text),7 mixed reality (MR) (the merging of real physical and virtual environments),10 and extended reality (XR) (the spectrum or combination of VR, AR, MR, and VW).5,7 A related concept is simulation: scenarios that utilize manikins, role-play, web-based applications, or virtual environments to reproduce clinical environments.5 By incorporating the latest technologies into their curriculum, nurse educators can prepare future nursing professionals to excel in complex health care environments.11
Research indicates that experiential learning is more effective than traditional forms of education: a lecture-only format has been shown to produce a 5% information retention rate over a 24-hour period compared with the use of demonstration (30%), discussion groups (50%), and practice activities (75%).7 Well-designed and implemented metaverse technology can provide realistic experiences to students12 while advancing knowledge, cognitive skills, engagement, and positive attitudes.9,11 Topics such as human anatomy,13 palliative care,14 mental health care through virtual standardized patients,15 interprofessional collaboration,16 clinical knowledge delivery, and skills training,17–19 have been successfully presented to health professions students through this technology.20 The metaverse is a viable pedagogy that can provide synergistic opportunities for nursing students and faculty, and its growing use is expected to profoundly impact nursing education.
Review Aims
Given the range of metaverse technologies, we conducted an umbrella review, a systematic review of reviews,21 to provide integrated evidence of metaverse use and identify knowledge gaps pertaining to nursing education. Our questions were as follows: (1) What are the characteristics of studies on using metaverse for nursing education? (2) What are the interventions and outcomes of studies that have systematically reviewed the use of metaverse in nursing education? (3) What are the trends in the use of metaverse as a pedagogical tool in nursing education?
Methods
The search process followed the phases outlined by the Joanna Briggs Institute (JBI) Umbrella Review21 and was reported following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 statement.22 An a priori protocol was developed and registered on Open Science Framework.23
Search Strategy
A preliminary search of PROSPERO, MEDLINE, the Cochrane Database of Systematic Reviews, and the JBI Evidence Synthesis found no existing or underway umbrella reviews on the topic. On November 16, 2021, MEDLINE (via PubMed), EMBASE (via Elsevier), CINAHL (via EBSCO), Web of Science (via Clarivate), and Education Full Text (via EBSCO) were searched. Complete reproducible search strategies, including language limits and search filters for all databases, are detailed in Supplemental Digital Content, Appendix I (available at: https://links.lww.com/NE/B224, Search strategies). From April 7 to 11, 2022, forward and backward citation tracking of included studies was undertaken manually using Web of Science or Scopus to identify additional articles, which were added for full-text review.
Eligibility Criteria
The following PICOTTS (Population, Intervention, Comparison, Outcome, Type of Study, Time frame, and Setting) framework24 was used to identify inclusion and exclusion criteria and select literature. Included were literature reviews in which (1) participants were nursing students or RNs in a student role, (2) metaverse (eg, VR, AR, MR, VW, XR) in nursing education was reviewed, and (3) learning outcomes were provided. Excluded were studies in which (1) less than 50% of participants were students or RNs in a student role; (2) the focus was simulation-based education, use of manikins (low/high fidelity), telemedicine, mobile health OR mHealth, m-learning, or e-learning; or (3) reporting insufficient data to determine outcomes of the intervention (eg, conference abstracts). Also excluded were literature reviews lacking a transparent systematic method or that incorporated theoretical studies or text, and articles that reviewed legal decisions, clinical records, technical documentation, or opinion pieces. No publication date range or geographic limits were applied. Detailed eligibility criteria are presented in Supplemental Digital Content, Table 1 (available at: https://links.lww.com/NE/B225, Eligibility criteria).
Selection of Evidence Sources
After the search, all identified articles were uploaded into Covidence25 for management and de-duplication, then reviewed independently by 2 researchers, and excluded if ineligible. Articles were excluded if they did not clearly meet the data elements based on the title and abstract and full-text review. Conflicts between the 2 reviewers were resolved through discussion at each stage of the selection process.
Data Extraction and Synthesis
One reviewer completed the extraction of predetermined criteria, and a second reviewer independently verified the extracted elements. Disagreements between the 2 reviewers on the extracted elements were resolved through discussion or by a third reviewer. The extracted data were summarized in a tabular form and a narrative summary. Metaverse interventions were assessed on the basis of the TPACK (Technological Pedagogical and Content Knowledge) model,26 and outcomes were assessed according to Kirkpatrick's 4 levels of evaluation (Reaction, Learning, Behavior, and Results).27 Per the recommendation of JBI, an extraction tool was created and piloted prior to implementation. The team refined the research questions, revised the data extraction tool, and updated the critical appraisal tool to ensure consistency.21 A comprehensive explanation of these updates is detailed explicitly in the updated study protocol.23
Quality Assessment
The JBI Critical Appraisal Checklist for Systematic Review assisted in assessing the trustworthiness, relevance, and results of included reviews.21 This tool consists of 11 questions that should be answered as “yes,” “no,” “unclear,” or “not applicable.”21 Two researchers independently completed the quality assessment and compared responses until a consensus was reached. Critical appraisal was not used to exclude any extracted reviews.
Results
A total of 15 reviews, including one from the reference lists of the 14 reviews, were selected for this study. A flowchart of the study selection process is presented in Supplemental Digital Content, Figure 1 (available at: https://links.lww.com/NE/B226, Flowchart of the study selection process).
Critical Appraisal
Overall, the included reviews contained elements indicating a range of quality. Because there is a lack of summary scores, results were viewed by study and by question. Individual review scores ranged from 3 to 10 for “yes,” 1 to 8 for “no,” 0 to 3 for “unclear,” and 0 to 4 for “not applicable.” Despite some weaknesses, the included reviews primarily synthesized and reported data in alignment with the guidelines for systematic reviews. A complete critical appraisal of the 15 reviews is presented in Supplemental Digital Content, Table 2 (available at: https://links.lww.com/NE/B227, Critical appraisal of included reviews).
Sample Characteristics
Included reviews were conducted between 2013 and 2021. Twenty-eight databases were searched, with the most searched databases being CINAHL (n = 15), PubMed (n = 12), PsycINFO (n = 7), Cochrane library (n = 6), and EMBASE (n = 6). The top 10 most searched databases are shown in Supplemental Digital Content, Figure 2 (available at: https://links.lww.com/NE/B228, The top 10 most searched databases). Review types included systematic review (n = 6),28–33 scoping review (n = 3),34–36 integrative review (n = 3),20,37,38 systematic review with meta-analysis/regression (n = 2),39,40 and systematic mapping review (n = 1).41 Examples of critical appraisal instruments used in the included reviews were the Mixed Methods Appraisal Tool (MMAT),37,38 a tool developed by Hawker et al,35,36 and the Medical Education Research Study Quality Instrument (MERSQI).30 The methods of evidence integration included thematic analysis (n = 4), narrative synthesis (n = 4), meta-analysis (n = 2), and content analysis and conceptual clustering (n = 1). Several reviews did not specify the exact method of evidence synthesis (n = 4).
Reviews on the current use of metaverse reported its advantages (eg, safety, cost-effectiveness)35,37 and challenges (eg, cybersickness, technology malfunction).28,40 Commonly reported learner outcomes were increased in knowledge, self-confidence, learner engagement, satisfaction, and performance; however, some reviews reported mixed findings in certain outcomes across their included studies.31,34,39 Detailed sample characteristics of each study are available in Supplemental Digital Content, Table 3 (available at: https://links.lww.com/NE/B229, Study characteristics).
Metaverse Intervention and Its Outcomes
Among 15 reviews, metaverse interventions included VR (n = 7), VW (n = 4), and MR (n = 1). Two reviews used VR and AR, and 1 review included VR and VW. Interventions were primarily conducted in clinical laboratories or classrooms. Various strategies were categorized according to use of technological, pedagogical, and content knowledge, or TPACK, which is a technology integration framework.26 We found that technology pedagogical knowledge (TPK), such as VSim, Second Life, haptic devices, or 3D environments were explored. As for pedagogical content knowledge (PCK), we noticed that simulation was most widely used, followed by gaming and multimedia use. Finally, technology content knowledge (TCK) included subject matter, such as fundamental nursing knowledge, anatomy, catheter insertion, and critical thinking skills.
According to Kirkpatrick and Kirkpatrick,27 most interventions evaluated student reaction, level 1 (eg, satisfaction, attitude, anxiety) or level 2 (eg, knowledge, skills, critical thinking). We found that a few reviews reported behavioral changes, level 3 such as performance of a community home visit or emergency crisis management; however, none reported organizational performance outcomes, level 4. A summary of the metaverse interventions and outcomes is presented in Supplemental Digital Content, Table 4 (available at: https://links.lww.com/NE/B230, Metaverse interventions and outcomes).
Trends of Using Metaverse as a Pedagogical Tool
The included reviews (N = 15) were mostly conducted in developed countries, such as the United States (n = 3), Australia (n = 3), South Korea (n = 2), and Singapore (n = 2). All reviews were conducted between 2013 and 2021, and most within the past 3 years (n = 11; 73%). Search periods of included reviews ranged from inception up to April 2021. The total number of studies across all reviews was 319, with 113 duplications. Of the included studies within each review, the publication dates varied from 2001 to 2020, with most between 2015 and 2018. The depiction of publication trends is presented in Supplemental Digital Content, Figure 3 (available at: https://links.lww.com/NE/B231, Publication trends by country and year).
The included reviews reported that use of metaverse as a pedagogical tool is of global interest; however, the literature is in the early stages, and there is limited evidence on ethical considerations (eg, security breaches). Most studies were from the United States and Western countries, indicating a geographical disparity. Although most reviews consider metaverse to be a promising pedagogical tool, further exploration of nurse educators' roles and metaverse as pedagogy is warranted.30,33,38
Data Summary
Because of the significant heterogeneity of the included reviews, an overall quality rating for practice recommendations was not applied to this body of evidence at the outcome level. In addition, there was mixed evidence reliability among the reviews: for example, Woon et al40 noted a low overall GRADE score, whereas Choi et al30 reported a moderate to high MERSQI score. Although a few reviews did include statistical information on analysis, the outcome measures were not consistent, limiting our ability to make practice recommendations. In addition, the heterogeneity between studies was attributed to varying levels of evidence, participant characteristics, and evaluation tools. The outcomes of interest were extracted and organized narratively at the review level with the aid of tables and figures.
Discussion
This umbrella review examined the literature to elucidate the use of metaverse in nursing education. In accordance with similar reviews,11,42,43 findings demonstrated that the metaverse engages students to learn multifaceted content. Metaverse research received attention during the COVID-19 pandemic42–44; yet, 7 of the included reviews were conducted before the pandemic. Although most reviews recognized metaverse as a promising pedagogical tool, several reviews31,34,35,39 reported mixed results in certain outcomes between metaverse education and traditional approaches, a not uncommon finding as similar studies comparing metaverse interventions with traditional methods reported mixed effectiveness.9,45–47
As most of the metaverse interventions focused on using VR to teach basic clinical nursing skills, and limited reviews evaluated the impact of metaverse on critical thinking skills,30,36,41 it is difficult to ascertain whether advanced nursing skills, such as problem-solving, can be successfully taught using metaverse technologies. This review questions the transferability of VR to clinical practice, which is consistent with the literature11 and possibly due to the immaturity of VR technology and overall lack of evidence.39 This umbrella review also found geographical disparities involving metaverse technologies, as did Tlili et al.43 Similar to Brown and colleagues,42 this review evaluated most intervention outcomes as levels 1 and 2 with the Kirkpatrick model, whereas level 3 outcomes were rare. None of the included reviews were scored as a level 4 outcome27; thus, it was difficult to determine whether the learning outcomes were attributable to the training or institutional performance over time. Concerning facilitators and barriers to metaverse use, the former included instructor ability and technical competency, as well as participant engagement,29,34,38 whereas the latter included lack of student comfort, technological limitations, and inconsistency in implementation,20,30,31 which correlate with challenges to VW use reported by Hack.12
Implications for Teaching and Learning
Advantages to the use of metaverse in nursing education include safety, time, cost-effectiveness, universality, and repeatability without risk of patient harm.30,35,37 Challenges include technical requirements, complex preparation, participant fear, limited reality, and skill retention over time.20,28,35,37,38 As traditional approaches (ie, manikin-based simulation, role-play) have the advantage of realism and familiarity, which may not be experienced in virtual environments,28 educators may consider using a blended learning approach that combines traditional and virtual methods.28,46 Several reviews have reported mixed skill retention over time, so faculty should consider offering refresher courses to promote retention30,31,35 and scheduling preparation time to address student anxiety and promote comfort.30,31 Few reviews reported misuse of the metaverse; those that did refer to improper use of online chat software via Second Life20,29 and blurred boundaries between education and entertainment.38
Limitations of the Study
This review had some predictable limitations.21 Because we synthesized findings from reviews, we cannot be sure of the biases that might have arisen from authors' selection of appraisal or extraction approaches. Relevant reviews may also have been missed during the search process. Because of limitations of available resources, we focused on reviews published in the English language. Gray literature was excluded because of the focus on empirical evidence, which may have caused publication and selection biases. Rapid reviews were included in the search, introducing data from simplified systematic strategies. In addition, the lack of an overall risk of bias score can be considered a limitation as well as the research team not following up with authors regarding insufficient data. Therefore, the interpretation of the findings should be considered accordingly.
Recommendations for Practice and Future Research
The potential benefits of metaverse as pedagogy are evident, but substantial research is needed to create synergy between nursing education and technology.38 To incorporate novel teaching methods into nursing curricula34 and use them safely and effectively, nurse educators need ongoing training and technical support to understand potential risks, adverse outcomes, limitations,43 and ethical aspects of virtual education.38 Future studies on this topic should explore security concerns (eg, privacy and confidentiality), ethical issues (eg, cybercivility),43,48 and inclusive strategies for diverse populations, such as students with disabilities or those impacted by the digital divide.43 Research supports that the COVID-19 pandemic has exacerbated inequalities affecting students without access to campus resources such as high-speed internet and computers, an issue warranting consideration and concern when employing metaverse interventions. International collaborations using metaverse technologies to address geographical disparities in nursing education should follow.
Conclusion
The aim of this umbrella review was to explore and synthesize published reviews on the use of metaverse in nursing education. Our findings support that metaverse is an evolving and versatile pedagogical tool with the potential to promote learning outcomes in nursing education, thus offering exciting opportunities and considerations for faculty and administrators. Future research should examine the application of metaverse to teach advanced nursing concepts and the impact of metaverse interventions longitudinally and globally.
References
1. Maier M, Ebrahimzadeh A, Rostami S, Beniiche A. The internet of no things: making the internet disappear and “see the invisible.” IEEE Commun Mag. 2020;58(11):76–82. doi:10.1109/MCOM.001.2000098
2. Go S, Jeong H, Kim J, Sin Y. Concept and developmental direction of the
metaverse. 2021. Accessed July 24, 2022.
http://www.koreascience.kr/article/JAKO202122450520317.pdf
3.
Metaverse roadmap: pathways to the 3D web. 2009. Accessed June 29, 2022.
https://metaverseroadmap.org
4. Hui A, Wagner C. Creative and Collaborative Learning Through Immersion. Springer; 2021.
5. Dodds HE. Immersive learning environments: designing XR into higher education. Pract Guide Instr Des High Educ. 2021. Accessed September 2, 2022.
https://edtechbooks.org/id_highered/immersive_learning_e
6. Arena F, Collotta M, Pau G, Termine F. An overview of
augmented reality. Computers. 2022;11(2):28. doi:10.3390/computers11020028
7. McGonigle D, Mastrian K. Nursing Informatics and the Foundation of Knowledge. 5th ed. Jones & Bartlett; 2021.
8. Foronda CL, Alfes CM, Dev P, et al. Virtually nursing: emerging technologies in
nursing education. Nurse Educ. 2017;42(1):14–17. doi:10.1097/NNE.0000000000000295
9. Nesenbergs K, Abolins V, Ormanis J, Mednis A. Use of augmented and
virtual reality in remote higher education: a systematic umbrella review. Educ Sci. 2020;11(1):8. doi:10.3390/educsci11010008
10. Antin D. The technology of the
metaverse, it's not just VR. 2020. Accessed July 1, 2022.
https://www.washingtonpost.com/technology/2021/08/30/what-is-the-metaverse
11. Kyaw BM, Saxena N, Posadzki P, et al.
Virtual reality for health professions education: systematic review and meta-analysis by the digital health education collaboration. J Med Internet Res. 2019;21(1):e12959. doi:10.2196/12959
12. Hack CJ. The benefits and barriers of using virtual worlds to engage healthcare professionals on distance learning programmes. Interact Learn Environ. 2016;24(8):1836–1849. doi:10.1080/10494820.2015.1057743
13. Richardson A, Hazzard M, Challman SD, Morgenstein AM, Brueckner JK. A “Second Life” for gross anatomy: applications for multiuser virtual environments in teaching the anatomical sciences. Anat Sci Educ. 2011;4(1):39–43. doi:10.1002/ase.195
14. Lee AL, DeBest M, Koeniger-Donohue R, Strowman SR, Mitchell SE. The feasibility and acceptability of using virtual world technology for interprofessional education in palliative care: a mixed methods study. J Interprof Care. 2020;34(4):461–471. doi:10.1080/13561820.2019.1643832
15. Reger GM, Norr AM, Gramlich MA, Buchman JM. Virtual standardized patients for mental health education. Curr Psychiatry Rep. 2021;23(9):57. doi:10.1007/s11920-021-01273-5
16. Liaw SY, Ooi SL, Mildon R, Ang ENK, Lau TC, Chua WL. Translation of an evidence-based
virtual reality simulation-based interprofessional education into health education curriculums: an implementation science method. Nurse Educ Today. 2022;110:105262. doi:10.1016/j.nedt.2021.105262
17. Khan R, Plahouras J, Johnston BC, Scaffidi MA, Grover SC, Walsh CM.
Virtual reality simulation training for health professions trainees in gastrointestinal endoscopy. Cochrane Database Syst Rev. 2018;8(8):CD008237. doi:10.1002/14651858.CD008237.pub3
18. Lorenzo-Alvarez R, Rudolphi-Solero T, Ruiz-Gomez MJ, Sendra-Portero F. Game-based learning in virtual worlds: a multiuser online game for medical undergraduate radiology education within Second Life. Anat Sci Educ. 2020;13(5):602–617. doi:10.1002/ase.1927
19. Wiecha J, Heyden R, Sternthal E, Merialdi M. Learning in a virtual world: experience with using Second Life for medical education. J Med Internet Res. 2010;12(1):e1. doi:10.2196/jmir.1337
20. De Gagne JC, Oh J, Kang J, Vorderstrasse AA, Johnson CM. Virtual worlds in
nursing education: a synthesis of the literature. J Nurs Educ. 2013;52(7):391–396. doi:10.3928/01484834-20130610-03
21. Aromataris E, Fernandez R, Godfrey C, Holly C, Khalil H, Tungpunkom P. Umbrella reviews. In: Aromataris E, Munn Z, eds. JBI Manual for Evidence Synthesis. JBI; 2020:chap 10. doi:10.46658/JBIMES-20-11
22. Page MJ, Moher D, Bossuyt PM, et al. PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews. BMJ. 2021;372:n160. doi:10.1136/bmj.n160
23. De Gagne JC, Rushton SR, Randall PS. Umbrella review of
metaverse used to teach nursing students. Published online December 23, 2021. doi:10.17605/OSF.IO/2RU6G
24. Haynes RB, Sackett DL, Richardson WS, Rosenberg W, Langley GR. Evidence-Based Medicine: How to Practice & Teach EBM. Vol 157. CMA Impact Inc; 1997.
25. Covidence Systematic Review Software. Veritas Health Innovation. Accessed April 8, 2022.
https://www.covidence.org
26. Koehler M, Mishra P, Cain W. What is technological pedagogical content knowledge (TPACK)? J Educ. 2013;193(3):13–19. doi:10.1177/002205741319300303
27. Kirkpatrick JD, Kirkpatrick WK. Kirkpatrick's Four Levels of Training Evaluation. ATD Press; 2016.
28. Shorey S, Ng ED. The use of
virtual reality simulation among nursing students and registered nurses: a systematic review. Nurse Educ Today. 2021;98:104662. doi:10.1016/j.nedt.2020.104662
29. Irwin P, Coutts R. A systematic review of the experience of using Second Life in the education of undergraduate nurses. J Nurs Educ. 2015;54(10):572–577. doi:10.3928/01484834-20150916-05
30. Choi J, Thompson CE, Choi J, Waddill CB, Choi S. Effectiveness of immersive
virtual reality in
nursing education: systematic review. Nurse Educ. 2022;47(3):E57–E61. doi:10.1097/NNE.0000000000001117
31. Foronda CL, Fernandez-Burgos M, Nadeau C, Kelley CN, Henry MN. Virtual
simulation in
nursing education: a systematic review spanning 1996 to 2018. Simul Healthc. 2020;15(1):46–54. doi:10.1097/SIH.0000000000000411
32. Jallad ST, Isik B. The effectiveness of
virtual reality simulation as learning strategy in the acquisition of medical skills in
nursing education: a systematic review. Ir J Med Sci. 2022;191(3):1407–1426. doi:10.1007/s11845-021-02695-z
33. Rourke S. How does
virtual reality simulation compare to simulated practice in the acquisition of clinical psychomotor skills for pre-registration student nurses? A systematic review. Int J Nurs Stud. 2020;102:103466. doi:10.1016/j.ijnurstu.2019.103466
34. Alhonkoski M, Salminen L, Pakarinen A, Veermans M. 3D technology to support teaching and learning in health care education—a scoping review. Int J Educ Res. 2021;105:101699. doi:10.1016/j.ijer.2020.101699
35. Fealy S, Jones D, Hutton A, et al. The integration of immersive
virtual reality in tertiary nursing and midwifery education: a scoping review. Nurse Educ Today. 2019;79:14–19. doi:10.1016/j.nedt.2019.05.002
36. Kim KJ, Choi MJ, Kim KJ. Effects of nursing
simulation using mixed reality: a scoping review. Healthcare (Basel). 2021;9(8):947. doi:10.3390/healthcare9080947
37. Coyne E, Calleja P, Forster E, Lin F. A review of virtual-
simulation for assessing healthcare students' clinical competency. Nurse Educ Today. 2021;96:104623. doi:10.1016/j.nedt.2020.104623
38. Shin H, Rim D, Kim H, Park S, Shon S. Educational characteristics of virtual
simulation in nursing: an integrative review. Clin Simul Nurs. 2019;37:18–28. doi:10.1016/j.ecns.2019.08.002
39. Chen FQ, Leng YF, Ge JF, et al. Effectiveness of
virtual reality in
nursing education: meta-analysis. J Med Internet Res. 2020;22(9):e18290. doi:10.2196/18290
40. Woon APN, Mok WQ, Chieng YJS, et al. Effectiveness of
virtual reality training in improving knowledge among nursing students: a systematic review, meta-analysis and meta-regression. Nurse Educ Today. 2021;98:104655. doi:10.1016/j.nedt.2020.104655
41. Plotzky C, Lindwedel U, Sorber M, et al.
Virtual reality simulations in nurse education: a systematic mapping review. Nurse Educ Today. 2021;101:104868. doi:10.1016/j.nedt.2021.104868
42. Brown N, Margus C, Hart A, Sarin R, Hertelendy A, Ciottone G.
Virtual reality training in disaster medicine: a systematic review of the literature. Simul Healthc. Published online June 14, 2022. doi:10.1097/SIH.0000000000000675
43. Tlili A, Huang R, Shehata B, et al. Is
metaverse in education a blessing or a curse: a combined content and bibliometric analysis. Smart Learn Environ. 2022;9(1):24. doi:10.1186/s40561-022-00205-x
44. Yang SY, Oh YH. The effects of neonatal resuscitation gamification program using immersive
virtual reality: a quasi-experimental study. Nurse Educ Today. 2022;117:105464. doi:10.1016/j.nedt.2022.105464
45. Hamilton D, McKechnie J, Edgerton E, Wilson C. Immersive
virtual reality as a pedagogical tool in education: a systematic literature review of quantitative learning outcomes and experimental design. J Comput Educ. 2021;8(1):1–32. doi:10.1007/s40692-020-00169-2
46. Hwang J, Kim H. Comparison of training effectiveness for iv injections: intravenous (IV) arm model versus computer simulator. J Korean Acad Fundam Nurs. 2014;21(3):302–310. doi:10.7739/jkafn.2014.21.3.302
47. Smith SJ, Farra SL, Ulrich DL, Hodgson E, Nicely S, Mickle A. Effectiveness of two varying levels of
virtual reality simulation. Nurs Educ Perspect. 2018;39(6):E10–E15. doi:10.1097/01.NEP.0000000000000369
48. Matthews B, See ZS, Day J. Crisis and extended realities: remote presence in the time of COVID-19. Media Int Aust. 2021;178(1):198–209. doi:10.1177/1329878X20967165
