BASIC RESEARCH

Does Virtual Reality Improve Procedural Completion and Accuracy in an Intramedullary Tibial Nail Procedure? A Randomized Control Trial

Orland, Mark D. BS; Patetta, Michael J. MD; Wieser, Michael MD; Kayupov, Erdan MD; Gonzalez, Mark H. MD, PhD

M. D. Orland, M. J. Patetta, E. Kayupoy, M. H. Gonzalez, Department of Orthopaedic Surgery, University of Illinois, Chicago, IL, USA

M. Wieser, Institute for Orthopaedic Surgery, Lima, OH, USA

M. J. Patetta, Department of Orthopaedics, University of Illinois at Chicago, 835 S. Wolcott Avenue, E270 MSS MC 844, Chicago, IL 60612 USA, Email: [email protected]

Each author certifies that neither he, nor any member of his immediate family, has funding or commercial associations (consultancies, stock ownership, equity interest, patent/licensing arrangements, etc) that might pose a conflict of interest in connection with the submitted article.

All ICMJE Conflict of Interest Forms for authors and Clinical Orthopaedics and Related Research® editors and board members are on file with the publication and can be viewed on request.

Clinical Orthopaedics and Related Research® neither advocates nor endorses the use of any treatment, drug, or device. Readers are encouraged to always seek additional information, including FDA approval status, of any drug or device before clinical use.

Each author certifies that his institution approved the reporting of this investigation and that all investigations were conducted in conformity with ethical principles of research.

This work was performed at the University of Illinois Hospital and Health Sciences System, Chicago, IL, USA.

Clinical Orthopaedics and Related Research 478(9):p 2170-2177, September 2020. | DOI: 10.1097/CORR.0000000000001362

Free
SDC
Corresponding Article

Abstract

Background

Artificial reality technologies are currently being explored as potential options to improve surgical education. Previous studies have primarily examined the efficacy of artificial reality in laparoscopic procedures, but to our knowledge, none have been performed in orthopaedically relevant procedures such as intramedullary tibial nailing, which calls for more versatile large-scale movements.

Questions/purposes

Does a virtual reality simulator with or without a standard technique guide result in (1) a higher proportion of participants who completed the insertion of an intramedullary tibial nail in a synthetic bones model and (2) greater procedural accuracy than does training with a technique guide alone?

Methods

Twenty-five first- and second-year medical students without prior exposure to intramedullary tibial nail insertion were recruited. Participants were randomly assigned to the technique guide control group (n = 8), the virtual reality group (n = 8), or the virtual reality and technique guide group (n = 9). The technique guide was adapted from a commercially available technique guide, which participants in the assigned groups could use to prepare as much as desired. The virtual reality simulation was based on the same procedure, and we used a commercially available virtual reality simulator that we purchased for this task. Participants in the virtual reality experimental groups completed the simulation on three separate sessions, at a set interval of 3 to 4 days apart. After 10 to 14 days of preparation, all participants attempted to insert an intramedullary nail into an intact, compact bone-model tibia that lacked surrounding soft tissue. Participants were given written hints if requested, but no other assistance was given. A procedure was considered complete if the nail and screw were properly placed. Procedural accuracy was defined as the number of incorrect steps normalized out of the 16 possible performed. After the procedure, one orthopaedic surgeon assessed a blinded video of the participant performing it so the assessor could not recognize the individual or that individual’s gender. Additionally, the assessor was unaware of which group each participant had been randomized to during the evaluation.

Results

A higher proportion of participants in the virtual reality group (6 of 8) and the virtual reality and technique guide group (7 of 9) completed the intramedullary nail than did participants in the technique guide group (2 of 8; p = 0.01). There was no difference in completion between the virtual reality groups (p = 0.89). Participants in the virtual reality and virtual reality and technique guide had fewer normalized incorrect steps than did participants in the technique guide group (3.2 ± 0.1 of 16 and 3.1 ± 0.1 of 16 versus 5.7 ± 0.2 of 16, respectively; p = 0.02 for comparisons of virtual reality groups to technique guide, p = 0.63 between the virtual reality group).

Conclusions

Level of Evidence

Level I, therapeutic study.

Introduction

The most common method of learning new procedures in surgical residencies involves surgical residents initially reading technique guides and watching videos of a procedure, while surgical skills are later learned and applied in a master-apprentice relationship [^{24, 30}]. One study found that 96% of residents taught themselves how to prepare for surgical procedures [¹⁵]. Another study found that fewer than 80% of graduating general surgery residents were rated as competent to perform more difficult but widely practiced “core procedures” on patients, highlighting a potential gap in surgical education and a need to explore supplementary training methods [⁸]. To address this deficit in surgical training, the Accreditation Council for Graduate Medical Education implemented the Next Accreditation System, which includes standardized surgical simulation programs with specific goals [¹⁶]. Additionally, there has been a shift from the “see one, do one, teach one” model to evidence-based medicine featuring increased simulation laboratories and team approaches [^{10, 16, 28}].

Augmented reality, mixed reality, and virtual reality are emerging in surgery. Augmented reality involves overlaying virtual simulated objects on real surroundings and allows interactions with the virtual objects; mixed reality also overlays virtual and real objects but incorporates interactions with both real and virtual objects; virtual reality immerses the user in a virtual environment with no use of real surroundings. These platforms have shown promising results in the development of arthroscopic surgical skills, especially in decreasing the learning curve for junior residents [^{2, 13, 25}]. However, all three platforms have been limited to surgeries requiring minimal movement, such as laparoscopic and endoscopic procedures, in the fields of neurosurgery, plastic surgery, general surgery and orthopaedic surgery [^{2, 7, 12, 13, 25}]. Because many of the most common procedures in the United States, such as cesarean sections and joint arthroplasties, require large-scale movements such as using a mallet, there is a need to explore the potential integration of these technologies into this class of surgical procedures [^{13, 21}]. There is a currently a lack of research on these technologies in procedures needing more versatile movements that are relevant to the practice of orthopaedic surgery.

We therefore asked: Does a virtual reality simulator with or without a standard technique guide result in (1) a higher proportion of participants who completed the insertion of an intramedullary tibial nail in a synthetic bones model and (2) greater procedural accuracy than does training with a technique guide alone?

Materials and Methods

Recruitment

Although orthopaedic residents might be ideal participants in a study like this, they were not used for the study because of the limited number of inexperienced residents at our institution and their variable exposure to trauma and intramedullary nails represented a substantial confounding variable. A review of virtual reality simulation training across 18 different studies, on average, had 23 to 24 participants and an effect size of 0.80 [^{12, 20}]. Additionally, an a priori power analysis estimated 21 participants were needed (effect size = 0.80, power = 0.80, alpha = 0.05). Based on the study and a priori analysis, we sought to include 31 medical students to account for projected attrition due to students’ schedules. After obtaining Institutional Review Board approval, 31 first- and second-year medical students from the University of Illinois College of Medicine at Chicago were recruited from August 14 to August 22 for the study via email. Six students were excluded from the study; five declined to participate after learning more about the study, and one student had previous experience with intramedullary tibial nail insertions. Twelve women and 13 men were included. A total of 17 first-year medical students and eight second-year medical students were included. Of the remaining 25 students, none had previous experience with virtual reality. Participants construction and power-tool background was also documented for later analysis.

This was a randomized control trial in which the assessor was blinded to students’ group assignments. Students were randomly assigned to one of three groups using a computer random number generator: a technique guide group (n = 8), a virtual reality group (n = 8), or a virtual reality and technique guide group (n = 9) (Fig. 1). Directly after running the number generator program, emails were individually sent by a different researcher. Evaluators were blinded and participants were assigned to groups without knowing who the other participants were and how others were assigned.

Fig. 1:
This flow diagram shows the virtual reality randomized control trial setup.

An intention-to-treat model was used, and all individuals were analyzed in the groups to which they were randomized. Before starting, all participants took a survey on the procedural steps of a tibial nail insertion to confirm that none had previous knowledge of it (see Appendix 1; Supplemental Digital Content 1, https://links.lww.com/CORR/A387).

Group Preparation

Participants in the three groups prepared to perform an intramedullary tibial nail insertion on a Sawbones compact bone model (Vashon, WA, USA) of a left tibia for 10 to 14 days. Spaced repetition, the learning method used for structuring virtual reality simulation spacing, has been associated with beneficial effects at 10 to 14 days [²⁶].

The technique guide group served as the control group and best represented the current model by which residents learn surgical procedures before entering the operating room [²⁴]. Participants in this group were given an online document with wording and images directly from the Zimmer (Warsaw, IN, USA) Natural Nail System Tibial Nail Technique Guide, which they were allowed to use at their discretion [¹⁷]. The virtual reality simulation was also based on the Natural Nail system [¹⁸]. However, because the virtual reality simulation did not include every step included in the technique guide, such as picking the appropriate reamer size for the patient or reaming up to 0.5 mm if the nail did not advance with impaction, steps were removed from the technique guide to match the virtual reality training. Students recorded how often and for how long the technique guide was accessed.

The virtual reality group and the virtual reality and technique guide groups were the experimental groups. The virtual reality platform was from OssoVR (Palo Alto, CA, USA), an orthopaedic virtual reality company that created a simulation of intramedullary tibial nail insertion [¹⁸]. The groups were structured to use spaced repetition with testing, an established learning theory [^{3, 11, 22, 23}]. One study found that found that the optimal gap of spaced repetition declines with time from 20% to 40% at a 1-week evaluation to 5% to 10% at a 1-year evaluation [⁴]. Given this finding and the nonlinear nature of recall, participants in the experimental groups participated in the virtual reality simulation in three separate sessions before the procedure at 10 to 14 days, requiring a gap of 20% to 31% (3 days to 4 days) between virtual reality sessions, with their first session being the first day of the study (Table 1).

Table 1.:
Examples of experimental participation schedules

During their first virtual reality session, each participant in an experimental group completed a tutorial on general orientation in the virtual reality space. During the first session, after completing the tutorial, participants were placed in a virtual operating room to perform the intramedullary nail procedure three times. The first time was designated as an opportunity for further acclimation to the virtual world and an introduction of how to use the tools correctly. Hints on how to perform the procedure were provided in the operating room by the software (Fig. 2). The second time, participants were told to focus on learning operative steps. During the third time through the simulation, all hints were turned off and participants were required to go through the procedure with at least 85% accuracy (Fig. 3). The participants had to go through 59 separately specified steps in the simulation to complete the procedure. Accuracy was the total steps subtracted from the number of hints used over the total number of steps. If they did not reach 85% accuracy, the participants had to repeat the procedure without hints until they achieved this percentage.

Fig. 2:
This image shows virtual reality simulation of intramedullary tibial nail insertion with hints. These images are reproduced with permission from OssoVR.

Fig. 3:
This image shows virtual reality simulation of intramedullary tibial nail insertion without hints. These images are reproduced with permission from OssoVR.

When the participants came in for their second and third virtual reality sessions, there was no tutorial, and they performed the simulation a minimum twice: once with hints and once without them. At the second and third sessions, participants were required to meet 90% and 95% accuracy respectively, or they had to repeat the simulation until reaching competency. The pass percentages were determined by an orthopaedic surgeon to reflect sufficient knowledge to perform the procedure with minimal difficulty. The number of hints used by each participant in the three sessions was recorded.

All participants in the virtual reality group passed with the required pass percentage (85%, 90%, and 95% for the first, second, and third sessions, respectively) on their first attempt. There was also a decrease in the number of hints these participants used from their first to second session (p < 0.001), but no difference from the second to third session (p = 0.05), with a mean of 5.7 ± 2.0 in the first session, 1.6 ± 2.0 in the second, and 0.3 ± 0.7 hints in the third session.

There was no difference in the preparation time of the virtual reality group (59 ± 5 minutes), virtual reality and technique guide (82 ± 21 minutes) compared with the technique guide group (66 ± 51 minutes; p = 0.13).

Artificial Compact Bone Model

On the day of compact bone nail insertion, before the procedure, all participants completed the same baseline survey they had initially taken. Participants were then instructed they would have 25 minutes to complete the procedure on the tibia with all of the necessary equipment included (Fig. 4). All students used an intact compact bone left tibia with a plastic cortical shell and an 11-mm canal without soft tissue. One study found that reaming did not add substantial time to an intramedullary tibial nail insertion; therefore, a plastic tibia with a canal that required minimal reaming was chosen [⁵]. After each procedure was completed, the canal was filled with wood putty and smoothed with sandpaper to prevent the next participant from knowing the entrance point without the proper knowledge from the technique guide or virtual reality simulation. Additionally, the wood putty created resistance in the tibia so the students could not slide the tibial nail in without using the proper technique.

Fig. 4:
This photograph shows the initial setup for intramedullary tibial nail compact bone insertion; all equipment is shown.

During the procedure, participants were not allowed to ask the researchers questions. Participants were instructed to complete the procedure as “effectively, efficiently, and independently” as possible, with no information regarding how they were being evaluated. Participants were given written hints if requested, but no other assistance was provided (see Appendix 2; Supplemental Digital Content 2, https://links.lww.com/CORR/A388). If the procedure was not completed in 26 minutes, the participant was told they were out of time and the nail insertion was stopped (Fig. 5).

Fig. 5:
Photograph demonstrating assembled drill cannula in the static hole slot of the insertion assembly.

Primary and Secondary Study Outcomes

Our primary study outcome was the proportion of participants in each of the three study groups who were able to complete the task. We defined a completed intramedullary nail as one where both the nail and screw were properly placed. After the procedure, an orthopaedic surgeon assessed a blinded video of the participant performing it so they could not recognize the individual or their gender. Additionally, they were unaware of which group each participant was in while evaluating them.

Secondary study outcomes included the proportion of incorrect steps in each group (defined as the number of incorrect steps compared with the number of steps performed), the number of hints requested during the test, and the mean time to completion of the task.

Statistical Methods

There were eight participants in the control technique guide group, eight in the virtual reality group, and nine in the virtual reality technique guide group. There was no difference in the demographics between the groups including the following: age, prior power-tool use, gender, and medical school class (Table 2). This was evaluated using a Kruskal-Wallis test for age and power tool use and a chi-square test for gender and medical school class.

Table 2.:
Results demonstrating no difference in preoperative variables between experimental groups.

Procedural objective parameters included whether the procedure was completed, procedure length (minutes), the number of hints requested, and the number of incorrect steps normalized out of the 16 possible performed. The number of hints requested and incorrect steps performed were normalized to the total steps completed by the participant. The completion proportion, mean time of completion, normalized mean number of hints used, and normalized mean number of incorrect steps per group were found (Table 3).

All collected data were blinded and analyzed using IBM SPSS Statistics, Version 25 (IBM Corp, Armonk, NY, USA); p values < 0.05 were considered significant. Kruskal-Wallis, Mann-Whitney U, and chi-square tests were used as appropriate for nonparametric, ordinal, or categorical variables.

Results

Proportion of Successful Procedures

Overall, participants in the virtual reality and virtual reality and technique guide experimental groups completed the tibial nail procedure more than the control technique guide group. A higher proportion of participants in the virtual reality group (6 of 8) and the virtual reality and technique guide group (7 of 9) completed the intramedullary nail than did participants in the technique guide group (2 of 8; p = 0.01). There was no difference in the completion proportion between the virtual reality-only group and the virtual-reality and technique-guide groups (Table 3).

Number of Errors and Other Secondary Endpoints

Participants in the virtual reality and virtual reality and technique guide had fewer normalized incorrect steps than did participants in the technique guide group (3.2 ± 0.1 of 16 and 3.1 ± 0.1 of 16 versus 5.7 ± 0.2 of 16, respectively; p = 0.02 for comparisons of virtual reality groups to technique guide, p = 0.63 between the virtual reality group).

Participants in the virtual reality and virtual reality and technique guide finished the procedure more quickly than those in the technique guide group (19 ± 7 minutes and 18 ± 8 minutes versus 24 ± 4, respectively; p = 0.03 for comparisons of virtual reality groups to technique guide, p = 0.70 between the virtual reality groups).

There was no difference in the normalized number of hints of the virtual reality and virtual reality technique guide groups compared with the technique guide group (2.1 ± 0.1 hints and 2.2 ± 0.2 hints versus 2.6 ± 0.1 hints, respectively; p = 0.30 for comparisons of virtual reality groups to technique guide; p = 0.92 between the virtual reality groups). The primary and secondary outcome values and p values varied (Table 3).

Discussion

Junior surgical residents often prepare themselves for procedures by reading technique guides and watching videos of a procedure [²⁴]. Virtual reality may offer a useful adjunct to these conventional training methods. To our knowledge, no studies have been performed in orthopaedically relevant procedures such as intramedullary tibial nailing. This study sought to determine whether virtual reality could improve inexperienced individuals’ surgical completion and accuracy of an intramedullary tibial nail procedure, which requires versatile movements that are relevant to the practice of orthopaedic surgery. We found that virtual reality increased procedural completion and decreased the number of errors when compared with solely using a technique guide. Virtual reality has the potential to be used in surgical residencies as a tool to catalyze learning a procedural workflow and to facilitate practice of required surgical movements.

Limitations

This study had several important limitations. First, while sufficient based on a priori analysis, only 25 medical students were included in the study. Thus, results are subject to statistical fragility; that is, a few more participants completing (or not completing) the procedure in one group or another could have changed a main research finding. For that reason, our conclusions are preliminary. Therefore, future studies with larger sample sizes are needed to more conclusively evaluate the ability of virtual reality to increase procedural completion and accuracy in complex surgical procedures.

Provided programming forced virtual reality groups to reach 85% to 95% competency before completing a training session, while equivalent standards were not applied to students using technique guides, introducing a potential procedural bias. Though no participants were required to repeat training at any session, future studies should eliminate this stipulation or attempt to create a similar standard for the technique guide group. Additionally, although our study surveyed power-tool use, other demographic controls should be obtained in future works, including a history of participation in sports or a survey of musical ability.

Medical students were used as participants in this study because of their naïveté to orthopaedic procedures and more flexible schedules; away rotations exposure and completed clinical rotations vary in junior residents and represented a substantial confounder. Regardless of variable exposure, there is value in examining junior residents in future studies and the efficacy of virtual reality training. Additionally, residents learn in different formats including videos, technique guides, and other resources and they often use guided learning before performing a surgery [²⁴]. This controlled environment will not exist in residency.

Although this study demonstrated virtual reality training improved procedural accuracy and completion, the simulated compact bone model procedure does not represent a perfect representation of a real surgical presentation. Furthermore, though all important steps were accounted for, it is difficult to emulate some haptic and technical aspects of the surgical environment, such as selection of appropriate reaming diameter.

Improved Completion of the Surgical Task and Error Reduction with Virtual Reality

We found that virtual reality as a training tool, with or without a technique guide, resulted in a higher proportion of participants being able to complete the surgical task we used to evaluate them compared with participants who used the technique guide alone. We also found that participants who trained with virtual reality made fewer errors during the procedure. This is potentially important because of the growing evidence that surgical residents are underprepared for independent practice; a training tool that can decrease procedural errors while providing extra repetitions to build competency may improve patient safety [^{8, 9}].

Practical Implications

Virtual reality has a potential use in surgical residency programs. However, the technology itself has several limitations that should be addressed before this application. As of now, the haptics of the system does not accurately emulate many orthopaedic procedures and is mostly limited to arthroscopy-based procedures [^{14, 31}]. Although there have been varying results on whether haptics are essential for laparoscopic procedures, large-scale procedures have yet to be thoroughly examined [¹⁹]. Additionally, the cost of the simulators when compared with their potential benefit must be further examined. Finally, given the degree of procedures orthopaedic surgeons are required to know, research is needed into how many and which procedures would best be learned using virtual reality and the degree of work needed from the simulation platforms to code this.

Application of virtual reality technology allows efficient surgical learning that can be applied safely at any point in training and is especially useful before performing an operation for the first time. Additional research is needed to examine the benefits of virtual reality training in other complex procedures and for optimizing its integration into surgical education. We believe virtual reality training would be most applicable to junior residents as a way to simulate performing the steps and movements required for a variety of procedures before ever stepping foot in the operating room. Developing technologies integrating real-world, tactile sensations into virtual reality simulations may advance the role of virtual reality as a training tool [²⁹].

The American Board of Surgery and the American College of Surgery [¹] endorse preparing medical students for surgical residency through preparatory courses and methods that can address quality care and patient safety before the start of their training. Virtual reality can address both areas of emphasis by improving a medical student’s preparation for surgical clerkships and improving their tactile skills, which are necessary for proper care once in residency. Courses are beginning to focus on developing surgical skills before clerkship [²⁷]. The implementation of virtual reality for clerkship preparation or as a supplement to the surgical rotation can improve a student’s knowledge and clinical skills. One study found that medical students desired more instruction and feedback during their surgical rotations, and approximately 50% of students thought they were an inconvenience to the service [⁶]. If implemented with surgical rotations, virtual reality could increase the instruction and feedback students receive because the virtual reality software can give hints that can be turned on for feedback and instruction and off for recall purposes [¹⁸].

Conclusions

Virtual reality increased both procedural accuracy and the completion proportion compared with a technique guide in medical students. Based on our findings, virtual reality may help residents learn the procedural workflow and movements required to perform surgical procedures. However, further research is needed using a larger sample sizes and evaluating simulations of other orthopaedic procedures before widespread application. Finally, future studies should evaluate the utility of virtual reality training at different levels of surgical education and residency training.

References

1. American Board of Surgery, American College of Surgeons, Association of Program Directors in Surgery, Association for Surgical Education. Statement on surgical preresidency preparatory courses. Am Surg. 2014;80:1085-1086.

2. Bissonnette V, Mirchi N, Ledwos N, Alsidieri G, Winkler-Schwartz A, Del Maestro RF. Artificial Intelligence Distinguishes Surgical Training Levels in a Virtual Reality Spinal Task. J Bone Joint Surg. 2019:102:e127.

3. Brown P, Roediger H III, McDaniel M. Make It Stick: The science of successful learning. Cambridge Massachusetts: The Belknap Press of Harvard University Press; 2014.

Cited Here

4. Cepeda NJ, Vul E, Rohrer D, Wixted JT, Pashler H. Spacing effects in learning: A temporal ridgeline of optimal retention: Research article. Psychol Sci. 2008;19:1095–1102.

5. Crist BD, Wolinsky PR. Reaming does not add significant time to intramedullary nailing of diaphyseal fractures of the tibia and femur. J Trauma. 2009;67:727-734.

6. De SK, Henke PK, Ailawadi G, Dimick JB, Colletti LM. Attending, house officer, and medical student perceptions about teaching in the third-year medical school general surgery clerkship. J Am Coll Surg. 2004;199:932-942.

7. Gasperin BDM, Zanirati T, Cavazzola LT. Can virtual reality be as good as operating room training? Experience from a residency program in general surgery. [In English, Portuguese]. Arq Bras Cir Dig. 2018;31:e1397.

8. George BC, Bohnen JD, Williams RG, Meyerson SL, Schuller MC, Clark MJ, Meier AH, Torbeck L, Mandell SP, Mullen JT, Smink DS, Scully RE, Chipman JG, Auyang ED, Terhune KP, Wise PE, Choi JN, Foley EF, Dimick JB, Choti MA, Soper NJ, Lillemoe KD, Zwischenberger JB, Dunnington GL, Darosa DA, Fryer JP. Readiness of US General Surgery Residents for Independent Practice. Ann Surg. 2017;266:582–594.

9. Grant-Orser A, Davies P, Singh SS. The lost sponge: Patient safety in the operating room. CMAJ. 2012;184:1275–1278.

10. Green CA, Kim EH, O’Sullivan PS, Chern H. Using Technological Advances to Improve Surgery Curriculum: Experience With a Mobile Application. J Surg Educ. 2018;75:1087-1095.

11. Kerfoot BP. Adaptive spaced education improves learning efficiency: a randomized controlled trial. J Urol. 2010;183:678-81.

12. Khan R, Plahouras J, Johnston BC, Scaffidi MA, Grover SC, Walsh CM. Virtual reality simulation training for health professions trainees in gastrointestinal endoscopy. Cochrane Database Syst Rev. 2018;8:CD008237.

13. Kim Y, Kim H, Kim YO. Virtual reality and augmented reality in plastic surgery: A review. Arch Plast Surg. 2017;44:179-187.

14. Li L, Yu F, Shi D, Shi J, Tian Z, Yang J, Wang X, Jiang Q. Application of virtual reality technology in clinical medicine. Am J Transl Res. 2017;9:3867-3880.

15. Mundschenk MB, Odom EB, Ghosh TD, Kleiber GM, Yee A, Patel KB, Mackinnon SE, Tenenbaum MM, Buck DW. Are Residents Prepared for Surgical Cases? Implications in Patient Safety and Education. J Surg Educ. 2018;75:403-408.

16. Norrell K, Marasigan J, Bogener J. New Paradigms in Post-Graduate Surgical Education. Mo Med. 2017;114:278-282.

17. Orthopedic Trauma | Zimmer® Natural Nail® System | Zimmer Biomet. Available at: https://www.zimmerbiomet.com/medical-professionals/trauma/product/zimmer-natural-nail-system.html. Accessed October 30, 2019.

18. OssoVR - Virtual Reality Surgical Training Platform. Available at: https://ossovr.com/. Accessed October 29, 2019.

19. Overtoom EM, Horeman T, Jansen FW, Dankelman J, Schreuder HWR. Haptic Feedback, Force Feedback, and Force-Sensing in Simulation Training for Laparoscopy: A Systematic Overview. J Surg Educ. 2019;76:242-261.

20. Park J, MacRae H, Musselman LJ, Rossos P, Hamstra SJ, Wolman S, Reznick RK. Randomized controlled trial of virtual reality simulator training: transfer to live patients. Am J Surg. 2007;194:205–211.

21. Pfuntner A, Wier L, Stocks C. Statistical Brief - Most Frequent Procedures Performed in U.S. Hospitals, 2010. Available at: http://www.hcup-us.ahrq.gov/reports/statbriefs/sb149.pdf. Accessed November 3, 2019.

22. Phillips JL, Heneka N, Bhattarai P, Fraser C, Shaw T. Effectiveness of the spaced education pedagogy for clinicians’ continuing professional development: a systematic review. Med Educ. 2019;53:886-902.

23. Price Kerfoot B, Brotschi E. Association for Surgical Education Online spaced education to teach urology to medical students: a multi-institutional randomized trial. Am J Surg. 2009;197:89-95.

24. Rapp AK, Healy MG, Charlton ME, Keith JN, Rosenbaum ME, Kapadia MR. YouTube is the Most Frequently Used Educational Video Source for Surgical Preparation. J Surg Educ. 2016;73:1072-1076.

25. Rebolledo BJ, Hammann-Scala J, Leali A, Ranawat AS. Arthroscopy Skills Development With a Surgical Simulator. Am J Sports Med. 2015;43:1526-1529.

26. Rohrer D, Dedrick RF, Burgess K. The benefit of interleaved mathematics practice is not limited to superficially similar kinds of problems. Psychon Bull Rev. 2014;21:1323-1330.

27. Sammann A, Tendick F, Ward D, Zaid H, O’Sullivan P, Ascher N. A surgical skills elective to expose preclinical medical students to surgery. J Surg Res. 2007;142:287–94.

28. Stautberg EF III, Romero J, Bender S, DeHart M. Orthopaedic Resident Practice Management and Health Policy Education: Evaluation of Experience and Expectations. Cureus. 201810:e2461.

29. Tao Z, Keyu W, Yanlin Y, Yi Z. Eye-around vibration haptics on VR immersion improvement. Virtual Real Intell Hardw. 2019;1:176.

30. Theelen L, Bischoff C, Grimm B, Heyligers IC. Current practice of orthopaedic surgical skills training raises performance of supervised residents in total knee arthroplasty to levels equal to those of orthopaedic surgeons. Perspect Med Educ. 2018;7:126-132.

31. Vaughan N, Dubey VN, Wainwright TW, Middleton RG. A review of virtual reality based training simulators for orthopaedic surgery. Med Eng Phys. 2016;38:59-71.

Secondary Logo

Journal Logo

Does Virtual Reality Improve Procedural Completion and Accuracy in an Intramedullary Tibial Nail Procedure? A Randomized Control Trial