Whether defining safe and minimally invasive trajectories to access neoplasms or training for techniques used in debulking a lesion, a consummate understanding of the regional anatomy is essential. To gain such understanding, a safe and effective environment is critical to provide an accurate emulation of the anatomy and deliberate practice for learning and maintenance of psychomotor skills.
To provide this environment, we explored various approaches, including physical modeling with material tissue correlates, virtual representation, and a hybrid approach. Obviously, there are advantages and disadvantages with either a physical or a virtual approach. Both approaches have issues with biofidelity, especially in the sense of visual accuracy and tissue interactions. Physical models can be elegantly constructed, especially with the advent of 3-dimensional printing. Physical models allow the trainee to use actual instruments to execute performance. However, cost and repetitive use can be limiting. Physical models often provide no anatomic or pathological variance. Virtual models permit relatively low-cost simulations. Computational technology and interface equipment have stabilized in utility and seen a reduction in cost. With the use of open-source software and with graphics processing unit video board developments being pursued and led by the gaming industry, these costs also continue to decline. Because the model is digital, repetition is not an issue. Anatomic and pathological variance can be captured in the acquisition, ie, using a specimen with variance, or the digital model often can be modified to emulate a variance. However, the field is relatively new, and there are limited interfaces that can accurately emulate hand-instrument and instrument-tissue interaction.
Nevertheless, we assert that there are distinct advantages in a virtual approach. Here, we present our methods of and results from our initial deployment of a fully virtual, interactive environment for learning neurosurgical techniques and user assessment. The context of this deployment was within Congress of Neurological Surgeons (CNS) simulation courses and was based specifically on procedures involving drilling (ie, pterional burr hole and presigmoid and retrosigmoid craniotomy).
The application of simulation to neurosurgery has evolved over the span of several decades. Kelly1 introduced the use of 3-dimensional reconstructions for use in the preoperative assessment and planning of volumetric stereotactic procedures. In 2006, Wang et al2 introduced realistic deformable models depicting prodding, pulling, and cutting of simulated soft tissues. In 2007, Lemole et al3 demonstrated a system for ventriculostomy training that used haptic feedback. Concomitantly, Acosta et al4 presented a haptic approach for a burr hole simulation. Both of these approaches combine natural viewing of the hands and synthesized visuals in an augmented reality approach, a hybrid of real and virtual components. Hofer et al5 discussed using navigated control for avoiding critical structures during surgical intervention. These approaches rely on a virtual model for accurate and precise planning and execution. More recently, Delorme and colleagues6 presented NeuroTouch, an integrated system including stereographics and haptic manual interfaces for microneurosurgical training. Through funding from the National Research Council Canada, the effort includes 20 sites participating in beta testing and validation.
Our early studies related to this effort correlated structural information from volumetric magnetic resonance data with functional data from electroencephalograms into integrated displays used for investigating drug and alcohol addictions and sleep disorders.7,8 Subsequent work involved the development and evaluation of 3-dimensional volumetric displays of patient-specific data compared with traditional methods in the study of brain and cranial base tumors.9-15 Concurrent work involved simulations for training anesthesia residents in the delivery of an epidural.16,17 The epidural anesthesia simulations were our first investigations into integrating volume graphics with haptics (force reflecting technology). Using volumetric techniques, we also simulated pelvic compression neuropathies associated with birthing.18 Subsequently, we were part of a multi-institutional effort to develop and evaluate a functional endoscopic sinus surgery simulator that integrated visual and haptic interfaces. This involved 2 parallel developments, the first focusing on surface-based representations19,20 and the second focusing on volumetric representations.21-26 These studies showed that although surface-based representations were expedient and could provide interactive rates, they lacked the complexity and realism found in volumetric displays.27 The ENT Surgical Trainer, as it has come to be known, has been identified as the first true procedural surgical simulation environment to undergo vigorous validation.28
We have developed a virtual simulation for use in the training of temporal bone dissection for the laboratory that combines multimodal representations, stereoscopic volume rendering, and haptic and aural (stereo) feedback.29 We have disseminated our temporal bone dissection simulator to 10 other institutions to obtain formative and preliminary summative evaluations.30 The study demonstrated that virtual representations were capable of providing introductory training equal to cadaveric models.31,32 The simulator is currently being used to conduct a multiple-institution randomized controlled trial to evaluate its efficacy for use in training, specifically in the integration of standardized metrics and automated assessment of performance.
Recently, we demonstrated translation of the otologic technique simulator for the emulation of skull base techniques used in neurosurgery.33 This simulation is completely virtual, providing visual, aural, and haptic (tactile) forces in an interactive, multisensory interface.
The course of our methods can be categorized into the following steps: data acquisition, preprocessing, segmentation, systems integration, and rendering and use (Figure 1).
To obtain digital models, specimens were imaged with a Siemens 64-detector Somatom computed tomography scanner with a modified inner ear protocol with a field of view of 119 mm, an in-plane resolution of 0.232, and a slice thickness of .6 mm.34 Imaging was conducted at the Ross Heart Hospital at The Ohio State University Wexner Medical Center. Digital data were then transferred to the Ohio Supercomputer Center for preprocessing.
Preprocessing and Segmentation
Preprocessing includes rescaling and filtering to clean and prepare the data for segmenting. Segmentation involves interactive user-mediated demarcation of critical structures, surfaces, and regions based on surgical significance. The segments are used to localize instruments and to determine violation of key and critical structures and the contextual use of force, metrics that will be used in quantitative automated assessment. In the neurosurgical application of our system, we have processed the data sets to provide for emulation of pterional, and presigmoid, and retrosigmoid approaches (Figure 2) to the lateral skull base.
Systems Integration, Rendering, and Use
The system comprises a desk-side computer (Intel quad-core 3.6-GHz processor with 16 GB of random-access memory running Windows7) with an NVidia (Quadro5000) graphics processing unit. Processed and segmented data are loaded into the graphics processing unit for interactive stereo (20-30 frames per second) visuals. We use a mix of custom rendering software using open-source (ie, OpenGL) methods of direct volume rendering to achieve these results, including interactive drilling and removal of bone. A Geomagic Sensable Omni is used as a dexterous device to provide direct emulation of the user's manipulation of instruments, specifically the drill burr used to remove bone and to gain access to underlying structures. Forces are calculated on the basis of regional voxel intensities. Resistance to the applied force is used to modulate stored sounds of drilling. Stereo glasses and a stereo-ready monitor and headphones complete the interface (Figure 3), providing a rich, multisensory environment based completely on the user's interaction with a virtual model.
Table 1 demonstrates the directive and methods of scoring used in the study.
Quantitative Automated Assessment
Our current approach is based on a set of rules defined by expert surgeons. These rules rely on the volume data comprising defined structures and regions (eg, sutures, sinuses, semicircular canals). On the basis of (virtual) procedures performed on the simulator by experts, we determine logical cutoff points for the amount of bone removal required for a certain level of expertise. For example, in the retrosigmoid craniotomy task, between 1% and 50% of the posterior fossa dural plate must be removed during the procedure to get credit (Figure 4). Refer to Table 1 for our complete rule-based grading system. Penalties can be given if excessive or insufficient amounts of a specified structure are removed. A breakdown of their score is provided to the residents on completion of the procedure, and results are saved for further analysis. In a deployment of this system in a classroom setting, residents (and their attendings) will be able to see their progress in procedures over time. Faculty time investment is optimized in that both formative and summative feedback is provided to the trainee independently of the faculty's physical presence.
For the pterional, presigmoid, and postsigmoid approaches, we provide quantitative automated scores to the final products of virtual models representing the application of procedural techniques.35,36 We asked 3 experts to perform the 3 techniques as many times as possible. This information is used to construct a composite score for determining expert variance of technique. By comparing the matching regions of voxels between high-quality expert products, ie, drilled virtual bones, and those of residents using various distance metrics, we obtain a feature vector that is used in a range of clustering and classification algorithms. Given a set of final products previously graded by an expert, we then can extract features from the volumes and construct a decision tree.37 Preliminary results with this method have resolved information for assessment more powerful than hand motion analysis in some metrics, obtaining κ scores >0.6 when comparing expert and automated grading scores for such metrics as “complete saucerization” and “antrum entered.” This approach provides scoring for metrics that are not easily defined in terms of strict structural boundaries and can be straightforwardly extended to different procedures.
We have presented our methods for constructing a virtual environment for procedural drilling techniques and their application to neurosurgery. We have conducted a small pilot study to gather feasibility data in practical courses that were held at the CNS 2011 Annual Meeting in Washington, DC, and the CNS 2012 Annual Meeting in Chicago, Illinois. A short didactic review was provided before the simulation study. Preassessment of skill was limited to the participants providing their rank. All participants were asked to perform the 3 techniques the best they could.
At this time, our data are not generalizable. We have seen a wide variance in user performance. In our initial data, we had 17 participants, not all of whom finished every task listed above. A summary of the results is available in Table 2. The highest possible score is 90, and the subjects were early-year residents. The low scores in general may indicate that the tasks are too difficult for early-year residents.
These preliminary pilot studies have, however, provided us valuable formative evaluation that will be integrated into the simulation to pursue the next level of controlled randomized validation studies required to further evaluate the efficacy of our designs in the neurosurgical curriculum. These studies will include pretest and posttest assessments, with a 2-arm approach of simulation vs traditional training techniques.
The automated assessment provides a means of learning procedural techniques with feedback that does not require supervision by an attending surgeon. The breakdown with immediate scores gives a more concrete indication for improvement than hand motion analysis, ie, economy of movement. Most important, all cases used for assessment are exactly equal in difficulty. This approach provides a more quantified and objective assessment of procedural skills.
We have presented the adaptation of virtual simulation techniques developed for otologic surgery to neurological techniques. Essential to this adaptation is the development of automated assessment techniques and their use in analysis of resident performance. These assessments are congruent with the CNS simulation initiative to provide bold and innovative methods for training and assessments. More specifically, it addresses the call for the formal integration of simulation as a training modality in the curriculum.38 This effort seeks to address the “technological gaps and limitations” of current simulators and to help to balance “the cost and time needed for development of new simulations.”
Through the adoption and adaptation of virtual techniques, we present the advantages of cost efficiency, cleanliness, standardization and repeatability, and provision of anatomic and eventually pathological variance. The current limitations of a virtual approach include lack of soft-tissue emulation and representation of the management of fluids such as bleeding and irrigation, although we are actively pursuing these capabilities.39 In summary, we contend not only that a virtual approach can more rapidly capitalize on new technological advances but also that the virtual representation can itself iteratively drive innovation in other technological domains such as imaging and robotics, as well as tool design and testing.
We have presented our efforts to translate an otologic simulation environment for use in neurosurgical training. We have demonstrated the initial proof of principles and now are considering the steps to integrate and validate the system as an adjuvant to the neurosurgical curriculum as part of the CNS simulation initiative.
This research was supported in part by the Congress of Neurological Surgeons. Additional developments were supported through funding from R01 DC011321001 A1 through the National Institute on Deafness and Other Communications Disorders (NIDCD) of the National Institutes of Health. Dr Rezai is currently the President of the Congress of Neurological Surgeons. D. Stredney and Dr Wiet have received material support provided by Medtronic and Stryker (instruments) and Cochlear Americas (specimens). The other authors have no personal financial or institutional interest in any of the drugs, materials, or devices described in this article.
We would like to acknowledge the efforts of Kimerly Powell, PhD, director of the Small Animal Imaging Shared Resource at The Ohio State University, for her expertise, guidance, and assistance in acquiring and preprocessing the data sets used in this research.
1. Kelly PJ. Quantitative virtual reality enhances stereotactic neurosurgery. Bull Am Coll Surg. 1995;80(11):13–20.
2. Wang P, Becker AA, Jones IA, Glover AT, Benford SD, Greenhalgh CM, et al.. A virtual reality surgery simulation of cutting and retraction in neurosurgery with force-feedback. Comput Methods Programs Biomed. 2006;84(1):11–18.
3. Lemole GM Jr, Banerjee PP, Luciano C, Neckrysh S, Charbel FT. Virtual reality in neurosurgical education: part-task ventriculostomy simulation with dynamic visual and haptic feedback. Neurosurgery. 2007;61(1):142–148; discussion 148-149.
4. Acosta E, Liu A, Armonda R, et al.. Burrhole simulation for an intracranial hematoma simulator. Stud Health Technol Inform. 2007;125:1–6.
5. Hofer M, Dittrich E, Scholl C, et al.. First clinical evaluation of the navigated controlled drill at the lateral skull base. Stud Health Technol Inform. 2008;132:171–173.
6. Delorme S, Laroche D, DiRaddo R, Del Maestro RF. NeuroTouch: a physics-based virtual simulator for cranial microneurosurgery training. Neurosurgery. 2012;71(1 suppl operative):32–42.
7. Lukas SE, Sholar M, Stredney D, Torello MW, May SF, Scheepers F. Integration of P300 Evoked Potentials With Magnetic Resonance Images (MRI) to Identify Dipole Sources in Human Brain. Washington, DC: Society of Neuroscience; 1993.
8. Lukas SE, Sholar MB, Stredney D, Torello MW, May SF, Scheepers F. Apparent Source of EEG Sleep Spindles and K-Complexes: Correlations With Anatomical Sites Using Magnetic Resonance Imaging (MRI). Boston, MA: American Sleep Disorders Association; 1994.
9. Stredney D, Yagel R, May SF, Torello M. Supercomputer Assisted Brain Visualization With an Extended Ray Tracer: Proceedings of the 1992 Workshop on Volume Visualization. Springfield, VA: ACM; 1992:33–38.
10. Stredney D, Crawfis R, Wiet GJ, Sessanna D, Shareef N, Bryan J. Interactive volume visualizations for synchronous and asynchronous remote collaboration. Stud Health Technol Inform. 1999;62:344–350.
11. Stredney D, Agrawal A, Barber D, et al.. Interactive medical data on demand: a high-performance imaged-based approach across heterogeneous environments. Stud Health Technol Inform. 2000;70:327–333.
12. Wiet GJ, Schuller DE, Goodman J, et al.. Virtual Simulations of Brain and Cranial Base Tumors. San Diego, CA: Otolaryngology, Head and Neck Surgery; 1994.
13. Wiet GJ, Stredney D, Yagel R, et al.. Cranial base tumor visualization through high-performance computing. Stud Health Technol Inform. 1996;29:43–59.
14. Wiet GJ, Stredney DL, Yagel R, Sessanna DJ. Using advanced simulation technology for cranial base tumor evaluation. Otolaryngol Clin North Am. 1998;31(2):341–356.
15. Wiet GJ, Stredney D, Schmalbrock P. Tumor visualization.
16. Stredney D, Sessanna D, McDonald JS, Hiemenz L, Rosenberg LB. A virtual simulation environment for learning epidural anesthesia. Stud Health Technol Inform. 1996;29:164–175.
17. Hiemenz L, Stredney D, Schmalbrock P. Development of the force-feedback model for an epidural needle insertion simulator. Stud Health Technol Inform. 1998;50:272–277.
18. McDonald JS, Yagel R, Schmalbrock P, Stredney D, Reed DM, Sessanna D. Visualization of compression neuropathies through volume deformation. Stud Health Technol Inform. 1997;39:99–106.
19. Edmond CV Jr, Heskamp D, Sluis D, et al.. ENT endoscopic surgical training simulator. Stud Health Technol Inform. 1997;39:518–528.
20. Weghorst D, Airola C, Openheimer P, Edmond CV, Patience T, Heskamp D, et al.. Validation of the Madigan ESS simulator. In: Westwood JD, ed. Proceedings of the MMVR6. Amsterdam, Netherlands: IOS Press; 1998:399–405.
21. Rudman DT, Stredney D, Sessanna D, et al.. Functional endoscopic sinus surgery training simulator. Laryngoscope. 1998;108(11 pt 1):1643–1647.
22. Wiet GJ, Yagel R, Stredney D, et al.. A volumetric approach to virtual simulation of functional endoscopic sinus surgery. Stud Health Technol Inform. 1997;39:167–179.
23. Yagel R, Stredney D, Wiet GJ, Schmalbrock P, et al.. Multisensory Platform for Surgical Simulation. Santa Clara, CA: IEEE VRAIS; 1996:72–78.
24. Rosenberg LB, Stredney D. A haptic interface for virtual simulation of endoscopic surgery. In: Weghorst S, et al., ed. Proceedings of the MMVR4. Amsterdam, Netherlands: IOS Press; 1996.
25. Yagel R, Stredney D, Wiet GJ, Schmalbrock P, Rosenberg L, Sessanna D, et al.. Towards Real-Time Multisensory Virtual Surgery. IEEE Multimedia; 1996.
26. Yagel R, Stredney D, Wiet GJ, Schmalbrock P, Rosenberg L, Sessanna DJ, et al.. Building a virtual environment for endoscopic sinus surgery simulation. Comp Graphics. 1996;20(6):813–823.
27. Stredney D, Wiet GJ, Yagel R, et al.. A comparative analysis of integrating visual representations with haptic displays. Stud Health Technol Inform. 1998;50:20–26.
28. Gallagher AG, Ritter EM, Satava RM. Fundamental principles of validation, and reliability: rigorous science for the assessment of surgical education and training. Surg Endosc. 2003;17(10):1525–1529.
29. Bryan J, Stredney D, Wiet GJ, Sessanna D.
30. Wan D, Wiet GJ, Welling DB, Kerwin T, Stredney D. Creating a cross-institutional grading scale for temporal bone dissection. Laryngoscope. 2010;120(7):1422–1427.
31. Wiet GJ, Rastatter JC, Bapna S, Packer M, Stredney D, Welling DB. Training otologic surgical skills through simulation-moving toward validation: a pilot study and lessons learned. J Grad Med Educ. 2009;1(1):61–66.
32. Wiet GJ, Stredney D, Kerwin T, et al.. Virtual temporal bone dissection system: OSU virtual temporal bone system: development and testing. Laryngoscope. 2012;122(suppl 1):S1–S12.
33. Prevedello DM, Stredney D, Rezai A. Skull base simulators: the evolution of neurosurgical training. CNSQ. 2011;12(3):7–8.
34. Wiet GJ, Schmalbrock P, Powell K, Stredney D. Use of ultra-high-resolution data for temporal bone dissection simulation. Otolaryngol Head Neck Surg. 2005;133(6):911–915.
35. Kerwin T, Wiet G, Stredney D, Shen HW. Automatic scoring of virtual mastoidectomies using expert examples. Int J Comput Assist Radiol Surg. 2012;7(1):1–11.
36. Kerwin T, Shen HW, Stredney D. Capture and review of interactive volumetric manipulations for surgical training. In: Volume Graphics. 2006:106.
37. Kerwin T, Stredney D, Wiet G, Shen HW. Virtual mastoidectomy performance evaluation through multi-volume analysis. Int J Comput Assist Radiol Surg. 2012;8(1):51–61.
38. Lobel DA, Rezai A. Frontiers in neurosurgery: simulation in resident education. CNSQ. 2011;4–6.
39. Kerwin T, Shen HW, Stredney D. Enhancing realism of wet surfaces in temporal bone surgical simulation. IEEE Trans Vis Comput Graph. 2009;15(5):747–758.