Selden, Nathan R. MD, PhD*; Origitano, Thomas C. MD‡; Hadjipanayis, Costas MD, PhD§; Byrne, Richard MD¶
Section Editor(s): Harrop, James S. MD; Bendok, Bernard R. MD
Multiple factors have increased the imperative for using simulation in the training of physicians and surgeons. The Accreditation Council for Graduate Medical Education (ACGME) duty hours introduced in 2003 and enhanced in 2011 have reduced the amount of time available for exposure to the clinical environment.1-3 Public and regulatory attention has increasingly focused on quality and outcome measures in surgery, including stringent demands for improved safety in the training environment.4,5
The ACGME (www.acgme.org) currently requires general surgery training programs to maintain simulation laboratories and otolaryngology training programs to maintain skull base dissection training laboratories. Many US medical schools and academic medical centers are currently constructing dedicated simulation centers for teaching both behavioral and procedural aspects of medicine and surgery.6-8
Simulation in surgery is not, however, a new phenomenon. Cadaveric dissection, which is a form of surgical simulation, has been widely used since the Renaissance.9 Simple clay and stone figurines and models have been used for crude medical simulation teaching since antiquity.10
Public perception of simulation training in the digital era generally focuses on the use of complex technological devices. Airline pilots train for commercial jet flight on huge, expensive, high-fidelity simulators that re-create not only the interior details and physical and computer controls of the flight cockpit in each individual airframe but also the pitch, yaw, and attitude of the plane in flight in response to pilot maneuvers. Nuclear engineers simulate emergencies in full-scale plant control rooms indistinguishable from the corresponding real-world environment. These highly complex simulated environments improve the fidelity of behavioral simulations of tasks and emergencies. Simultaneously, simulator performance provides an objective assessment of trainee performance.
Although high-tech simulation has come later to surgery, a number of surgical fields have incorporated simulation into residency training, particularly general surgery, emergency medicine, obstetrics and gynecology, otolaryngology, cardiothoracic surgery, and recently neurosurgery. Use of simulators in surgical and interventional specialties has advanced most quickly for endoscopic and endovascular techniques, which lend themselves to relatively focused, high-fidelity simulation.11-16 The first neurosurgery haptic simulators have similarly focused on point-to-point navigation for ventriculostomy placement.17 In contrast, simulation of the complexities of open surgical procedures for variable pathologies is a daunting challenge.18
The education of early learners in surgical and procedural specialties provides particular opportunities and challenges for effective simulator-based training. Residents at the beginning of training perform invasive procedures under direct supervision, concentrating on isolated technical maneuvers while an experienced teacher who is physically present manages the overall procedural flow and response to unanticipated problems. Thus, new residents should benefit more from simulation that breaks down isolated technical aspects of procedures into individual steps that can be mastered and practiced in the simulated environment. Similarly, early learners should benefit most from behavioral and decision-making simulation and assessment targeted at the initial management steps for common and urgent clinical problems that are frequently encountered in the course of their initial clinical duties, including events that happen during nighttime call shifts, occur in the emergency department, or are associated with a change in patient condition on an inpatient clinical unit.
Such fundamental skills and behavior training for entering postgraduate year 1 (PGY1) residents have traditionally been provided by general surgery residency programs, according to locally developed and variable curricula. In 2009, all ACGME-accredited US neurosurgical training programs simultaneously incorporated PGY1 training into neurosurgical residency under the purview of neurosurgical program directors. This systematic change created a nationwide need for appropriate introductory training for approximately 200 beginning residents each year, as well as an opportunity to shape such training as part of a national curriculum.
Based on pilot courses in 2009, the Society of Neurological Surgeons (SNS; www.societyns.org) introduced a systematic curriculum in 2010 that is carried out at 6 regional centers and is available to all entering PGY1 residents in ACGME-accredited neurosurgical training programs.19,20 By 2011, 100% of US residents participated in the program.21
This article describes in particular the use of both technical and behavioral simulation for entering PGY1 neurosurgery residents as part of a national curriculum, focusing on the development of model-based simulators for early learners and refinement of initial simulation models as a defined process of iterative curricular and course improvement. Other parallel efforts to use model-based simulation as a pedagogical tool for early neurosurgical learners are also discussed.
In 2009, courses to introduce entering PGY1 neurosurgical residents to relevant skills and knowledge were piloted at 2 centers in Portland, Oregon, and Chicago, Illinois. These courses included simulation of procedural and basic cranial surgical skills.19 On the basis of preliminary experience with these pilot courses, the SNS, comprising neurosurgery residency directors, department chairs, and other educational leaders, undertook development of a formal “boot camp” curriculum for entering neurological surgery PGY1 residents. This curriculum is provided annually in July at 6 regional centers to incoming residents in all ACGME-accredited US residency programs. In the first year of the courses (2010), 94% of entering residents participated. Subsequently, participation has been 100%.20,21
Details of the curriculum development process are described elsewhere.20 Procedural skills chosen for simulation included those commonly performed by US neurosurgical residents during intensive care rotations in the PGY1 year, as judged by members of the Committee on Resident Education, Subcommittee on Resident Courses of the SNS. The Council and membership of the SNS validated this curriculum. The Residency Review Committee for neurosurgery of the ACGME contemporaneously developed a similar list that was incorporated into curricular requirements for accredited US neurosurgical training programs.22 The ACGME requires structured procedural education “equivalent to that available through the boot camps offered by the Society of Neurological Surgeons” before progression to independent supervision in the live clinical environment.23 The SNS boot camp procedures now include (1) ventriculoperitoneal shunt tap and valve programming; (2) lumbar puncture and drain placement; (3) intracranial pressure (ICP) monitor placement; (4) external ventricular drain placement; (5) central and arterial line placement; (6) clinical crisis management scenarios (these 6 use model-based simulations); (7) microscope safety and instrument use, and (8) surgical positioning. The surgical skills included in the boot camp curriculum included the individual components of routine trauma craniotomies such as evacuation of a subdural hematoma: (1) drilling and bone dissection, (2) cranial flap creation, (3) dural opening and closure, (4) craniotomy flap fixation, (5) cranioplasty, and (6) skin closure.
Available clinical equipment and relevant anatomic models were obtained for use in the boot camp courses when available. Additional models were developed and improved on each year on the basis of faculty suggestions and resident and faculty postcourse evaluations.19-21 Course directors created a boot camp resource guide for preparatory learning and review that includes specific enumeration of each step necessary to prepare for and carry out the procedures and to avoid and manage complications related to them; this guide is used as a guide for simulation participation during the courses.24
Immediate postcourse surveys of all resident attendees were conducted with Survey Monkey (www.surveymonkey.com), as previously described.20 Each survey included the following question for each surgical and procedural skill station: “Quality of hands on materials? (Excellent, Very Good, Good, Fair, Poor).” The percentage of all survey respondents in each course year (2010, 2011, or 2012) who answered “excellent” for each surgical and procedural simulation station was compared by use of a repeated measures analysis of variance that contrasted those stations undergoing or not undergoing annual, iterative simulation model improvement. In addition, the magnitude of improvement for stations undergoing or not undergoing iterative improvement from 2010 to 2012 was compared by use of a 2-tailed t test.
Similarly, other organizations within North American neurological surgery have developed model-based and computer-aided procedural simulations for early learners in neurosurgical residency. These approaches are also discussed here.
A systematic national curriculum using model-based simulation of neurosurgical procedures, basic cranial neurosurgical operative skills, and critical decision making was taught to 591 entering PGY1 residents in the United States between 2010 and 2012 and continues each July. The curriculum combines pedagogy specific to early procedural learners with simple simulation models that break down complex processes into digestible components appropriate for the learning stage.
Model-based simulation can only imperfectly reproduce the visual and haptic feedback and mechanical constraints of live surgical procedures. Individual physical maneuvers that represent critical and/or risky steps of the larger procedural intervention such as drilling and craniotomy formation, however, may be reproduced in their fundamental aspects by relevant models. Drilling to create a burr hole or craniotomy flap is a critical skill for early neurosurgical learners but poses an inherently high risk of injury to underlying neural tissue. At the SNS boot camp courses, standard clinical, high-speed electric craniotomy drills and bits (Stryker, Kalamazoo, Michigan) are used to perform bone dissection on a freeze-dried beef scapula (Figure 1). This model material is very low cost and does not pose the health risks or regulatory burdens of using human cadaver material.
Simulation based on physical models is also of relatively limited value in re-creating a high-fidelity experience of an entire procedural or operative intervention but may re-create key or challenging components.25 For example, physical models may be very useful in simulating the individual components of surgical procedures such as dural repair (Figure 2). The initial dural repair simulation developed for the SNS boot camp courses used a thin sheet of green rubber, representing dura, stretched over a soft plastic “cerebral hemisphere.” Iterative improvement of the model between subsequent course years evolved to the use of a plastic cranium, suitable for drilling and model craniotomy, with an inserted sheet of rubber artificial dura. The thickness and color of the rubber sheet now more closely match those of natural dura. Tacking sutures may be inserted, and the bony edges re-create the mechanical limitations of closing dura within a craniotomy opening. Dural repair is carried out with a sheet of clinically available dural substitute material and 4-0 braided nylon suture (Stryker).
The SNS boot camp courses have also included other basic trauma craniotomy skills, including plate fixation of craniotomy flaps, cranioplasty, and skin closure. The craniotomy fixation skill simulation uses commercially available artificial skull models and clinically available fixation systems (Stryker; not shown). The cranioplasty skill simulation uses, in addition, clinically available bone cement (BoneSource BVF, Stryker; not shown). These simulated skill exercises have not appreciably changed in curriculum or materials since they were first introduced.
Finally, the craniotomy skills simulation includes skin closure (not shown). Initially, in 2010, the skin closure simulation involved commercially available kits containing artificial skin, disposable instruments, and suture material (Ethicon, Inc, Cornelia, Georgia). These kits have been improved for subsequent courses to include a skin model specifically representing scalp with relevant closure suture exercises.
Basic skills training for incoming PGY1 surgical residents may, in many cases, take advantage of well-established, validated, and widely available physical models for routine procedures used across surgical and medical specialties such as central venous access.26 The SNS boot camp courses, like many of the general surgical PGY1 courses they replaced, use upper-torso manikins with simulated venous and arterial anatomy for teaching and assessing central venous catheter placement skills (Figure 3). In 2010, only 3 courses used formal central line placement model simulators. In 2011 and 2012, all courses used these validated models, markedly improving resident course attendee's quality assessment of the hands-on materials for this skill simulation (Table).
In other cases, simulators designed for simple and widely performed procedures such as lumbar puncture may be adapted for more complex and specialty specific procedures such as lumbar drain placement. During a pilot course undertaken for boot camp curricular development, lumbar drain placement safety was taught by reviewing the anatomic landmarks used for the procedure and the proper sequencing and connecting of the various insertion kit components, which were reviewed and physically handled (Figure 4). By the time the national courses were initiated, lumbar drain placement teaching used a commercially available lower-torso lumbar spinal surgery model, which was designed for teaching surgical anatomy but not for lumbar puncture. Subsequently, the courses have used a commercially available lumbar puncture model (Kyoto Kagaku Co, Ltd, Kyoto, Japan) that provides flow of artificial cerebrospinal fluid after the interlaminar space is successfully entered and allows threading the lumbar catheter through the needle into the model thecal sac. The learner can perform the procedure while visualizing the internal bony anatomic landmarks or, as in the live clinical environment, with those landmarks hidden from view by artificial skin.
Similarly, components of model-based simulations used for entirely different purposes may also be adapted to create a simple neurosurgical procedural simulator. During the initial boot camp courses, the ventriculoperitoneal shunt tap procedure was taught by reviewing the typical location of shunt valves relative to anatomic landmarks and tapping a naked shunt reservoir with attached tubing and saline flow (Medtronic, Inc, Minneapolis, Minnesota; Figure 5). Subsequently, a shunt system with continuous flow of saline has been used, with the valve covered by artificial scalp (Ethicon, Inc). Learners can palpate and tap the valve with proprioceptive feedback similar to that of the actual clinical procedure and visualize the relationship between palpated landmarks and the orientation and components of the valve and reservoir.
At the time the boot camp pilot and subsequent national courses were developed, sophisticated, computer-controlled, and haptic ventriculostomy placement simulators were commercially available but too expensive for use in these resident courses. Therefore, during the pilot course, ventriculostomy placement teaching involved reviewing anatomic landmarks with a skull model and teaching the proper sequencing and connecting of the various insertion kit components (Figure 6). By the time the national courses were initiated, ventriculostomy teaching stations used a frameless navigation system with electromagnetic tracking (Stealth Station S7 with AxiEM, Medtronic Inc), preloaded with computed tomographic imaging of “normal” and “large” ventricles, plus a hollow skull model created by resin stereolithography (Medtronic, Inc). Although frameless navigation systems are also expensive technology, they are nearly universally available in the academic hospitals and surgical teaching centers at which resident courses are taught and have been loaned free of cost by surgical technology vendors.20
Furthermore, iterative improvement of ventriculostomy placement simulation has been possible within the structure of the boot camp courses each year. Currently, the station uses a more refined stereolithography-generated resin skull model with a polished and clear-coated surface that is compatible with a dry erase marker (Medical Modeling, Inc, Golden, Colorado), so each learner can be tested on anatomic landmarks for burr hole placement before catheter passage. The model has a built-in electromagnetic fiducial to interact with the AxiEM tracking system. The resin skull model itself is lined with a 3-mm-thick silicone sheet to simulate dural puncture and filled with 5-mm-diameter BBs (Crosman Airsoft BBs, Crosman, Inc, Bloomfield, New York), re-creating the consistency of brain tissue, to simulate catheter passage with higher fidelity (Figure 6).
The ICP monitor placement procedural simulation uses a plastic skull or skull cap, access drill, and clinically available ICP bolt monitoring system (Camino; Integra LifeSciences Corp, Plainsboro, New Jersey) to train residents to insert, secure, and “zero” the monitoring system. There have not been major changes to this simulation during the courses.
Decision-making and Behavioral Simulation
Early learners in surgical training are likely to be supervised initially for most procedural and all operative interventions.27 In contrast, these early learners may face a need for critical decision making about important clinical events very early in training. In the SNS boot camp courses, clinical models have also been used to support critical decision-making and behavioral simulation (Figure 7). Various commercial systems enable simulation of critical decision making with assessment of trainee knowledge and behavior in response to purpose-designed clinical scenarios. The boot camp courses have used a scenario involving sudden neurological deterioration in the emergency department caused by a rapidly expanding epidural hematoma. PGY1 surgical residents were provided a brief history of a patient who fell off a ladder sustaining a head injury. Each learner was expected to develop a differential diagnosis related to a traumatic closed-head injury, to recognize the development of elevated ICP, and to suggest management strategies related to the airway/circulation, reduction of ICP, and calling for supervisor/attending help. Commercially available manikin simulators used for this teaching are capable of modeling verbal, physiological, and even some neurological responses (eg, pupillary dilatation; Sim Man, Laerdal Medical, Wappinger Falls, New York). Thus, simulators initially designed to teach more universal skills such as resuscitation and endotracheal intubation have been adapted for training neurosurgical residents.
Immediate postcourse surveys were answered by nearly all resident attendees (186 of 186 in 2010, 100%; 187 of 201 in 2011, 93%; 164 of 206 in 2012, 80%). The percentage of resident attendee respondents who described the model-based simulation hands-on materials for each surgical or procedural skills station as “excellent” is given for each of the 3 course years in the Table. Those procedural and surgical skills stations that underwent specific iterative improvements in their model-based simulations between 2010 and 2012 showed improvement in rankings, including the dural and skin closure surgical skills simulations and the ventriculoperitoneal shunt tap, lumbar drain, external ventricular drain, and central line placement procedural skills simulations, whereas other stations did not [analysis of variance F(2,18) = 5.567; P < .02]. The average magnitude of improvement in “excellent” ratings for simulation model hands-on materials was 15% for those stations undergoing iterative improvement and no change (0%) for other stations (P < .01).
The value of model-based simulation in PGY1 neurosurgical residents was assessed by surveys of resident learners and faculty attending the 2010 boot camp courses.20,21 Residents most highly valued those simulations of procedures that they were called on more frequently to perform during their PGY1 year.21 All simulations, but particularly those related to neurosurgical operative interventions and neurosurgery-specific intensive care unit procedures, were highly valued by resident learners.20
Validated assessments of procedural simulation are important to assess the pedagogy itself, to gauge the performance of the resident, and to assess the resident's ability to progress to subsequent training stages.28,29 In general, validated assessments are often lacking from surgical simulation models. In 2012, the SNS boot camp courses introduced pilot, checklist-based technical assessments for 3 course procedural stations: ventriculostomy, ICP monitor, and lumbar drain placement. As part of the ongoing process of iterative course improvement, these assessments are being validated and broadened to additional course components.
Simulation training in neurosurgery is particularly challenging because of the wide variety of anatomic approaches and pathology routinely dealt with in the field.30,31 Nevertheless, early learners appear to particularly benefit from model-based simulation designed to teach common neurosurgical procedures or narrow skills components of more complex operative interventions that are commonly performed by residents in their initial clinical years.
Model-based simulation for early neurosurgical learners benefits from a defined curriculum, within which each technical or procedural skill gains relevance. For example, in the approach used by the SNS boot camp courses, each PGY1 resident is exposed to a simulated manikin patient that experiences sudden deterioration in the emergency department resulting from an expanding traumatic intracranial hematoma. In the skills laboratory, the resident is tasked with practicing and then performing the various component skills of a standard hemispheric craniotomy for traumatic extra-axial hematoma evacuation, including bone drilling, craniotomy formation, dural tack-up placement and closure, and skin closure.
The boot camp courses effort has produced significant positive results in its first 3 years of national attendance.20,21 Despite these successes, further work is necessary to ensure a fully robust educational process. Although challenging and time intensive, the creation of validated assessments for each fundamental skill must be undertaken. This complexity and effort are justified by the scale of the enterprise and its reach across the breadth of US residency training.
Various other educational organizations that include early learners use simulation in neurosurgical teaching. The Congress of Neurological Surgeons simulation initiative has created, refined, or adopted the use of various procedural and surgical simulators and aggregated their use into simulation courses for neurosurgical residents, allowing intensive experience relevant to multiple procedures during attendance at national neurosurgical meetings. An important component of simulator development by this program has been the creation of validated assessment instruments.32 The American Association of Neurological Surgeons has created a recurring competitive event for residents using ventriculostomy and pedicle screw placement simulations, the “Top Gun” program, which also generates numeric accuracy assessments for each resident performing these simulated procedures.33
Procedural simulation for early learners is particularly useful as part of an overarching and validated residency curriculum, with explicit tracking of educational outcomes. In this context, the SNS, on behalf of residency program directors and in collaboration with the neurosurgery Residency Review Committee and ACGME, has formulated a “Matrix” curriculum for US neurosurgery residency training. The SNS is currently working to coordinate that curriculum, including its simulated and live procedural components, with the ACGME milestones instrument, which is required for educational outcomes aggregation and reporting.34,35
More synthetic simulation of complex neurosurgical procedures will continue to benefit from recent advances in computational processing.30,36 In some cases, simulation can re-create the normal and pathological anatomy of individual patients using their preoperative, 3-dimensional imaging, allowing experienced neurosurgeons to plan or practice complex cases in advance of performing them in the operative environment.37
Several simulation modules have been used in endovascular neurosurgery, which is particularly suited to the use of high-fidelity simulators with haptic feedback capability and visual interfaces.38 These systems enable realistic simulation of intracranial and extracranial aneurysm coiling, stent placement, liquid embolization, and stroke thrombolysis.39 Training with endovascular simulators improves subsequent simulator-based skills testing.40-42 Simulator-trained residents may also perform better in the live clinical environment, for example, scoring significantly higher on a validated rating scale than nontrained residents while performing a pair of supervised endovascular procedures for occlusive vascular disease.43
Simulators are now becoming available in various subspecialty areas of neurosurgical practice. Spinal neurosurgeons have developed various model-based simulations of bony anatomy for the placement of pedicle screws and other spinal instrumentation. Virtual reality simulators are also used for 3-dimensional reconstruction of spinal anatomy when simulating or planning actual pedicle screw placement.44-46 More recently, simulators have been developed for laminoplasty and dural repair.47 Model-based simulators for more complex spinal pathology, including scoliosis, degenerative disease, spinal stenosis, and deformity, are in development.32
Many lesions of the cranial base demand complex surgical approaches involving intricate anatomy comprising various delicate, vulnerable structures such as vasculature and cranial nerves. Three-dimensional cranial base models are now available for both simulation and teaching of these approaches.48 Additional simulators for cranial base surgery, currently under development, will include haptic feedback for drilling and soft tissue manipulation, as well as endoscopic approaches.49
There are a number of additional benefits of developing relevant neurosurgical simulation models. First, particularly simple and inexpensive model-based simulation methods may be accessible and scalable to international training programs in less resource-rich countries. Thus, simulation development in North America may aid the growth and development of endogenous training mechanisms in areas of the world currently with inadequate access to neurosurgical care. In addition, model-based simulation in North American residency programs, if offered to medical students, may increase their interest in surgical careers.50
Early neurosurgical learners benefit from procedural and surgical skills simulations, including those based on relatively simple model systems using available materials and technology. Simulation is most effective when used to accomplish specific goals within a larger, validated curriculum. The development of relevant model-based simulators has been possible using an iterative improvement process in the context of national boot camp courses for residents entering neurosurgical training in the United States, as well as in a number of related settings. Validation of simulation assessment measures is a principal ongoing challenge for continued development of simulation in neurosurgical education. Validated assessments will also assist in more accurately evaluating the impact of iterative simulation model improvements.
A podcast related to this article can be accessed online (http://links.lww.com/NEU/A573).
Dr Selden is chair of the Committee on Resident Education of the SNS and a member of the executive committee of the Congress of Neurological Surgeons. Dr Origitano serves as a member of the Committee on Resident Education Subcommittee on Resident Courses, SNS. Dr Hadjipanayis is codirector of the Southeast Region SNS boot camp course. Dr Byrne is chair of the Committee on Resident Education Subcommittee on Resident Courses and a member of the executive committee of the Congress of Neurological Surgeons. No author has any financial or proprietary interest in the courses or any course materials. The SNS resident courses are sponsored by educational grants to the SNS from Stryker, Integra, and Synthes, plus additional in-kind donations from Medtronic, Ethicon, Teleflex, and Zeiss. The Congress of Neurological Surgeons and American Association of Neurological Surgeons serve as administrative sponsors for the SNS boot camp and SNS junior resident course, respectively. No financial or material support is given to any SNS resident course directors or faculty. Only funds necessary for the direct administration of the courses and expense reimbursements are distributed or retained.
We thank Shirley McCartney, PhD, for assistance with the manuscript and Andrew Rekito, MS, for assistance with the figures. We thank the leadership and membership of the SNS for sponsorship of the boot camp courses and the 6 dozen or so course directors and faculty who participate each year. We acknowledge assistance with development and provision of certain materials for the external ventricular drain and lumbar drain model simulations of Andrew Koert and Tom Poss (Medtronic, Inc). We also acknowledge the important contributions to the early development and curriculum of the boot camp courses by the late Dr Christopher Getch.
1. Fitzgibbons SC, Chen J, Jagsi R, Weinstein D. Long-term follow-up on the educational impact of ACGME duty hour limits: a pre-post survey study. Ann Surg. 2012;256(6):1108–1112.
2. Fletcher KE, Nickoloff S, Whittle J, Jackson JL, Frank M, Schapira MM. Why residents consider working beyond the duty hour limits: implications of the ACGME 2011 Duty Hour Standards. J Grad Med Educ. 2011;3(4):571–573.
3. Philibert I, Nasca T, Brigham T, Shapiro J. Duty-hour limits and patient care and resident outcomes: can high-quality studies offer insight into complex relationships? Annu Rev Med. 2013;64:467–483.
4. Wachter RM, Pronovost PJ. The 100,000 Lives Campaign: a scientific and policy review. Jt Comm J Qual Patient Saf. 2006;32(11):621–627.
6. Danzer E, Dumon K, Kolb G, et al.. What is the cost associated with the implementation and maintenance of an ACS/APDS-based surgical skills curriculum? J Surg Educ. 2011;68(6):519–525.
9. McLachlan JC, Patten D. Anatomy teaching: ghosts of the past, present and future. Med Educ. 2006;40(3):243–253.
10. Meller G, Tepper R, Bergman M, Anderhub B. The tradeoffs of successful simulation. Stud Health Technol Inform. 1997;39:565–571.
11. Chaudhry A, Sutton C, Wood J, Stone R, McCloy R. Learning rate for laparoscopic surgical skills on MIST VR, a virtual reality simulator: quality of human-computer interface. Ann R Coll Surg Engl. 1999;81(4):281–286.
12. Coates PJ, Zealley IA, Chakraverty S. Endovascular simulator is of benefit in the acquisition of basic skills by novice operators. J Vasc Interv Radiol. 2010;21(1):130–134.
13. Duffy AJ, Hogle NJ, McCarthy H, et al.. Construct validity for the LAPSIM laparoscopic surgical simulator. Surg Endosc. 2005;19(3):401–405.
14. Grantcharov TP, Kristiansen VB, Bendix J, Bardram L, Rosenberg J, Funch-Jensen P. Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg. 2004;91(2):146–150.
15. Hyltander A, Liljegren E, Rhodin PH, Lönroth H. The transfer of basic skills learned in a laparoscopic simulator to the operating room. Surg Endosc. 2002;16(9):1324–1328.
16. Sherman V, Feldman LS, Stanbridge D, Kazmi R, Fried GM. Assessing the learning curve for the acquisition of laparoscopic skills on a virtual reality simulator. Surg Endosc. 2005;19(5):678–682.
17. Banerjee PP, Luciano CJ, Lemole GM Jr, Charbel FT, Oh MY. Accuracy of ventriculostomy catheter placement using a head- and hand-tracked high-resolution virtual reality simulator with haptic feedback. J Neurosurg. 2007;107(3):515–521.
18. Aucar JA, Groch NR, Troxel SA, Eubanks SW. A review of surgical simulation with attention to validation methodology. Surg Laparosc Endosc Percutan Tech. 2005;15(2):82–89.
19. Selden NR, Barbaro N, Origitano TC, Burchiel KJ. Fundamental skills for entering neurosurgery residents: report of a Pacific region “boot camp” pilot course, 2009. Neurosurgery. 2011;68(3):759–764.
20. Selden NR, Origitano TC, Burchiel KJ, et al.. A national fundamentals curriculum for neurosurgery PGY1 residents: the 2010 Society of Neurological Surgeons boot camp courses. Neurosurgery. 2012;70(4):971–981.
21. Selden N, Anderson V, McCartney S, Origitano TC, Burchiel KJ, Barbaro N. Society of Neurological Surgeons boot camp courses: knowledge retention and relevance of hands on learning after 6 months of postgraduate year 1 training [published online ahead of print April 6, 2013]. J Neurosurg. doi:10.3171/2013.3.JNS122114.
25. Duschek N, Assadian A, Lamont PM, et al.. Simulator training on pulsatile vascular models significantly improves surgical skills and the quality of carotid patch plasty. J Vasc Surg. 2013;57(4):1148–1154.
26. Dong Y, Suri HS, Cook DA, et al.. Simulation-based objective assessment discerns clinical proficiency in central line placement: a construct validation. Chest. 2010;137(5):1050–1056.
28. Fernandez GL, Page DW, Coe NP, et al.. Boot cAMP: educational outcomes after 4 successive years of preparatory simulation-based training at onset of internship. J Surg Educ. 2012;69(2):242–248.
29. Sutton E, Chase SC, Klein R, et al.. Development of simulator guidelines for resident assessment in flexible endoscopy. Am Surg. 2013;79(1):14–22.
30. Malone HR, Syed ON, Downes MS, D'Ambrosio AL, Quest DO, Kaiser MG. Simulation in neurosurgery: a review of computer-based simulation environments and their surgical applications. Neurosurgery. 2010;67(4):1105–1116.
31. Spiotta A, Schlenk R. Simulation in neurosurgical residency training: a new paradigm. CNS Q. 2011;18–20.
32. Harrop J, Sharan A, Traynells V. Spine simulation. CNS Q. 2011;12–13.
33. Yudkowsky R, Luciano C, Banerjee P, et al.. Practice on an augmented reality/haptic simulator and library of virtual brains improves resident's ability to perform a ventriculostomy. Simul Healthc. 2013;8(1):25–31.
34. Selden NR, Abosch A, Harbaugh RE, et al.. Neurosurgery milestones. J Grad Med Educ. In press.
35. Swing S, Beeson M, Carraccio C, et al.. Educational milestone development in 7 specialties entering the next accreditation system. J Grad Med Educ. In press.
36. Choudhury N, Gélinas-Phaneuf N, Delorme S, Del Maestro R. Fundamentals of neurosurgery: virtual reality tasks for training and evaluation of technical skills [published online ahead of print November 23, 2012]. World Neurosurg. doi:10.1016/j.wneu.2012.08.022.
37. Mori K, Yamamoto T, Nakao Y, Esaki T. Development of artificial cranial base model with soft tissues for practical education: technical note. Neurosurgery. 2010;66(6 suppl operative):339–341.
38. Tsang JS, Naughton PA, Leong S, Hill AD, Kelly CJ, Leahy AL. Virtual reality simulation in endovascular surgical training. Surgeon. 2008;6(4):214–220.
39. Aoun S, Bendok B, Mocco J, Levy EI. Simulation in vascular neurosurgery. CNS Q. 2011;9–11.
40. Aggarwal R, Black SA, Hance JR, Darzi A, Cheshire NJ. Virtual reality simulation training can improve inexperienced surgeon's endovascular skills. Eur J Vasc Endovasc Surg. 2006;31(6):588–593.
41. Dawson DL, Meyer J, Lee ES, Pevec WC. Training with simulation improves resident's endovascular procedure skills. J Vasc Surg. 2007;45(1):149–154.
42. Hsu JH, Younan D, Pandalai S, et al.. Use of computer simulation for determining endovascular skill levels in a carotid stenting model. J Vasc Surg. 2004;40(6):1118–1125.
43. Chaer RA, Derubertis BG, Lin SC, et al.. Simulation improves resident performance in catheter-based intervention: results of a randomized, controlled study. Ann Surg. 2006;244(3):343–352.
44. Eftekhar B, Ghodsi M, Ketabchi E, Rasaee S. Surgical simulation software for insertion of pedicle screws. Neurosurgery. 2002;50(1):222–223.
45. Klein S, Whyne CM, Rush R, Ginsberg HJ. CT-based patient-specific simulation software for pedicle screw insertion. J Spinal Disord Tech. 2009;22(7):502–506.
46. Lee DH, Lee SW, Kang SJ, et al.. Optimal entry points and trajectories for cervical pedicle screw placement into subaxial cervical vertebrae. Eur Spine J. 2011;20(6):905–911.
47. Anderson P. Surgical simulation: dural repair. CNS Q. 2011;16–17.
48. Mori K, Yamamoto T, Nakao Y, Esaki T. Surgical simulation of cerebral revascularization via skull base approaches in the posterior circulation using three-dimensional skull model with artificial brain and blood vessels. Neurol Med Chir (Tokyo). 2011;51(2):93–96.
49. Prevedello D, Stredney D, Rezal A. Skull base simulators: the evolution of neurosurgical training. CNS Q. 2011;7–8.
50. Lee JT, Qiu M, Teshome M, Raghavan SS, Tedesco MM, Dalman RL. The utility of endovascular simulation to improve technical performance and stimulate continued interest of preclinical medical students in vascular surgery. J Surg Educ. 2009;66(6):367–373.