BACKGROUND: Simulation has been adopted as a powerful training tool in many areas of health care. However, it has not yet been systematically embraced in neurosurgery because of the absence of validated tools, assessment scales, and curricula.
OBJECTIVE: To use our validated microanastomosis module and scale to evaluate the effects of an educational intervention on the performance of neurosurgery residents at the 2012 Congress of Neurological Surgeons Annual Meeting.
METHODS: The module consisted of an end-to-end microanastomosis of a 3-mm vessel and was divided into 3 phases: (1) a cognitive and microsuture prelecture testing phase, (2) a didactic lecture, and (3) a cognitive and microsuture postlecture testing phase. We compared resident knowledge and technical proficiency from the pretesting and posttesting phases.
RESULTS: One neurosurgeon and 7 neurosurgery residents participated in the study. None had previous experience in microsurgery. The average score on the microsuture prelecture and postlecture tests, as measured by our assessment scale, was 32.50 and 39.75, respectively (P = .001). The number of completed sutures at the end of each procedure was higher for 75% of participants in the postlecture testing phase (P = .03). The average score on the cognitive postlecture test (12.75) was significantly better than that of the cognitive prelecture test (8.38; P = .001).
CONCLUSION: Simulation has the potential to enhance resident education and to elevate proficiency levels. Our data suggest that a focused microsurgical module that incorporates a didactic component and a technical component can enhance resident knowledge and technical proficiency in microsurgical anastomosis.
ABBREVIATIONS: NOMAT, Northwestern Objective Microanastomosis Assessment Tool
OSATS, objective structured assessment of technical skill
PGY, postgraduate year
*Department of Neurological Surgery and
¶Department of Radiology, Northwestern University, Feinberg School of Medicine, Chicago, Illinois;
‡Department of Neurological Surgery, University of Texas Southwestern, Dallas, Texas;
§Department of Neurological Surgery, Thomas Jefferson University, Philadelphia, Pennsylvania
Correspondence: Bernard R. Bendok, MD, MSCI, FACS, FAANS, FAHA, Department of Neurological Surgery, Northwestern University Feinberg School of Medicine, 676 N St. Clair, Ste 2210, Chicago, IL 60611. E-mail: email@example.com
Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s Web site (www.neurosurgery-online.com).
Received April 23, 2013
Accepted July 11, 2013
In November 1999, the Institute of Medicine released its landmark report, To Err is Human: Building a Safer Health System.1 The report suggested that between 44 000 and 98 000 deaths annually in the United States occur as a result of medical errors that could have been prevented.1 Since this report, there has been greater concern among the public and healthcare professionals regarding patient safety. It is suggested that higher levels of proficiency should be obtained by raising performance standards. Simultaneous to this enhanced focus on patient safety, a number of strains have arisen on neurosurgical training. A shift of care from tertiary medical centers to community hospitals, the decreased resident work hours, and the increasingly multimodal nature of procedural care have made it more challenging for residents or even practitioners to master surgical techniques. On the basis of the success in other industries, simulation has been proposed as a mechanism to overcome the challenges of surgical training and to help students and practitioners achieve greater proficiency and reduce patient risk.2-8 According to the 2011 statement of the Association of American Medical Colleges, “simulation has the potential to revolutionize healthcare and address patient safety issues if appropriately utilized and integrated into the educational and organizational improvement process.”9,10
Simulation has become an important component of medical student and resident education in many subspecialties of medicine but has not achieved mainstream acceptance in neurosurgery because of a lack of validated tools, assessment methods, and curricula. As a step toward such validation, we created a surgical microanastomosis module and a corresponding objective assessment scale to quantify and follow surgical performance of neurosurgical residents. The module includes pretesting of knowledge and technical skills, followed by a didactic lecture and then posttesting of both knowledge and technical skills. This module was incorporated into the 2012 Congress of Neurosurgical Surgeons Annual Simulation Course. Here, we report on how this module impacted microsurgical knowledge and skill in course participants.
Surgical Microanastomosis Module
An objective assessment scale, derived from the objective structured assessment of technical skill (OSATS), was developed at our institution to measure surgical skill in a neurosurgical microanastomosis module. The scale, known as the Northwestern Objective Microanastomosis Assessment Tool (NOMAT), consists of 14 surgical “metrics” that were carefully selected and defined by 2 experienced vascular neurosurgeons (B.R.B. and H.H.B.) and a postdoctoral research fellow (S.G.A.; see Supplemental Digital Content 1, http://links.lww.com/NEU/A572, which shows the NOMAT scale). These metrics cover the most important technical aspects of the microanastomosis, including operator posture, use of the microscope, instrument and tissue handling, and quality of stitches, as well as an evaluation of the end result on and off a pump. Each surgical metric is independently graded on a scale of 1 to 5 in which grade 1 represents poor performance and grade 5 represents excellent performance. Of note, grades 1, 3, and 5 of each item are strictly defined, creating 3 solid points for an objective assessment of the surgical skill. On the other hand, grades 2 and 4 of each item remain undefined to allow some grading flexibility, particularly in cases when the surgical skill does not meet the criteria defined for grades 1, 3, and 5. The NOMAT scale has a minimum score of 14 and a maximum score of 70. Construct validity and face validity were demonstrated in an internal study performed at our institution that involved medical students, research fellows, and neurosurgery residents (manuscript in preparation). The simulation module consisted of an end-to-end microanastomosis of 3-mm synthetic vessels (Kezlex; Japan) using a microsurgical kit (jewelers, Lawton needle holders, and microscissors), 8-0 nylon microsurgical sutures, and Zeiss microscopes. A fluid pump was also used to assess flow.
Our microanastomosis module was incorporated into the simulation-based neurosurgical training course held at the 2012 Congress of Neurosurgical Surgeons Annual Meeting. Both national and international neurosurgical residents attended this course and participated in the microanastomosis training module (Figure 1). During the course, prelecture and postlecture microsurgical knowledge and technical skills were assessed to determine the impact of this module on relevant knowledge and skills. The module was divided into 3 phases: (1) a cognitive and practical prelecture testing phase, (2) a didactic lecture, and (3) a cognitive and practical postlecture testing phase. Three stations were set up, each containing an operative microscope, a full microsurgical kit, two 3-mm synthetic vessels, 8-0 nylon sutures, and a pump.
Initially, the subjects performed a cognitive prelecture test, which consisted of 15 multiple-choice questions assessing general knowledge about bypass surgery. This was followed by a practical prelecture test in which each participant was asked to perform a 3-mm vessel end-to-end microanastomosis within 20 minutes. Three instructors evaluated the performance of the participants using a 15-point score for the cognitive prelecture test and the NOMAT scale for the practical prelecture test. After the prelecture testing phase, the subjects received a 30-minute didactic lecture presented by the senior author (B.R.B.) discussing basic knowledge and technical nuances of bypass surgery and end-to-end microanastomosis. The technique of end-to-end microanastomosis was illustrated in pictures and videos highlighting different technical aspects and surgical nuances. The lecture also illustrated the use of surgical instruments under the operative microscope. After the lecture, the subjects repeated the cognitive test and the technical exercise.
We compared cognitive and technical performances before and after the didactic lecture in a group of 8 subjects. The aim of the analysis was to determine whether a statistically significant difference can be seen between the prelecture and postlecture test scores. The data set was analyzed with a paired Student t test. The statistical analysis was performed with SPSS 22.214.171.124 (IBM) for OS X (Mac).
A total of 8 subjects participated in the course: 7 neurosurgery residents and 1 neurosurgery faculty. The participating subjects originated from 6 different countries, including the United States. The postgraduate year (PGY) level of residents ranged between PGY-2 and PGY-5; all residents had no previous experience in microvascular surgery. Subject characteristics are summarized in Table 1. None of the subjects were able to complete the technical exercise within the 20-minute time frame in both the prelecture and postlecture testing phases. Nevertheless, the number of completed sutures at the end of each procedure was higher for 75% of participants in the postlecture testing phase (P = .03; Table 2). The average score on the practical prelecture and postlecture tests, as measured by the NOMAT scale, was 32.50 (range, 21-46) and 39.75 (range, 26-49), respectively. This increase was found to be statistically significant (P = .001; Table 3). The practical test scores of all subjects are illustrated in Figure 2. The average score on the cognitive postlecture test (12.75; range, 10-15) was significantly better than that of the cognitive prelecture test (8.38; range, 6-12; P = .001; Table 4). The cognitive test scores of all subjects are illustrated in Figure 3.
A recent survey illustrates the support of neurosurgery program directors for incorporating simulation into resident training.11 The absence of validated simulation tools, curricula, and assessment tools, however, has hindered this important step. We chose microanastomosis because it is both a fundamental skill in neurosurgery and a skill that is very difficult to master in neurosurgical training programs. It is conceivable that enhanced proficiency with microanastomosis skills could translate into greater microsurgical skills in general.12 The road to achieving mastery through formal instruction and practice is described by the Dreyfus model of skill acquisition.13 Five stages of increasing skill are defined: (1) novice, (2) advanced beginner, (3) competent, (4) proficient, and (5) expert. The goal of resident training is to achieve at least proficiency level on the Dreyfus scale. One of the challenges of using this scale is that it does not provide a tool to objectively assess or quantify the skill level.
The OSATS scale is one of the first evaluation tools designed to objectively assess surgical skill.14,15 This scale was validated for use in general surgery in the mid-1990s14,15 and is currently being modified and adapted in surgical subspecialties such as cardiothoracic surgery and plastic surgery.3,4,6,16,17 The NOMAT is an OSATS-derived global rating scale designed to assess skill in a neurosurgical microanastomosis module. It consists of 14 items or metrics that were carefully selected by 2 experienced vascular neurosurgeons (H.H.B. and B.R.B.) and a postdoctoral research fellow (S.G.A.; Supplemental Digital Content 1). The items reflect the main features examined by the OSATS such as operator respect for tissue, time, and motion; knowledge of instruments; instrument handling; and forward planning.14 Moreover, they cover the most important technical aspects related to the microanastomosis, including needle handling and care, needle bite uniformity, spacing of sutures, knot-tying efficiency, and the quality of knots and the anastomosis (off-pump, on-pump, and lumen). The NOMAT scale was shown to have construct validity and face validity in an internal study performed at our institution (manuscript in preparation).
Using the simulation module and the NOMAT scale as tools for assessment of surgical skill, we chose to combine a didactic lecture with a practical, hands-on technical exercise into 1 surgical curriculum to study its effect on the performance of neurosurgical residents. This was based on data suggesting that combining both the cognitive component of a given procedure and the practical, hands-on experience can dramatically increase the educational benefits.7,16 The importance of knowledge in acquiring a new psychomotor skill was described in 1971 by Joseph Kopta,18 who based his work on the previous work of Paul Fitts and Michael Posner.19 Kopta mentioned that, in the first phase of learning, “the learner must intellectualize what he wants to do and then plan the steps necessary to accomplish the task.”18 He emphasized the importance of creating comprehensive surgical curricula for the development of surgical skill.18
Satterwhite et al20 studied the efficacy of a Web-based curriculum on the microanastomosis cognitive knowledge and practical technique of plastic surgery residents. They randomized 17 residents of various levels into 2 groups: an experimental group who had access to the Web course for a 1-week period and a control group who did not.20 Both groups underwent a baseline written knowledge test and microanastomosis procedure. After the 1-week period, both groups repeated the written knowledge test and underwent a second microanastomosis session. The results showed that residents who took the online course had significantly greater increases in their cognitive test scores compared with those who did not take the course (17% vs −1%, respectively).20 Additionally, the experimental group experienced significantly greater reductions in their procedural times compared with the control group (−4.5 minutes vs −1.33 minutes, respectively).20 Although the study did not use an objective assessment scale to detect improvement in residents’ surgical performance, the results suggest that creating and implementing structured curricula in surgical resident training can enhance residents’ knowledge of the technical and cognitive aspects of a given procedure.20
In our simulation module, we use synthetic blood vessels to perform the end-to-end microanastomosis. This can be justified by the fact that synthetic vessels are more cost-effective, easier to handle, and more logistically feasible compared with animal models.7,16 Moreover, with the use of synthetic vessels, there is a greater potential for repetitive use, a lower risk of transmission of infectious disease, and fewer ethical concerns compared with animal models.16 In fact, residents can practice on synthetic vessels in call rooms or even in their homes using portable microscopes or loupes. Although animal models carry much higher fidelity and realism compared with synthetic vessel models, acquisition of surgical skill with a lower-fidelity model has been shown to be comparable to that obtained from high-fidelity models, particularly for junior residents.7,16 Virtual reality microvascular simulators appear to be the future of microsurgical simulation; however, they are currently costly and not widely available.21
Our study demonstrated the positive effect of the simulation-based curriculum on the cognitive and technical performance of neurosurgical residents. The majority of participants showed improvement in knowledge related to surgical microanastomosis (Table 4). Moreover, all participants (100%) performed better on the postlecture practical test than on the prelecture practical test (Table 3). Six of 8 participants (75%) completed a higher number of sutures at the end of the postlecture practical test, suggesting faster technique and higher efficiency (Table 2). The positive effect of the surgical curriculum, namely the didactic lecture and the hands-on experience, on the technical performance of residents is further supported by the fact that all participants, of various training levels, had no previous microvascular experience (Table 1). The findings obtained from our study are consistent with results obtained from simulation-based courses designed in other surgical specialties.3,4,6-8,16,20
Despite the established effectiveness of simulation-based surgical curricula, the importance of attending feedback and deliberate practice remains paramount.3,6,8,22 In a report on their boot camp experience, Fann et al6 evaluated the effect of focused training on 33 PGY-1 cardiothoracic surgery residents performing a coronary artery microanastomosis (porcine model). An OSATS-derived objective assessment scale was used to assess surgical skill. While performing the microanastomosis, residents were directly supervised by an expert surgeon who provided constructive feedback on the various technical aspects of the procedure. Their data showed that residents performed significantly better in the middle and at the end of the procedure compared with at the beginning (P < .001).6 The study suggested that deliberate practice, along with constructive feedback from an expert, can improve residents’ surgical performance.6 In another study by Price et al,22 39 PGY-1 and PGY-2 cardiovascular surgical residents were randomized to receive either an expert-guided course on a simulator alone or an expert-guided course on a simulator combined with self-directed, deliberate practice (performing 10 anastomoses). The 2 groups were assessed by 2 blinded expert surgeons using the OSATS scale. The results of the study showed that residents who performed independent, deliberate practice scored significantly higher than those who did not (P = .003).22 Additionally, residents who underwent didactic training performed the procedure much faster (P = .04) and had a better anastomosis end product (P = .02) compared with those who did not.22 The study emphasized the importance of deliberate practice in acquiring surgical skill and suggested implementing repetitive practice into future surgical curricula.22
To detect the ideal training pattern that can lead to a better surgical performance, Moulton et al8 compared the effectiveness of en masse practice (only 1 day of training) with that of time-distributed practice (training on weekly basis). Thirty-eight residents were randomized to receive either en masse or time-distributed training. Residents in both groups practiced the same amount of time (total of 4 training sessions). Results of the study demonstrated that all residents performed better immediately after each session; however, residents in the time-distributed practice group performed significantly better on the final procedure compared to the en masse practice group (P < .05).8 The study recommended implementing deliberate practice into surgical curricula and emphasized the importance of a time-distributed training pattern.8
Surgical performance can positively or adversely affect operative and postoperative complication rates and patient outcomes.23,24 A high level of proficiency at the time of residency graduation in neurosurgery is critical. The growing constraints on postgraduate medical training threaten to reduce the proficiency levels of graduating physicians. Simulation has the potential to enhance resident education and to elevate levels of proficiency. Our data suggest that a focused microsurgical module that incorporates a didactic component and a technical component can enhance resident knowledge and technical proficiency in microsurgical anastomosis. This enhanced skill may translate into improved microsurgical skills.
A podcast related to this article can be accessed online (http://links.lww.com/NEU/A577).
The authors have no personal financial or institutional interest in any of the drugs, materials, or devices described in this article.
The authors thank William Kunkler and Susan Crown, as well as Dr Joseph and Mrs Nadia Tamari for their generous donations and support of the microanastomosis simulation laboratory at Northwestern Medicine.
1. Institute of Medicine. To Err Is Human: Building a Safer Health System. In: Kohn LT, Corrigan JM, Donaldson MS, eds. Washington, DC: National Academy Press; 1999.
2. El Ahmadieh TY, El Tecle NE, Aoun SG, Yip BK, Ganju A, Bendok BR. How can simulation thrive as an educational tool? Just ask the residents. Neurosurgery. 2012;71(6):N18–N19.
3. Nesbitt JC, St Julien J, Absi TS, et al.. Tissue-based coronary surgery simulation: medical student deliberate practice can achieve equivalency to senior surgery residents. J Thorac Cardiovasc Surg. 2013;145(6):1453–1458.
4. Atkins JL, Kalu PU, Lannon DA, Green CJ, Butler PE. Training in microsurgical skills: does course-based learning deliver? Microsurgery. 2005;25(6):481–485.
5. Aoun SG, McClendon J Jr, Ganju A, Batjer HH, Bendok BR. The Association for Surgical Education’s roadmap for research on surgical simulation. World Neurosurg. 2012;78(1-2):4–5.
6. Fann JI, Calhoon JH, Carpenter AJ, et al.. Simulation in coronary artery anastomosis early in cardiothoracic surgical residency training: the boot camp experience. J Thorac Cardiovasc Surg. 2010;139(5):1275–1281.
7. Matsumoto ED, Hamstra SJ, Radomski SB, Cusimano MD. The effect of bench model fidelity on endourological skills: a randomized controlled study. J Urol. 2002;167(3):1243–1247.
8. Moulton CA, Dubrowski A, Macrae H, Graham B, Grober E, Reznick R. Teaching surgical skills: what kind of practice makes perfect? A randomized, controlled trial. Ann Surg. 2006;244(3):400–409.
10. Gaba DM. The future vision of simulation in healthcare. Simul Healthc. 2007;2(2):126–135.
11. Ganju A, Aoun SG, Daou MR, et al.. The role of simulation in neurosurgical education: a survey of 99 United States neurosurgery program directors [published online ahead of print November 24, 2012]. World Neurosurg. 2012. doi: 10.1016/j.wneu.2012.11.066. Accessed May 15, 2013.
12. Crosby NL, Clapson JB, Buncke HJ, Newlin L. Advanced non-animal microsurgical exercises. Microsurgery. 1995;16(9):655–658.
13. Dreyfus HC, Dreyfus SE. A Five-Stage Model of the Mental Activities Involved in Directed Skill Acquisition. Bolling AFB, Washington, DC: US Air Force Office of Scientific Research; 1980.
14. Martin JA, Regehr G, Reznick R, et al.. Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg. 1997;84(2):273–278.
15. Faulkner H, Regehr G, Martin J, Reznick R. Validation of an objective structured assessment of technical skill for surgical residents. Acad Med. 1996;71(12):1363–1365.
16. Grober ED, Hamstra SJ, Wanzel KR, et al.. The educational impact of bench model fidelity on the acquisition of technical skill: the use of clinically relevant outcome measures. Ann Surg. 2004;240(2):374–381.
17. Wanzel KR, Matsumoto ED, Hamstra SJ, Anastakis DJ. Teaching technical skills: training on a simple, inexpensive, and portable model. Plast Reconstr Surg. 2002;109(1):258–263.
18. Kopta JA. The development of motor skills in orthopaedic education. Clin Orthop Relat Res. 1971;75:80–85.
19. Fitts PM, Posner MI. Human Performance. Belmont, CA: Brooks/Cole Publishing; 1967.
20. Satterwhite T, Son J, Carey J, et al.. Microsurgery education in residency training: validating an online curriculum. Ann Plast Surg. 2012;68(4):410–414.
21. Erel E, Aiyenibe B, Butler PE. Microsurgery simulators in virtual reality: review. Microsurgery. 2003;23(2):147–152.
22. Price J, Naik V, Boodhwani M, Brandys T, Hendry P, Lam BK. A randomized evaluation of simulation training on performance of vascular anastomosis on a high-fidelity in vivo model: the role of deliberate practice. J Thorac Cardiovasc Surg. 2011;142(3):496–503.
23. Batjer HH, Aoun SG, Rahme RJ, Bendok BR. Honoring our public responsibility: creating milestone and matrix-based training in an era of duty hour restrictions. Clin Neurosurg. 2012;59:70–74.
24. McCaslin AF, Aoun SG, Batjer HH, Bendok BR. Enhancing the utility of surgical simulation: from proficiency to automaticity. World Neurosurg. 2011;76(6):482–484.