Surgical skills training has changed dramatically with the adoption of simulation-based approaches to education. This shift from the traditional apprenticeship model is occurring because of growing concerns about patient safety, cost pressures, operating room efficiency, work-hour restrictions, and the need to acquire increasingly complex surgical skills in less time.1-3 With simulation-based courses in dedicated laboratories, trainees can learn surgical skills in a risk-free environment before performing the procedures on patients.
The acquisition of surgical skills through simulation-based training is demonstrated by two key concepts: transferability and retention. Transferability is ascertained when the skills learned by trainees in simulation-based programs are transferred to actual performance in a patient-based setting.4,5 Hence, it is the most effective measure of the predictive validity of simulator training. Evidence from orthopaedic and other surgical specialties indicates that trainees who reach proficiency in a simulation-based program do better in the patient-based setting than do their counterparts without this training.6-10 The transferability of skills is not guaranteed in simulator-based training, however; other factors, such as inherent learning style, the nature and type of feedback, and opportunities for learning reinforcement, also determine whether acquired skills are successfully applied at high levels outside the laboratory.4
Although skill acquisition and transfer are initial steps in the learning process, skill retention is the de facto indicator of later performance. Therefore, learning and retention should be considered together; learning cannot occur without retention.11
Retention of Surgical Skills After Simulation-based Training
According to Edgar Dale’s Cone of Experience, learners retain more information or skills when they “do” a task rather than “hear, read or observe” it12,13 (Figure 1). Learning by doing is the fundamental principle in simulation-based surgical skills training, and the goals are retention and later performance of these skills. Reinforcing retention of learned content can be challenging for educators when creating training programs, however. This challenge is even more prominent in simulation-based surgical skills training, during which the four major modalities of learning—visual, auditory, kinesthetic, and tactile—must be used simultaneously in a coordinated manner.
Skill retention can be arbitrarily divided into three periods: immediate (minutes to hours), short-term (up to 3 months), and long-term (3 months to years). Any learning process, particularly skills training for future surgeons, should promote long-term retention. In the healthcare professions, where human lives are at stake, decay of learned skills can have drastic consequences. Despite this awareness, published evidence regarding skill retention after simulation-based training in orthopaedic and other surgical subspecialties is limited.
State of the Evidence in Orthopaedic Surgery
Few studies in the orthopaedic surgery literature have reported on the durability of simulation-acquired skills. Sonnadara and colleagues13,14 studied retention rates after a 30-day intensive surgical skills course that included basic fracture fixation techniques, application of casts and splints, and familiarization with basic surgical instruments. Immediately after the course, Objective Structured Assessment of Technical Skills standard checklist and global rating scale (GRS) scores were significantly better in the laboratory-trained group (n = 6) than in the standard residency group (n = 16; P < 0.01).13 Six months after the initial assessments, a second assessment was done using the GRS.14 The scores of the group that had participated in the intensive skills course did not significantly decrease (mean GRS immediately after the course, 4.3; mean GRS at 6 months after the course, 4.3). Although the scores of the junior residents who did not participate in the intensive training course (ie, the control group) significantly improved after 6 months (mean GRS at 1 month, 3.4; mean GRS at 7 months, 3.7; P < 0.01), these scores were still significantly lower than those of the residents who took the course (P < 0.001).14
It should be noted that the GRS scores of the control group increased significantly at 6 months as a result of their training, and the difference between the scores of the two groups showed a tendency to decrease.14 This observation raises the question, “Would the scores of the two groups be different if retention rates were tested beyond 6 months after the course?” It is conceivable that after sufficient practice, trainees become skillful regardless of the training technique used. The goal of simulation-based courses should be to make training to proficiency levels quicker and safer than with traditional methods, not necessarily to make surgeons expert in the long term.
Howells et al15 investigated the retention of unfamiliar arthroscopic skills acquired using a simulator in a laboratory. In this study, fellowship-trained lower-limb surgeons (n = 6) with no previous experience in arthroscopic Bankart repair were given standardized instructions for performance of an arthroscopic Bankart suture on a laboratory-based simulator. An electromagnetic motion tracking system recorded the total path length of the surgeon’s hands, the number of hand movements, and the time taken to place the sutures. Although all surgeons improved their performance with repetition on the simulator, after 6 months without practice, they performed as if they were doing the procedure for the first time. Contrary to findings reported by Sonnadara et al,14 the skills learned using an arthroscopy simulator completely decayed after 6 months of latency. Of note, participants in the Sonnadara et al14 study continued routine training on the activities covered in the intensive course without a latency period, emphasizing the importance of regular use of certain skills in maintaining optimal performance levels.
Using a similar research design, Jackson et al16 studied the technically demanding skills required of orthopaedic residents during arthroscopic meniscal repair and the impact of task repetition versus lack of practice on skill retention. The residents had never done arthroscopic meniscal repair and were not allowed to do real-life meniscal repair during the study period. A validated motion analysis tracking system was used to objectively assess the performance of each resident on a knee simulator; outcomes were the time taken, the distance covered, and the number of hand movements. At the end of the initial training phase, the residents were randomly assigned to one of three groups for a 6-month interim phase. Group A (n = 7) performed the meniscal repair task once each month, group B (n = 6) performed the task once at 3 months after the initial training phase, and group C (n = 6) did not perform the task during the 6-month period. All three groups performed meniscal repair tasks for the final assessment. Interestingly, although group C subjects performed no task repetitions during the interim period, their performance did not deteriorate after the 6-month break (P > 0.05). The authors suggested that the retention of arthroscopic skills may have been affected by task-specific and surgical group-specific factors and that generic guidelines on minimum task frequency may not be appropriate for learning or for maintaining optimal performance.
Although results from the previously discussed studies are inconsistent regarding skill retention after a period of nonpractice, it should be noted that the number of residents included in these groups was small, and the interval used for assessing retention rates after initial training was only 6 months. Larger studies with longer follow-up intervals will provide more conclusive evidence regarding long-term retention rates of simulation-acquired skills in orthopaedic surgery.
State of the Evidence in Other Specialties
Simulation-based skills training has been effectively integrated into the educational programs of other surgical specialties. General surgery programs incorporated simulators into the educational curriculum relatively early, with high-quality research studies focusing mainly on training of endoscopic surgery skills.17-22 Other specialties, such as internal medicine, anesthesiology, and obstetrics and gynecology, have investigated long-term retention rates and skill decay after simulation-based training.23-29Table 1 summarizes the findings from some of the studies on skill retention in specialties other than orthopaedic surgery.17,19,20,24-26,28,30,31 In general, randomized controlled trials provide some evidence that decay is a concern when trainees do not continue using the skills learned during simulation-based training.
Potential Solutions for Skill Decay
To ensure the durable proficiency of surgical skills acquired in orthopaedic simulation laboratories, retesting of trainees’ skill retention is essential. Although several studies have raised concerns about long-term retention,17,19,20,25,30,31 the issue of skill decay has not been thoroughly assessed or even sufficiently addressed by the rapidly evolving field of simulation-based surgical education. Despite limited evidence, potential solutions for decay include repracticing of the skills learned in simulators at appropriate times, and determination and reinforcement of areas that are sensitive to decay, such as arthroscopic knot-tying.
Repeated Practice of Skills Learned on Simulators
Repetitio est mater studiorum (ie, repetition is the mother of studying/learning)32 is a notable Latin phrase that indicates the importance of repeated practice in learning. As in any basic learning process, repeated practice of surgical skills on simulators gradually and proportionally improves performance until a plateau level of proficiency is achieved.26,33 However, retention is most likely achieved with “spaced rehearsal” of learned skills, a technique in which increasing amounts of time are incorporated between subsequent practice sessions to exploit the psychological spacing effect. Trainees may learn skills more effectively and more easily by practicing them repeatedly over a longer period using a spacing effect, rather than trying to achieve proficiency in a short time.34
Hein et al30 demonstrated that skills learned in a clinical simulation unit were retained at a significantly higher rate when they were rehearsed before a performance assessment. In this study, subjects received simulation-based training for insertion of a laryngeal airway mask. After a lapse of 6 months, the intervention group watched the same instructional video shown in the initial training program and practiced with the simulator for 10 minutes before their performance was assessed; the control group did not watch the video or practice before the assessment. The intervention group had a significantly shorter median time to first successful insertion (P = 0.029) and required fewer attempts to achieve success (P = 0.033) than did the control group. These results suggest that repracticing of skills learned in a simulator, even after a period of lapse, may substantially mitigate skill decay. Published evidence supports the finding that short-term pretrial practice of surgical procedures (ie, 15- to 20-minute preoperative warmup with simple surgical exercises) substantially increases proficiency in surgical skills during follow-up tasks.35
Randomized controlled studies in which performance was assessed after a period of lapse have also demonstrated decay in skills learned on simulators.15,17,20,30 In contrast, most studies that did not include a lapse in the practice of simulation-based skills showed no significant decay.14,24,26,28 This pattern also supports the suggestion that repetition of skills learned in simulators enhances retention.
The effect of a lapse or period of decreased use of skills on skill decay is also important. The longer the lapse time or period of minimal skill use, the more extensive the decay will be. Maagaard et al19 showed that novices retained laparoscopic skills acquired on a simulator after a 6-month break from practicing these skills; however, their laparoscopic skills returned to the pretraining level after 18 months without practice.
Determining and Reinforcing Decay-sensitive Components of Procedural Skills
As with analytical approaches to problem solving,36 breaking up certain simulation-based procedural skills into smaller components can help determine which skills are sensitive to decay and how to reinforce them to maximize long-term retention.
Surgical skills training can be abstractly divided into cognitive and manual phases. In the cognitive phase, trainees acquire basic knowledge related to the performance of procedural skills, such as recognition of specific facts, patterns, and concepts that help develop surgical proficiency.37 For example, in this phase, trainees may receive theoretical instructions, read about a procedure, and/or watch web-based or video demonstrations.25 In the manual phase, trainees practice sets of surgical skills using hands-on simulated models. In general, trainees should master the cognitive aspects of surgical skills before moving on to the manual phase using simulators. Although the review of theoretical materials through rehearsal in the cognitive phase may reinforce surgical skill retention, these skills will likely not be retained in the long term without repracticing of the manual components.
Preisner et al25 reported on a group of trainees who participated in a simulation-based course on knee and shoulder aspirations and injections and were retested 6 to 30 months later. Trainees who reviewed web-based instructional materials without repracticing manual skills before retesting outperformed the group that neither reviewed the web-based materials nor repracticed on a simulator. This included higher scores on obtaining informed consent for the shoulder procedure and on giving postprocedural instructions for both shoulder and knee aspirations and injections—areas that could be considered cognitive components of the procedures. No differences between the groups were found in terms of procedural proficiency, which was evaluated using a checklist of manual skills. Compared with their performance immediately after the simulation course, however, both groups demonstrated considerable deterioration in scores for all three procedural components at the time of follow-up.
During the manual phase of simulated surgical procedures, trainees translate theoretical knowledge into hands-on task performance by associating cognitive aspects of the procedural skills with musculoskeletal maneuvers. The manual phase of a simulated surgical procedure may include several components with varying skill requirements and task difficulties.38 For example, achieving proficiency in laparoscopic suturing and knot tying may require more practice on a simulator than relatively easier laparoscopic tasks, such as peg transfer and pattern-cutting skills.38 Peg transfer and pattern cutting are two exercises in the Fundamentals of Laparoscopic Surgery simulation module. In peg transfer, trainees first lift objects with a grasper in the nondominant hand and transfer the objects midair to the dominant hand. They then place each object on a peg on the right side of the board. Once all six pegs have been transferred, the process is reversed. Each peg is lifted from the right side of the pegboard using the dominant hand, then transferred midair to the left hand and placed on the pegs on the left side of the board. In pattern cutting, trainees use both hands in a complementary manner; for example, one hand uses a grasper to provide traction on a piece of gauze and to place the gauze at the best possible angle for the cutting hand.
Predictably, the tasks that require more training with simulators also tend to decay over time. Edelman et al31 found that performance of extracorporeal and intracorporeal knot tying markedly deteriorated 7 to 8 months after simulation-based Fundamentals of Laparoscopic Surgery training compared with performance immediately after training (P < 0.0001 and P = 0.029, respectively). However, trainee performance on peg transfer and pattern-cutting tasks showed no significant deterioration at the same follow-up time (P = 0.726 and P = 0.114, respectively).
Likewise, simulated arthroscopic surgery skills training involves several tasks of varying difficulty. Achieving proficiency in simulated surgical arthroscopic procedures, such as meniscus suturing, débridement, and loose body removal, may take longer and require more practice than reaching proficiency in simulated basic or diagnostic arthroscopy tasks. In the former, trainees must learn how to hold instruments and to bend the patient’s knee or shoulder while cutting, shaving, or grasping, whereas in the latter, trainees practice the concept of triangulation and handling instruments without causing damage inside a joint. To date, no study has compared simulated training of surgical arthroscopy skills with that of basic or diagnostic arthroscopy skills in terms of time to achieve task proficiency or deterioration of learned skills over time.
Professional Association Perspectives on Simulation-based Training
Postgraduate Year 1 Requirements and Issues With Skill Retention
In 2012, the American Board of Orthopaedic Surgery (ABOS) and the Residency Review Committee for Orthopaedic Surgery collaborated on new program requirements for postgraduate year 1 (PGY 1) training. At that time, it was clear that orthopaedics had fallen behind other surgical disciplines in providing focused skills training for residents. The new requirements, which went into effect in July 2013, increased the time on orthopaedic rotations to 6 months for PGY 1 and required laboratory-based surgical skills training. The training curriculum is designed to provide basic clinical skills for the care of injured patients and basic surgical skills to facilitate operating room experiences.
Program directors of orthopaedic surgery residency programs in the United States think that formal skills training is important at the junior level. In a 2013 survey, most program directors indicated that the primary learners for skills training were orthopaedic junior residents (95%) and orthopaedic senior residents (70%).39 Furthermore, the ABOS developed a curriculum of skill modules that is available on its website for free. Although the modules are not mandatory, they have been widely used, with most being accessed by many institutions.40 It seems evident that surgical training can be improved through dedicated practice of simple simulation-based skills.
Since 2013, residency programs have used either focused rotations for skills training, such as a one-time concentrated boot camp‒style introduction to laboratory-based skills education, or longitudinal experiences throughout PGY 1. Despite the success of these new training programs, there is little evidence that skills learned at the PGY 1 level are transferring to the operating room to facilitate quicker, safer, or better acquisition of actual surgical skills. It is not known if a one-time concentrated boot camp‒style introduction to laboratory-based skills training enhances skill retention more effectively than does longitudinal experiences provided throughout the year. Furthermore, given the early training targets, the degree to which skills learned at the junior level are retained and improve performance later in training are important unaddressed issues. If the skills obtained during PGY 1 are not retained, these time-consuming and resource-intensive programs are of limited value.
Skill retention issues can be addressed in several ways. Assessments of skill acquisition should be repeated at points remote from the actual training. A revision of the PGY 1 modules that is under way will focus partially on assessment and skill retention. Multi-institutional educational research that involves a sufficient number of subjects to demonstrate that simulation-based training is effective in improving surgical skills and that those skills obtained are retained later in training also is needed. Skill development programs should be instituted in the later years of training as well, rather than in PGY 1 only. Currently, the PGY 1 modules focus on basic surgical skills (ie, breaking up more complex skills into basic elements). Ideally, basic skills learned in the simulation laboratory are being reinforced, enhanced, and translated with greater ease in acquiring higher-level skills.
Assessment of Orthopaedic Skill Competency
The word competency, which indicates adequate qualifications to perform a specific skill set successfully,41 is at the core of medical licensure and board certification. The certification process for orthopaedic surgeons includes passing a computer examination rich in questions that reflect orthopaedic knowledge and patient care. Candidates for board certification must then practice orthopaedic surgery for 2 years to be peer reviewed by professionals in their local community and must pass an oral examination based on actual surgical practice.
Independent external evaluation of surgical skills is not included in orthopaedic board certification; however, the ABOS is assessing mechanisms to develop and validate surgical skills tests as part of this process. Testing of surgical skills would most likely take the form of surgical simulation in a laboratory setting, which would enable the observation and evaluation of common skills, such as simulated arthroscopy and simulated fluoroscopic wire navigation. Simulation-based surgical skills training is now required in the PGY 1 orthopaedic curriculum. The next step might be to require simulated skills assessment prior to being eligible for the initial orthopaedic board certification examination. Eventually, skills practice and assessment of skills in a simulated environment may become part of lifelong learning. Sequential skill assessment over time would detect progression or decay of skills that were at the proficiency level during supervised orthopaedic surgery training.
Of note, assessment of surgical competency for board certification will require standard, objective, and validated measurements. In orthopaedics, published evidence is limited regarding the most feasible, reliable, and unbiased techniques for assessing simulated surgical competency, especially at the level of high-stakes examinations, and this is an obstacle to moving forward. In general surgery, simulation was formally acknowledged as being based on available high-quality evidence after 2002, which led to full endorsement of simulated surgical skills training by the American College of Surgeons.22 Surgical simulation was subsequently integrated into the surgical residency curriculum, and the American Board of Surgery mandated the use of the Fundamentals of Laparoscopic Surgery education module for board certification in 2009.42
The transition from an apprenticeship model to simulation-based surgical education is under way. Retention of skills and their transferability to clinical situations are essential in simulation-based surgical education. However, research into long-term skill retention after simulation-based training in orthopaedic surgery is needed to support the efficiency and success of this paradigm shift. Furthermore, assessment of long-term retention of skills may help reveal latent deficiencies and areas that are sensitive to skill decay in a way that improves simulation-based training for education of more competent surgeons in the near future.
Evidence-based Medicine: Levels of evidence are described in the table of contents. In this article, references 6, 8, 9, 20, and 22 are level I studies. References 5, 13-19, 23-26, 28-30, 33, 35, and 38 are level II studies. References 4, 7, 10, 21, and 41 are level III studies. References 27, 31, 39, and 40 are level IV studies. References 1-3, 11, 12, 34, 36, and 37 are level V expert opinion.
References printed in bold type are those published within the past 5 years.
1. Pedowitz RA, Marsh JL: Motor skills training in orthopaedic surgery: A paradigm shift toward a simulation-based educational curriculum. J Am Acad Orthop Surg 2012;20(7):407-409.22751159
2. Thomas GW, Johns BD, Marsh JL, Anderson DD: A review of the role of simulation in developing and assessing orthopaedic surgical skills. Iowa Orthop J 2014;34:181-189.25328480
3. Atesok K, Mabrey JD, Jazrawi LM, Egol KA: Surgical simulation
in orthopaedic skills training
. J Am Acad Orthop Surg 2012;20(7):410-422.22751160
4. Dawe SR, Pena GN, Windsor JA, et al. Systematic review of skills transfer after surgical simulation
-based training. Br J Surg 2014;101(9):1063-1076.24827930
5. Buckley CE, Kavanagh DO, Traynor O, Neary PC: Is the skillset obtained in surgical simulation
transferable to the operating theatre? Am J Surg 2014;207(1):146-157.24238602
6. Howells NR, Gill HS, Carr AJ, Price AJ, Rees JL: Transferring simulated arthroscopic skills to the operating theatre: A randomised blinded study. J Bone Joint Surg Br 2008;90(4):494-499.18378926
7. Dawe SR, Windsor JA, Broeders JA, Cregan PC, Hewett PJ, Maddern GJ: A systematic review of surgical skills transfer after simulation-based training: Laparoscopic cholecystectomy and endoscopy. Ann Surg 2014;259(2):236-248.24100339
8. Larsen CR, Soerensen JL, Grantcharov TP, et al. Effect of virtual reality training on laparoscopic surgery: Randomised controlled trial. BMJ 2009;338:b1802.19443914
9. Deutschmann MW, Yunker WK, Cho JJ, Andreassen M, Beveridge S, Bosch JD: Use of a low-fidelity simulator to improve trans-nasal fibre-optic flexible laryngoscopy in the clinical setting: A randomized, single-blinded, prospective study. J Otolaryngol Head Neck Surg 2013;42(1):35.23688331
10. Pokroy R, Du E, Alzaga A, et al. Impact of simulator training on resident cataract surgery. Graefes Arch Clin Exp Ophthalmol 2013;251(3):777-781.23007233
11. Schmidt RA, Bjork RA: New conceptualizations of practice: Common principles in three paradigms suggest new concepts for training. Psychol Sci 1992;3(4):207-217.
12. Dale E: The cone of experience, in Audio-visual Methods in Teaching. New York, NY, Dryden Press, 1946, pp 37-51.
13. Sonnadara RR, Van Vliet A, Safir O, et al. Orthopedic boot camp: Examining the effectiveness of an intensive surgical skills course. Surgery 2011;149(6):745-749.21236456
14. Sonnadara RR, Garbedian S, Safir O, et al. Orthopaedic boot camp II: Examining the retention rates of an intensive surgical skills course. Surgery 2012;151(6):803-807.22652121
15. Howells NR, Auplish S, Hand GC, Gill HS, Carr AJ, Rees JL: Retention of arthroscopic shoulder skills learned with use of a simulator: Demonstration of a learning curve and loss of performance level after a time delay. J Bone Joint Surg Am 2009;91(5):1207-1213.19411470
16. Jackson WF, Khan T, Alvand A, et al. Learning and retaining simulated arthroscopic meniscal repair skills. J Bone Joint Surg Am 2012;94(17):e132.22992861
17. Stefanidis D, Acker C, Heniford BT: Proficiency-based laparoscopic simulator training leads to improved operating room skill that is resistant to decay. Surg Innov 2008;15(1):69-73.18387999
18. Spruit EN, Band GP, Hamming JF: Increasing efficiency of surgical training: Effects of spacing practice on skill acquisition
and retention in laparoscopy training. Surg Endosc 2015;29(8):2235-2243.25318372
19. Maagaard M, Sorensen JL, Oestergaard J, et al. Retention of laparoscopic procedural skills acquired on a virtual-reality surgical trainer. Surg Endosc 2011;25(3):722-727.20927549
20. Varley M, Choi R, Kuan K, et al. Prospective randomized assessment of acquisition and retention of SILS skills after simulation training. Surg Endosc 2015;29(1):113-118.25005011
21. Zendejas B, Brydges R, Hamstra SJ, Cook DA: State of the evidence on simulation-based training for laparoscopic surgery: A systematic review. Ann Surg 2013;257(4):586-593.23407298
22. Seymour NE, Gallagher AG, Roman SA, et al. Virtual reality training improves operating room performance: Results of a randomized, double-blinded study. Ann Surg 2002;236(4):458-464.12368674
23. Barsuk JH, Cohen ER, McGaghie WC, Wayne DB: Long-term retention of central venous catheter insertion skills after simulation-based mastery learning. Acad Med 2010;85(10 suppl):S9-S12.20881713
24. Moazed F, Cohen ER, Furiasse N, et al. Retention of critical care skills after simulation-based mastery learning. J Grad Med Educ 2013;5(3):458-463.24404310
25. Preisner R, Jasti H, Elnicki M, Jeong K: Impact of web-based review on long-term retention of simulation-acquired knee and shoulder aspiration and injection skills. J Grad Med Educ 2012;4(4):460-466.24294422
26. Jiang G, Chen H, Wang S, et al. Learning curves and long-term outcome of simulation-based thoracentesis training for medical students. BMC Med Educ 2011;11:39.21696584
27. Kuduvalli PM, Parker CJ, Leuwer M, Guha A: Retention and transferability of team resource management skills in anaesthetic emergencies: The long-term impact of a high-fidelity simulation-based course. Eur J Anaesthesiol 2009;26(1):17-22.19122546
28. Ortner CM, Richebé P, Bollag LA, Ross BK, Landau R: Repeated simulation-based training for performing general anesthesia for emergency cesarean delivery: Long-term retention and recurring mistakes. Int J Obstet Anesth 2014;23(4):341-347.25066817
29. Vadnais MA, Dodge LE, Awtrey CS, Ricciotti HA, Golen TH, Hacker MR: Assessment of long-term knowledge retention following single-day simulation training for uncommon but critical obstetrical events. J Matern Fetal Neonatal Med 2012;25(9):1640-1645.22191668
30. Hein C, Owen H, Plummer J: A training program for novice paramedics provides initial laryngeal mask airway insertion skill and improves skill retention
at 6 months. Simul Healthc 2010;5(1):33-39.20383089
31. Edelman DA, Mattos MA, Bouwman DL: FLS skill retention
(learning) in first year surgery residents. J Surg Res 2010;163(1):24-28.20605587
32. List of Latin phrases (R). https://en.wikipedia.org/wiki/List_of_Latin_phrases_(R
). Accessed May 2, 2016.
33. MacDonald J, Williams RG, Rogers DA: Self-assessment in simulation-based surgical skills training. Am J Surg 2003;185(4):319-322.12657382
34. Yeh DD, Park YS: Improving learning efficiency of factual knowledge in medical education. J Surg Educ 2015;72(5):882-889.25920573
35. Kahol K, Satava RM, Ferrara J, Smith ML: Effect of short-term pretrial practice on surgical proficiency in simulated environments: A randomized trial of the “preoperative warm-up” effect. J Am Coll Surg 2009;208(2):255-268.19228538
36. Jones MD: The Thinker's Toolkit: 14 Powerful Techniques for Problem Solving. New York, NY, Three Rivers Press, 1998.
37. Bloom BS (ed), Engelhart MD, Furst EJ, Hill WH, Krathwohl DR: Taxonomy of Educational Objectives, Handbook I: The Cognitive Domain. New York, NY, David McKay, 1956.
38. Stefanidis D, Gardner C, Paige JT, Korndorffer JR, Nepomnayshy D, Chapman D: Multicenter longitudinal assessment of resident technical skills. Am J Surg 2015;209(1):120-125.25466766
39. Karam MD, Pedowitz RA, Natividad H, Murray J, Marsh JL: Current and future use of surgical skills training laboratories in orthopaedic resident education: A national survey. J Bone Joint Surg Am 2013;95(1):e4.23283381
40. Westerlind BO, Marsh JL, ABOS Surgical Skills Task Force: Program directors’ survey of ABOS surgical skills curriculum for PGY-1. Poster #1743 presented at the 2015 Annual Meeting of the American Orthopaedic Association/Council of Residency Directors, Providence, Rhode Island, June 24-27, 2015.
41. Ericsson KA, Krampe RT, Tesch-Rhomer C: The role of deliberate practice in the acquisition of expert performance. Psychol Rev 1993;100(3):363-406.
42. The American Board of Surgery: ABS to require ACLS, ATLS and FLS for general surgery certification. https://www.absurgery.org/default.jsp?news_newreqs
. Accessed April 7, 2016.