Developing expertise is an important goal in any line of work. The desire to be good at one’s work grows out of the aspiration to lead a virtuous and productive life. For a dedicated anesthesiologist, performing at a high level means delivering good care to patients and providing excellent service to surgeons, anesthesia colleagues, other providers, and the hospital. Expertise may help the anesthesiologist reduce the chance of complications1 and assist in recovering from difficult situations.2–4 It is generally characterized by an organized approach to performing anesthesiology.5 Being an expert in one’s profession leads to higher self-esteem and long-term career satisfaction.6–8 It could also increase job security.
Reliable, continued improvement in skills and performance results from deliberate practice, a training method in which the learner is (1) given a task exceeding his or her current skill level, (2) motivated to practice extensively and improve, (3) provided with comprehensive and effective feedback, and (4) prompted to reflect on the learning experience.9 Once a goal has been met, the trainee advances to a more difficult task. Deliberate practice was first identified as the critical factor in achieving expertise in motor skill domains,10 and the steps may be familiar to people who have been involved with organized sports, learning music, or other performance arts. The method is also effective in other fields, including medicine.11
The goals of this article are to review the attributes that characterize expert performance and to discuss how deliberate practice cultivates and maintains individual expertise. It will review the general topic of expertise from the perspectives of education and psychology and then describe specific applications in anesthesiology. We will also discuss how training programs, continuing education modalities, and the individual practitioner can use deliberate practice to promote that goal.
CHARACTERIZING AND DEVELOPING EXPERT PERFORMANCE
Distinguishing Features of Experts
Expertise may be manifest qualitatively by superior perceptual, motor, and/or cognitive skills.12 For instance, experts in typing, tennis, and other pursuits are better than nonexperts in anticipating future movements controlling actions. Chess experts have a better memory of mid-game chess piece positions that they use to identify the best moves. According to K. Anders Ericsson, an authority on expert performance, consistently superior performance is the common characteristic of expertise in any field. Ericsson et al.12 suggest that expertise level can be quantified with standardized tests, such as high-fidelity simulations; examples are presented later in this article.
DEVELOPING EXPERT PERFORMANCE
Can High Level Expertise Be Nurtured?
Since the time of the ancient Greeks, numerous philosophers, social scientists, and psychologists have debated whether the capacity for elite performance is inborn or predominantly learned.13 If innate ability were a required prerequisite for high-level achievement, it would be futile to seek educational methods that would foster expert performance in every pupil. The psychologist Benjamin Bloom conducted extensive field research on the foundations of expertise. After reviewing the early lives of >120 star performers in the arts, sports, and scientific fields, he concluded that achieving excellence depended on hard work and training rather than exceptional native ability alone.14 Bloom’s observational work is supported by controlled experiments showing that proper training can lead unselected people with normal baseline abilities to amazing accomplishments,12 such as memorizing random strings of 80 or more digits15 or performing thousands of pushups in 1 sitting.16 Thus, it would be reasonable to expect hard work and sophisticated training methods to boost anesthesia performance to high levels.
The section below discusses the training requirements for acquiring expertise and contrasts the results that may be expected from routine practice versus deliberate practice. The sections that follow review the evidence that deliberate practice can improve learning in anesthesiology.
Attaining Expert Performance Through Deliberate Practice
Humans develop the ability to perform commonplace activities, such as tying shoes or riding a bicycle after a relatively short period of simple practice.12 Performance plateaus at an acceptable level (lowest curve in Fig. 1 adapted from Ericsson’s work10) because further improvement carries little benefit. Professional and competitive domains require higher level performance, and skill acquisition must continue for a longer period of time, as shown by the middle curve in the same figure. However, acceptability can still blunt the drive to improve, so routine repetition of complex tasks does not inevitably lead to outstanding performance.17–20 Thus, recreational athletes peak at an acceptable, modest playing ability, and choir members become proficient in singing to local audiences, but nothing more. On the other hand, learners who use deliberate practice in their training can experience prolonged improvement throughout most of a career (uppermost curve in Fig. 110).
;)
Figure 1: The performance trajectory for people learning new skills can follow various courses. Individuals generally develop adequate expertise in an everyday skill after a short training period, 50 hours or less. Actions become automatic at that point and performance plateaus. In contrast, people who pursue a competitive or highly technical professional activity continue to improve for a longer period, but performance will eventually arrest if training occurs in a rote manner. In contrast, individuals who follow an intensive training method called deliberate practice will improve for a longer, possibly indefinite period and reach an expert level of performance after 10 to 15 years of training. The deliberate practice strategy is unique in actively seeking improvement (see text). If the professional ceases deliberate practice, their improvement will cease and their performance will arrest at a lower level.
10 Used with permission from Taylor & Francis Ltd,
www.tandfonline.com.
To understand why routine practice does not allow improvement to continue indefinitely, consider the benefit a basketball player would receive by only practicing shooting baskets during games, an example of routine performance. He or she would have 1 chance for each shot, no chance to repeat the shot until it worked, no immediate feedback from the coach, and no opportunity to reflect on performance. Outside of competition, the player could engage in deliberate practice by working repeatedly on mastering specific shots that may only arise a few times per game, varying distance, angle, or defensive pressure and receiving feedback from a coach.21 The game environment limits opportunity for improvement, while deliberate practice facilitates it.
How much deliberate practice is needed to reach expert performance levels? In a classic study, Simon and Chase22 observed that at least 10,000 hours of self-reported practice over 10 years or more were necessary for chess players to achieve elite performance in competitive play against grandmasters. Accumulating 10,000 hours of deliberate practice over 10 to 15 years14 appears to be linked consistently to development of exceptional performance across multiple disparate skill domains, including music, sports, and science.11,23,24
EXPERTISE IN ANESTHESIOLOGY
Does Anesthesiology Residency Produce Expertise?
We would have reasonable confidence that anesthesiology residents would achieve consistently superior performance if residency included 10,000 hours of deliberate practice,5 the number necessary for expertise in other disciplines.11,22–24 Residents in the authors’ anesthesiology program work 270 days per year. Most humans are limited to a maximum of about 4 hours of deliberate practice per day because of the intense effort required to combine practice and close attention to feedback.9 Thus, about 3240 hours of deliberate practice could occur over the course of a 3-year residency if all opportunities were used. This is slightly less than one-third of the 10,000-hour milestone for expertise. By these figures, anesthesiology trainees would be far from achieving high-level performance at the conclusion of residency, even under the best circumstances. Several additional years of hard work would be required to reach the 10,000-hour mark and the corresponding high-level performance,9,14,22,23 amounting to 7 to 12 years and an age in the mid-30s to early 40s.
Do Anesthesiologists Continue Improving After Residency?
A relevant question is whether anesthesiologists routinely proceed to high-level performance after residency. Simulations involving anesthetic crises are appropriate models for testing high-level anesthesiology performance because managing critical incidents requires skills that numerous authorities associate with anesthesiology expertise.25–31
Henrichs et al.32 used quantitative assessment tools to evaluate how 35 practicing anesthesiologists (2–26 years in practice) performed with 8 scripted critical intraoperative events. In advance, the authors had suggested that accomplished anesthesiologists should perform at least 75% of the steps considered key for appropriate management. In fact, subjects accomplished 67% ± 12% of the key steps overall. Performance was approximately 60% for myocardial ischemia, blocked endotracheal tube, and total spinal scenarios and <25% for malignant hyperthermia and hyperkalemia. While some anesthesiologists performed uniformly well across the different scenarios, the number of individuals who had mediocre scores and consistently ineffective responses surprised the investigators.
We examined additional studies on management of critical events by experienced anesthesiologists to investigate whether sub-par performance32 represents a general state of affairs. PubMed and Google Scholar searches were conducted using the terms “anesthesiologist” AND “crisis management” for articles through mid-March 2014 with no beginning date. We screened articles by title, journal, and abstract for studies that referenced critical situations relevant to anesthesiology and explicitly involved anesthesiologists. We scrutinized the text of articles passing the initial screen to identify those that measured performance quantitatively and recruited anesthesiologist subjects who had finished training. To increase the yield, we also performed ancestor and descendant searches on the articles that satisfied selection criteria. The ancestor search examined the reference list from the selected articles, while descendants were identified by searching on PubMed for articles related to each study making the cut.
Our search identified 7 research articles that evaluated the performance of postgraduate anesthesiologists in crisis situations from among the 1255 articles that referenced critical anesthesiology situations and involved anesthesiologists.31–37 Although anesthesiologists managed some of the crises well, each of the studies included at least 1 scenario where performance decreased below the acceptable 75% mark, among them hyperkalemia, malignant hyperthermia, ventricular fibrillation, anuria, myocardial ischemia, and anaphylaxis (Table 1). Performance below expectations could occur in part because the observations were made in the laboratory. However, it appears that a subset of anesthesiologists have not achieved skills in handling unanticipated or unusual situations, one of the hallmarks of expertise, in spite of finishing residency.
Table 1: Examples of Poor Performance of Experienced Anesthesiologists on Simulated Critical Events
For some of the anesthesiologists, skill may have deteriorated after training, while in others the anesthesiologists may have never mastered the requisite skills. The Murray et al.31 study provides insight in this regard because it also evaluated anesthesia residents (not included in Table 1). For example, residents performed better than experienced anesthesiologists in managing hyperkalemia, suggesting that physicians had forgotten management details after training. On the other hand, Murray et al.’s study found progressive improvement in management of myocardial ischemia and total spinal scenarios in the experienced practitioners, but average scores did not meet the satisfactory mark. Whatever the cause for low performance, the practicing anesthesiologists may have benefited in our opinion if residency had prepared them better for managing crises and if their training had included instruction in lifelong learning techniques to maintain and improve expertise.38–40
DELIBERATE PRACTICE AND ANESTHESIOLOGY TRAINING
Is Deliberate Practice Effective in Anesthesiology?
In the cognitive science field, it is commonly accepted that deliberate practice is necessary, though perhaps not sufficient, to become an expert in domains involving both motor and cognitive skills.41,42 We would expect deliberate practice to be effective for fostering expertise in our specialty, just as in other disciplines.11,43 Inductive reasoning would allow us to accept such a conclusion if we found instances where deliberate practice contributed to acquisition of anesthesia skill. Thus, we conducted literature searches for studies that evaluated the impact of deliberate practice on anesthesiologists’ skills. The training had to involve anesthesia-related procedures or cognitive skills, and the study had to assess the educational benefit of deliberate practice in a quantitative manner. Furthermore, the deliberate practice exercise could not be concurrent with the testing. A study was deemed to use deliberate practice if trainees practiced a skill more than once and received feedback from a qualified instructor. Acceptable study designs were a repeated measures design that compared test results before and after deliberate practice or a design with at least 1 control group and 1 group with a deliberate practice intervention.
PubMed and Google Scholar searches were conducted on January 30, 2014, using the terms “deliberate practice” AND “anesth*,” plus “deliberate practice” and “medical procedures.” The search included articles through January, 2014, with no beginning date. We also identified possible studies from the reference list of a review article on deliberate practice for medical procedures by McGaghie et al.44 We screened title, journal, and abstract for studies that referenced educational methods and explicitly involved anesthesiologists practicing procedures/cognitive skills relevant to anesthesia. We examined the text of articles that passed the first screen to select articles that used a deliberate practice method, reported quantitative results, and used an acceptable study design. To increase the yield, we performed ancestor and descendant searches on articles that met the acceptance criteria, as described for the anesthesia crisis search.
Our search identified 11 studies that met the inclusion criteria out of 1958 articles that passed the first screen, namely, referencing educational methods and involving anesthesiologists practicing anesthesia skills. The studies covered such topics as airway management, nontechnical skills, crisis management, weaning from cardiopulmonary bypass, regional anesthesia, and electroencephalogram interpretation (Tables 2 and 3).45–55 Deliberate practice resulted in a significant improvement compared to controls in 10 of the 11 studies. In 5 studies, deliberate practice had a greater effect than a control training method. The other 5 positive results were in comparison to pretest skills or skills in an untrained control group. Thus, deliberate practice is an effective training method for the types of anesthesia knowledge and procedures represented in Tables 2 and 3.
Table 2: Effect of Deliberate Practice on Selected Anesthesia Skills: Quantitative Educational Outcome
Table 3: Effect of Deliberate Practice on Selected Anesthesia Skills: Description of Studies
One would expect deliberate practice training to be better than no training. More interesting questions are whether deliberate practice has benefits compared to other educational endeavors and whether deliberate practice compares favorably with interventions that hold students’ attention in an equally engaging fashion. Two of the studies in Table 2 deserve attention on these issues. The Bruppacher et al.53 investigation compared deliberate practice training incorporated into simulation versus an interactive seminar, both 2 hours, for teaching residents how to wean patients from cardiopulmonary bypass. The seminar attempted to engage students by discussing real cases, using multimedia aids and conducting a back-and-forth discussion between the residents and faculty. The deliberate practice training proved superior to the seminar, and the difference persisted for at least 5 weeks, indicating a period of skill retention. Since the outcome was measured in weaning an actual patient from bypass, deliberate practice had an apparent impact on real-life practice, an outcome that matters.56
The other example, Johnson et al.,50 compared 2 methods of training residents how to manage adverse airway and respiratory events during anesthesia. Both groups received 40 didactic presentations on anesthetic management and simulation sessions covering 5 different scenarios. In the control group, the didactic sessions covered the knowledge and abilities a clinician would need, and the simulation sessions focused on steps that would be taken to manage airway and respiratory events. In contrast, the 15 of the 40 didactic sessions presented to the experimental group were organized around part task training, a method to break medical tasks into smaller sequential components following a checklist. The participants mentally rehearsed the sequences until they were ingrained, that is, successful deliberate practice. The simulation training for the experimental group also differed in requiring deliberate practice on crisis management. The outcome was that the experimental group demonstrated greater skill in handling unfamiliar adverse anesthetic events during a simulation test. The noteworthy point about this study was that it controlled to some extent for level of engagement by giving experimental and control groups the same simulation scenarios. Thus, the difference in outcome could be attributed in part to deliberate practice.
Deliberate Practice Is Feasible During Anesthesiology Training
The elements of deliberate practice can be incorporated into clinical teaching encounters in various forums.57 The focus can be on any anesthetic skill where opportunities to work on a problem occur repeatedly. For example, every case presents opportunities to communicate with the patient and the operative team and to assess the patient’s preoperative condition. Common anesthetic issues, such as treating hypotension, assessing anesthetic depth, evaluating volume responsiveness, or managing treatment, occur so frequently in anesthetized patients that residents may face a particular problem and practice it repeatedly during a single case. Other management issues, such as controlling the airway or planning emergence, can be practiced many times over the course of a few days. Deliberate practice for special issues, such as management of aortic stenosis or how to perform an ultrasound-guided regional anesthetic, is facilitated by subspecialty rotations, which focus residents on specific operations, procedures, and management issues within a brief time period.
Discussion formats allow teaching in a setting that does not compete with clinical care. They are appropriate for deliberate practice with nontechnical skills and cognitive skills. For example, a resident can practice communication skills, planning, and decision making by engaging in preoperative discussions with the faculty supervisor before each case. Mock oral examinations give residents practice in exercising knowledge and a range of cognitive skills, as well as preparation for the real examinations. Since adult learners tend to respond well in active learning situations,58,59 discussions should elicit audience participation, make students exercise decision-making skills, and push them to extend their expertise.
Some events, such as some anesthetic crises and rare operations, are so uncommon that residents may require training outside of the clinical arena to gain experience.60 Simulation provides opportunities to schedule training, control the environment, provide nearly unlimited chances to practice, and avoid risk to patients.61–64 Motor skills, such as airway management, spinal and epidural placement, nerve blocks, line insertion, and ultrasound techniques, can be practiced on partial task trainers,65–67 simple simulators that focus on teaching specific skills or concepts rather than management of the whole patient.
OPTIMIZING DELIBERATE PRACTICE
In recent years, cognitive psychology research has addressed the learning forms that increase the effectiveness of deliberate practice and identified several effective learning strategies. An article by Dunlosky68 provides a good nontechnical review.
One strategy is to consider testing not just for assessment, but also as a potent learning event stimulating retrieval from long-term memory.69 In one illustrative experiment,70 subjects underwent an acute test of recall after studying a list of words. They displayed greater accuracy on a delayed final test than did subjects who performed an equivalent amount of additional study in lieu of the acute testing. Testing contributes to learning across numerous domains and is effective in both laboratory and natural settings. The benefit is most profound when the learner receives feedback about the correct test answers,71 consistent with the importance of feedback in deliberate practice in general.
Testing-based learning can be incorporated into deliberate practice by alternating study and testing. For example, instead of reading a section of text a second time, learners could review the topic heading, test themselves by recalling as much of the section content as possible, and then read the section content to obtain feedback about their recall. In transesophageal echocardiography, an instructor could guide a resident in obtaining the standard views, test performance by asking the resident to demonstrate the complete examination, and then comment on the view quality.
A second robust learning strategy is distributed practice. For a fixed amount of learning time, splitting the time into multiple sessions over hours or days (distributing the practice) yields markedly better long-term retention of learning than does performing all the practice in a single long sitting. A body of evidence supports the effectiveness of distributed practice for a variety of tasks in both laboratory and authentic educational contexts.72,73 Distributed practice can arise naturally in anesthesiology education, since training events, such as management of hypotension, frequently recur from one patient to the next.
A third promising approach to optimize deliberate practice is task interleaving. For example, if 4 distinct types of problems (A, B, C, and D) require practice, long-term learning is enhanced by interleaving (mixing or shuffling) those problem types during practice (i.e., A, B, C, D, B, C, D, A, etc.) as opposed to blocking them (e.g., A, A, A, A, B, B, B, B, etc.). In a study involving 4 different computational tasks, interleaving practice yielded a 300% increase in accuracy on a delayed test compared to a blocking strategy.74 In the previous echocardiography example, the interleaving could involve practice on obtaining specific views interspersed with practice on assessing contractility and on valve function. The benefit of task interleaving might be augmented by distributing the echocardiography practice over several sessions as well.
Another potentially beneficial method is guided discovery learning, in which a learner is provided with limited explicit instruction and then must discover the solution to a problem alone. This method from constructivist theory is based on the premise that people learn effectively by building their own knowledge.75,76 Assisted, or enhanced, discovery learning typically yields better outcomes than does direct instruction when it includes a support framework (e.g., an outline of how to solve the problem), worked examples, timely feedback to direct the learner away from incorrect solution paths, and the ability for the learner to solicit explanations.77,78 These principles are fully consistent with the tenets of deliberate practice. An example with regard to echocardiography would be for the instructor to allow the resident to discover how to optimize the various views independently. The instructor would provide a rough algorithm about probe manipulations and might help the resident work through one view before letting the trainee work independently on the other views. Providing guidance is extremely important. In pure discovery learning, unguided students may not encounter the necessary circumstances or have the wherewithal to learn new principles on their own.78 This is compatible with Ericsson’s observation that routine on-the-job practice leads to stagnated abilities rather than continued improvement.10
Finally, deliberate practice can be enhanced at a more general level by cultivating metacognitive (i.e., thinking about thinking) skills. Metacognition can follow an iterative cycle: forethought or planning, performance, and critical self-evaluation to improve future performance.77 The forethought phase encompasses goal setting, strategic planning, cultivation of self-motivating beliefs, and an orientation toward learning. The performance phase includes selecting effective learning strategies, time management, help seeking, self-monitoring, and keeping records of one’s performance. The reflection phase involves self-evaluation, tracing the cause of critical events, and identification of performance aspects in need of improvement. The features of metacognition overlap with deliberate practice and can complement the technique. Anesthesiology residents will engage in metacognitive activities when they help plan educational endeavors, review goals at the start of the day, plan the educational endeavors, and reflect on the training outcome.
BARRIERS TO DELIBERATE PRACTICE IN ANESTHESIOLOGY
Unfamiliarity with Deliberate Practice
Deliberate practice strategies are not well known in anesthesia education. A PubMed search on the terms “anesth* or anaesth*” combined with “deliberate practice” recovered only 6 citations. In contrast, a similar search substituting “simulation” for “deliberate practice” yielded >3000 hits.a Thus, an information campaign may be necessary for a training program to promote routine use of the technique.
An initial step would be to provide faculty and residents with lectures and enrichment activities about deliberate practice and how it fits with training and lifelong learning. Anesthesiology attendings need instruction because only 10% to 20% of the faculty in most programs will have had formal education in teaching.79–81 Their ability to guide deliberate practice would benefit from instruction in formative feedback,82,83 assessing progress, planning teaching episodes, and accommodating instructional activity within the clinical timeframe. Articles with useful hints on clinical teaching84–91 could be circulated to faculty or used as the basis for the clinical education lectures. Clinical teaching tools, videos, and other online resources are also available.b
Providing instructors with feedback from their pupils is an effective method to promote the quality of clinical teaching. The anesthesiology program at Massachusetts General Hospital found that instructional performance improved significantly when faculty received regular quantitative individual summaries and narrative comments about teaching.92 Specific comments about deficiencies appear critical for helping teachers improve. Evaluation systems that include narrative lead to improvement, but reporting only numerical ratings to faculty is ineffective.93,94 Narrative from residents constitutes feedback, enabling faculty to enhance their clinical teaching through deliberate practice.
Problems in Selecting Daily Topics for Deliberate Practice
To lead deliberate practice, the attending anesthesiologist must address a gap in the resident’s ability with a practice activity. Identifying gaps can be difficult if the attending does not have knowledge about the trainee’s current performance, what tasks have been practiced, what issues remain, and what the resident’s next goals should be. The problem could be ameliorated if instructors wrote field notes about a resident’s educational status that could help subsequent supervisors plan teaching activities.61 Residents could help guide their own program by keeping track of their practice goals and communicating with attendings about their educational needs, examples of metacognitive skills used in pursuit of their learning.
Planning educational activities can present a challenge because much of anesthesia practice is made up of routine cases with ASA 1 and 2 patients. For example, an inguinal herniorrhaphy might not present obvious deliberate practice opportunities for residents who have handled large numbers of the operation. Anesthesia nontechnical skills, which include communication task management, teamwork, situation awareness, and decision-making ability, could be appropriate teaching topics in these situations.95 Suggestions for guiding deliberate practice on anesthesia nontechnical skills are available in the literature.96–99 Another option when the educational direction is not obvious would be to work on general topics, perhaps drawn from a list that the training program or the instructor prepared in advance. The University of California San Francisco anesthesiology training programs, where 1 of the authors worked, has used this approach in the past.
Time Pressure
Clinical teaching must be balanced with clinical care, frequently in multiple operating rooms and at other sites in the hospital, administrative duties, and demands for scholarly activity. Competing demands can limit time for engaging residents in deliberate practice.83,92,100,101 However, strategies are available for efficient clinical teaching within the constraints of patient care responsibilities.76,90
The 5-step microskills model is also known as the “One-Minute Preceptor.”90 In steps 1 and 2, the preceptor asks the trainee to explain an observation and then to support the claim. The final steps are to deliver feedback, reinforce positive behaviors, and identify areas for improvement. In anesthesia, for example, the attending could ask the resident to propose the most likely mechanism for a modest decline in oxygen saturation. After an opinion is given, the resident would be asked to support the diagnosis with relevant information or by diagnostic tests. The attending could discuss the resident’s reasoning, point out diagnostic maneuvers that might have been missed, praise insightful statements, and talk about causes of hypoxemia during anesthesia. The microskills model is consistent with deliberate practice because it can address gaps in resident skills and it allows residents to manage a clinical episode with faculty supervision and feedback.91,102
The BID model (briefing, intraoperative teaching, and debriefing) works well for guiding deliberate practice on procedures as well as cognitive skills.76 The briefing consists of a discussion of areas where the resident needs practice and relevant learning objectives. The surrounding discussion sets the educational objectives, focuses the learner on goals, and guides the instructor’s teaching. At the beginning of a day of regional anesthesia, for example, the attending might learn that the resident has difficulty keeping the needle in view during ultrasound-guided infraclavicular blocks because of the steep angle required to reach the cords.
During the teaching phase, the attending could suggest steps that the resident should practice and ask the resident to describe the actions in his or her own words. The attending would provide ongoing feedback and encouragement as needed. Debriefing, the interactive feedback period, should include reflection on the resident’s performance, reiteration of the rules or principles that were discussed, reinforcement of the areas where improvement occurred, and summary of issues that need correction and require more practice. In the infraclavicular block example, the attending might ask the resident to summarize actions that helped in visualizing the needle. The attending could reinforce important points, give additional insight, and suggest the next goal to practice, perhaps the technique for visualizing the middle cord.
PURSUING IMPROVEMENT BY PRACTICE AFTER RESIDENCY
Although anesthesiologists need several years of experience after residency to secure a consistently superior level of performance, entry into the workforce separates physicians from academic resources and formal educational opportunities that were readily at hand during postgraduate training. Thus, continuing the journey toward expertise depends on the individual’s motivation to excel and to pursue lifelong learning.40,103,104 Opportunities for deliberate practice after residency could include work on current skill sets and new ones, workshops and simulation courses that emphasize active learning, and active methods of self-study, such as the iterative study-testing technique described earlier.
Obtaining appropriate feedback may be a challenge within the job environment. However, Ericsson10 believes that individuals approaching expert status acquire the ability to monitor, critique, and refine their own performance. For example, a budding piano virtuoso knows how a piece of music should sound, can compare their own sound to the standard, and can work to eliminate the differences. Presumably, anesthesiologists also develop self-assessment skills as they mature, satisfying part of their need for feedback. Self-assessment is straightforward when mastering a new procedure because the procedural outcome provides a modicum of feedback. Anesthesiologists could obtain additional performance measures by timing their own procedures or comparing success rates with published values. For procedures that can be recorded, such as endoscopy or ultrasound, the practitioner could review the video to identify problems in technique. Using video as part of debriefing is common in simulation courses and has been an effective source of feedback for teaching epidural anesthesia skills.105 Ericsson106 suggests that viewing video recordings of medical procedures performed by experts may also be useful as an instructional tool. Videos can capture a procedure from multiple perspectives simultaneously and may be valuable in learning uncommon procedures.106Benchmarks for excellent performance on nontechnical skills are subtler, and objective self-assessment may be more difficult. Studies suggest that physicians in the United States are inaccurate in evaluating themselves and that less skilled individuals are more likely to overestimate the quality of their performance.107,108 Thus, practicing anesthesiologists would do well to seek objective data on their work from outside sources. Anesthesiologists could ask patients after surgery for comments about the anesthesiologists’ performance. Debriefing from surgeons and nurses after an operation could be valuable. Along these lines, anesthesiologists in Denmark use surveys to elicit patient feedback about their communication skills.109
Direction by a knowledgeable mentor is important for growth. Professional athletes and entertainers receive the tutelage of coaches, even though they already perform at the highest level. Coaches provide objectivity, guide training, and recognize problems or deficiencies that a performer would not perceive alone. Thus, an anesthesiologist committed to continuous improvement might profit through feedback and guidance from a qualified adviser. Atul Gawande,110 the well-known surgeon and medical journalist, has acknowledged that he regularly asks a fellow surgeon to critique his own performance and extols the benefits of coaching.
Continuing medical education activities are avenues for improvement after residency, but the practitioner should choose wisely. Courses that present information through passive learning modes, such as lectures, do not necessarily change behavior or lead to demonstrable improvement.111 The anesthesiologist should look for events that provide active learning with opportunities for practice and feedback, such as simulation courses or workshops.63,112,113
An anesthesiologist who has been in practice for 10 to 15 years and achieved a consistent pattern of excellent performance might be inclined to adopt a relaxed training regimen. However, age-related decline in performance occurs in the later stages of a career (note the decrease in the expert performance curve after years of practice in Fig. 1). Ericsson’s research suggests that deliberate practice can forestall the loss of ability in senior professionals in piano performance114 and in elite runners.115 Thus, deliberate practice is important for improvement at the beginning of a career, for becoming an expert during the middle phase and for maintaining expertise in later years.
CONCLUSIONS
The primary message of this review article is that anesthesiology trainees, practicing anesthesiologists, and senior practitioners need continual deliberate practice as a means to augment and maintain their professional skill. Deliberate practice is effective because individuals are challenged in areas needing improvement, they receive feedback to correct mistakes, and they practice to the point of mastery. The training method was identified as essential in sports and musical fields, but it is also necessary for advancement in medical professions,63,116,117 including anesthesiology (Tables 2 and 3).
The 3-year U.S. anesthesiology residency does not allow a sufficient number of deliberate practice hours for residents to finish with a high level of expertise. Furthermore, anesthesiologists do not universally proceed to superior levels of performance in the years after residency (Table 1). Accordingly, we suggest that efforts are needed to augment the use of deliberate practice during residency and to inculcate anesthesiologists with the principles necessary to continue improving after residency. We suggest a number of steps, by no means an exhaustive list, that could be taken. They include training anesthesiology faculty and residents how to perform deliberate practice, methods to motivate faculty to provide the necessary supervision and feedback, and avenues for deliberate practice after residency.
These steps have a cost in time and/or money, and they require buy-in at multiple levels. The anesthesiology department must promote deliberate practice and be willing to allow the time for such efforts. Clinical instructors must commit to improving their teaching skills, to planning daily deliberate practice, and to spending sufficient time with residents for effective application of the technique. Residents must take an active role in deliberate practice work and should be diligent in using the technique for their entire careers. Finally, the practicing anesthesiologist must find ways to incorporate deliberate practice into their workplace, strive to evaluate their own performance, compare the worth of continuing medical education offerings in improving their expertise, and consider the benefit of recruiting a mentor.
In our view, the benefits of adopting deliberate practice as a training principle far outweigh the costs. Providing clinical training is one of the primary purposes of an academic anesthesiology department and should be one of the reasons an anesthesiologist chooses to join a department, rather than practicing elsewhere. Improving training quality has to be highly valued in that environment. Educational responsibility is not the sole motivation, however. Deliberate practice can yield the rewards of enhanced service and care, pride in work, and satisfaction in the practice of our demanding branch of medicine.
DISCLOSURES
Name: Randolph H. Hastings, MD, PhD.
Contribution: This author helped design the study, conduct the study, analyze the data, write the manuscript, and perform the literature review.
Attestation: Randolph H. Hastings approved the final manuscript.
Name: Timothy C. Rickard, PhD.
Contribution: This author helped write the manuscript and perform the literature review.
Attestation: Timothy C. Rickard approved the final manuscript.
This manuscript was handled by: Franklin Dexter, MD, PhD.
FOOTNOTES
a PubMed searches were conducted on January 11, 2014, using the terms anesth* or anaesth* and “deliberate practice”; anesth* or anaesth* and “simulation.”
Cited Here
b www.practicaldoc.ca.
Cited Here
REFERENCES
1. Smith AF, Arfanis K. “Sixth sense” for patient safety. Br J Anaesth. 2013;110:167–9
2. Gaba DM. Anaesthesiology as a model for patient safety in health care. BMJ. 2000;320:785–8
3. Heine MF, Lake CL. Nature and prevention of errors in anesthesiology. J Surg Oncol. 2004;88:143–52
4. Botney R. Improving patient safety in anesthesia: a success story? Int J Radiat Oncol Biol Phys. 2008;71:S182–6
5. Smith AF, Greaves JD. Beyond competence: defining and promoting excellence in anaesthesia. Anaesthesia. 2010;65:184–91
6. Lindfors PM, Meretoja OA, Töyry SM, Luukkonen RA, Elovainio MJ, Leino TJ. Job satisfaction, work ability and life satisfaction among Finnish anaesthesiologists. Acta Anaesthesiol Scand. 2007;51:815–22
7. Finset KB, Gude T, Hem E, Tyssen R, Ekeberg O, Vaglum P. Which young physicians are satisfied with their work? A prospective nationwide study in Norway. BMC Med Educ. 2005;5:19
8. Chang WY, Ma JC, Chiu HT, Lin KC, Lee PH. Job satisfaction and perceptions of quality of patient care, collaboration and teamwork in acute care hospitals. J Adv Nurs. 2009;65:1946–55
9. Ericsson KA, Krampe RT, Tesch-Romer C. The role of deliberate practice in the acquisition of expert performance. Psychol Rev. 1993;100:363–403
10. Ericsson KA. The scientific study of expert levels of performance: general implications for optimal learning and creativity. High Ability Studies. 1998;9:75–100
11. Ericsson KA. Deliberate practice and acquisition of expert performance: a general overview. Acad Emerg Med. 2008;15:988–94
12. Ericsson KA, Nandagopal K, Roring RW. Toward a science of exceptional achievement: attaining superior performance through deliberate practice. Ann N Y Acad Sci. 2009;1172:199–217
13. Markie P. Rationalism vs. empiricism. Stanford Encyclopedia of Philosophy. 2013
14. Bloom BS Developing Talent in Young People. 1985 New York Ballantine Books
15. Ericcson KA, Chase WG, Faloon S. Acquisition of a memory skill. Science. 1980;208:1181–2
16. Guiness World Records 2013. 2012 Stamford, CT Guiness World Records
17. Doane SM, Pellegrino JW, Klatzky RL. Expertise in a computer operating system: conceptualization and performance. Hum Comput Interact. 1990;5:267–304
18. Reif F, Allen S. Cognition for interpreting scientific concepts: a study of acceleration. Cogn Instruct. 1992;9:1–44
19. Beam CA, Conant EF, Sickles EA. Association of volume and volume-independent factors with accuracy in screening mammogram interpretation. J Natl Cancer Inst. 2003;95:282–90
20. Butterworth JS, Reppert EH. Auscultatory acumen in the general medical population. JAMA. 1960;174:32–4
21. Williams T My Turn at Bat: The Story of My Life. 1988 New York Simon & Schuster, Inc.
22. Simon HA, Chase WG. Skill in chess. Am Scientist. 1973;61:394–403
23. Helsen WF, Hodges NJ, Van Winckel J, Starkes JL. The roles of talent, physical precocity and practice in the development of soccer expertise. J Sports Sci. 2000;18:727–36
24. Raskin E. Comparison of scientific and literary ability: a biographical study of eminent scientists and letters of the nineteenth century. J Abnormal Soc Psych. 1936;31:20–35
25. Fletcher GC, McGeorge P, Flin RH, Glavin RJ, Maran NJ. The role of non-technical skills in anaesthesia: a review of current literature. Br J Anaesth. 2002;88:418–29
26. Klemola UM, Norros L. Analysis of the clinical behaviour of anaesthetists: recognition of uncertainty as a basis for practice. Med Educ. 1997;31:449–56
27. Norros L, Klemola UM. Methodological considerations in analysing anaesthetists’ habits of action in clinical situations. Ergonomics. 1999;42:1521–30
28. Gravenstein JS. Training devices and simulators. Anesthesiology. 1988;69:295–7
29. Greaves JD, Grant J. Watching anaesthetists work: using the professional judgement of consultants to assess the developing clinical competence of trainees. Br J Anaesth. 2000;84:525–33
30. Kearney RA. Defining professionalism in anaesthesiology. Med Educ. 2005;39:769–76
31. Murray DJ, Boulet JR, Avidan M, Kras JF, Henrichs B, Woodhouse J, Evers AS. Performance of residents and anesthesiologists in a simulation-based skill assessment. Anesthesiology. 2007;107:705–13
32. Henrichs BM, Avidan MS, Murray DJ, Boulet JR, Kras J, Krause B, Snider R, Evers AS. Performance of certified registered nurse anesthetists and anesthesiologists in a simulation-based skills assessment. Anesth Analg. 2009;108:255–62
33. DeAnda A, Gaba DM. Role of experience in the response to simulated critical incidents. Anesth Analg. 1991;72:308–15
34. Schwid HA, O’Donnell D. Anesthesiologists’ management of simulated critical incidents. Anesthesiology. 1992;76:495–501
35. Lindekaer AL, Jacobsen J, Andersen G, Laub M, Jensen PF. Treatment of ventricular fibrillation during anaesthesia in an anaesthesia simulator. Acta Anaesthesiol Scand. 1997;41:1280–4
36. Jacobsen J, Lindekaer AL, Ostergaard HT, Nielsen K, Ostergaard D, Laub M, Jensen PF. Management of anaphylactic shock evaluated using a full-scale anesthesia simulator. Acta Anaesthesiol Scan. 2001;45:315–9
37. Devitt JH, Kurrek MM, Cohen MM, Fish K, Fish P, Noel AG, Szalai JP. Testing internal consistency and construct validity during evaluation of performance in a patient simulator. Anesth Analg. 1998;86:1160–4
38. Cooper JB, Murray D. Simulation training and assessment: a more efficient method to develop expertise than apprenticeship. Anesthesiology. 2010;112:8–9
39. Schwartz AJ. Education: an essential leg for anesthesiology’s four-legged stool! Anesthesiology. 2010;112:3–5
40. Schartel SA, Metro DG. Evaluation: measuring performance, ensuring competence, achieving long-term excellence. Anesthesiology. 2010;112:519–20
41. Campitelli G, Gobet F. Deliberate practice: necessary but not sufficient. Curr Dir Psychol Sci. 2011;20:280–5
42. Macnamara BN, Hambrick DZ, Oswald FL. Deliberate practice and performance in music, games, sports, education, and professions: a meta-analysis. Psychol Sci. 2014;25:1608–18
43. Issenberg SB, McGaghie WC, Hart IR, Mayer JW, Felner JM, Petrusa ER, Waugh RA, Brown DD, Safford RR, Gessner IH, Gordon DL, Ewy GA. Simulation technology for health care professional skills training and assessment. JAMA. 1999;282:861–6
44. McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB. Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence. Acad Med. 2011;86:706–11
45. Abrahamson S, Denson JS, Wolf RM. Effectiveness of a simulator in training anesthesiology residents. J Med Educ. 1969;44:515–9
46. Ovassapian A, Yelich SJ, Dykes MH, Golman ME. Learning fibreoptic intubation: use of simulators v. traditional teaching. Br J Anaesth. 1988;61:217–20
47. Herman NL, Carter B, Van Decar TK. Cricoid pressure: teaching the recommended level. Anesth Analg. 1996;83:859–63
48. Yee B, Naik VN, Joo HS, Savoldelli GL, Chung DY, Houston PL, Karatzoglou BJ, Hamstra SJ. Nontechnical skills in anesthesia crisis management with repeated exposure to simulation-based education. Anesthesiology. 2005;103:241–8
49. Fahy BG, Chau DF, Owen MB. The effectiveness of a simple novel approach on electroencephalograph instruction for anesthesiology residents. Anesth Analg. 2008;106:210–4
50. Johnson KB, Syroid ND, Drews FA, Ogden LL, Strayer DL, Pace NL, Tyler DL, White JL, Westenskow DR. Part task and variable priority training in first-year anesthesia resident education: a combined didactic and simulation-based approach to improve management of adverse airway and respiratory events. Anesthesiology. 2008;108:831–40
51. Orebaugh SL, Bigeleisen PE, Kentor ML. Impact of a regional anesthesia rotation on ultrasonographic identification of anatomic structures by anesthesiology residents. Acta Anaesthesiol Scand. 2009;53:364–8
52. Zausig YA, Grube C, Boeker-Blum T, Busch CJ, Bayer Y, Sinner B, Zink W, Schaper N, Graf BM. Inefficacy of simulator-based training on anaesthesiologists’ non-technical skills. Acta Anaesthesiol Scand. 2009;53:611–9
53. Bruppacher HR, Alam SK, LeBlanc VR, Latter D, Naik VN, Savoldelli GL, Mazer CD, Kurrek MM, Joo HS. Simulation-based training improves physicians’ performance in patient care in high-stakes clinical setting of cardiac surgery. Anesthesiology. 2010;112:985–92
54. Ashurst N, Rout CC, Rocke DA, Gouws E. Use of a mechanical simulator for training in applying cricoid pressure. Br J Anaesth. 1996;77:468–72
55. Park CS, Rochlen LR, Yaghmour E, Higgins N, Bauchat JR, Wojciechowski KG, Sullivan JT, McCarthy RJ. Acquisition of critical intraoperative event management skills in novice anesthesiology residents by using high-fidelity simulation-based training. Anesthesiology. 2010;112:202–11
56. Steadman RH. Improving on reality: can simulation facilitate practice change? Anesthesiology. 2010;112:775–6
57. Baker PA, Weller JM, Greenland KB, Riley RH, Merry AF. Education in airway management. Anaesthesia. 2011;66(Suppl 2):101–11
58. Curry S. The adult learner. Int Anesthesiol Clin. 2008;46:17–26
59. de Oliveira Filho GR, Schonhorst L. Problem-based learning implementation in an intensive course of anaesthesiology: a preliminary report on residents’ cognitive performance and perceptions of the educational environment. Med Teach. 2005;27:382–4
60. Rodriguez-Paz JM, Kennedy M, Salas E, Wu AW, Sexton JB, Hunt EA, Pronovost PJ. Beyond “see one, do one, teach one”: toward a different training paradigm. Postgrad Med J. 2009;85:244–9
61. Houben KW, van den Hombergh CL, Stalmeijer RE, Scherpbier AJ, Marcus MA. New training strategies for anaesthesia residents. Curr Opin Anaesthesiol. 2011;24:682–6
62. McGaghie WC, Issenberg SB, Petrusa ER, Scalese RJ. Effect of practice on standardised learning outcomes in simulation-based medical education. Med Educ. 2006;40:792–7
63. McGaghie WC, Siddall VJ, Mazmanian PE, Myers J. Lessons for continuing medical education from simulation research in undergraduate and graduate medical education. Chest. 2009;135:62S–8S
64. Issenberg SB, McGaghie WC, Petrusa ER, Lee Gordon D, Scalese RJ. Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach. 2005;27:10–28
65. Stackhouse RA. Fiberoptic airway management. Anesthesiol Clin North America. 2002;20:933–51
66. Barsuk JH, Cohen ER, Caprio T, McGaghie WC, Simuni T, Wayne DB. Simulation-based education with mastery learning improves residents’ lumbar puncture skills. Neurology. 2012;79:132–7
67. Barsuk JH, McGaghie WC, Cohen ER, O’Leary KJ, Wayne DB. Simulation-based mastery learning reduces complications during central venous catheter insertion in a medical intensive care unit. Crit Care Med. 2009;37:2697–701
68. Dunlosky J. Strengthening the student toolbox. Am Educator. 2013;37:12–21
69. Roediger HL, Karpicke JD. The power of testing memory: basic research and implications for educational practice. Persp Psychol Pract. 2006;1:181–210
70. Carpenter SK, Pashler H, Vul E. What types of learning are enhanced by a cued recall test? Psychon Bull Rev. 2006;13:826–30
71. Vojdanoska M, Cranney J, Newell BR. The testing effect: the role of feedback and collaboration in a tertiary classroom setting. Appl Cog Psychol. 2010;24:1183–95
72. Cepeda NJ, Pashler H, Vul E, Wixted JT, Rohrer D. Distributed practice in verbal recall tasks: a review and quantitative synthesis. Psychol Bull. 2006;132:354–80
73. Gerbier E, Toppino TC, Koenig O. Optimising retention through multiple study opportunities over days: the benefit of an expanding schedule of repetitions. Memory. 2014;12:1–12
74. Rohrer D, Taylor K. The shuffling of mathematical problems improves learning. Instr Sci. 2007;35:481–98
75. Brooks AL, Aldrich PJ, Tenenbaum HR. Does discovery-based instruction enhance learning? J Educ Psychol. 2011;103:1–18
76. Roberts NK, Williams RG, Kim MJ, Dunnington GL. The briefing, intraoperative teaching, debriefing model for teaching in the operating room. J Am Coll Surg. 2009;208:299–303
77. Zimmerman BJ. Investigating self-regulation and motivation: historical background, methodological developments and future prospects. Am Educ Res. 2008;45:166–83
78. Mayer RE. Should there be a three-strikes rule against pure discovery learning? The case for guided methods of instruction. Am Psychol. 2004;59:14–9
79. Macario A, Edler A, Pearl R. Training attendings to be expert teachers: the Stanford Anesthesia Teaching Scholars program. J Clin Anesth. 2008;20:241–2
80. Kiki M, Lombarts JMH, Bucx MJL, Arah OA. Development of a system for the evaluation of the teaching qualities of anesthesiology faculty. Anesthesiology. 2009;111:709–16
81. Schwartz AJ. Resident/fellow evaluation of clinical teaching: an essential ingredient of effective teacher development and educational planning. Anesthesiology. 2010;113:516–7
82. Steinert Y, Mann K, Centeno A, Dolmans D, Spencer J, Gelula M, Prideaux D. A systematic review of faculty development initiatives designed to improve teaching effectiveness in medical education: BEME Guide No. 8. Med Teach. 2006;28:497–526
83. De Oliveira GS Jr, Rahmani R, Fitzgerald PC, Chang R, McCarthy RJ. The association between frequency of self-reported medical errors and anesthesia trainee supervision: a survey of United States anesthesiology residents-in-training. Anesth Analg. 2013;116:892–7
84. Lake FR, Ryan G. Teaching on the run tips 13: being a good supervisor–preventing problems. Med J Aust. 2006;184:414–5
85. Lake FR, Hamdorf JM. Teaching on the run tips 6: determining competence. Med J Aust. 2004;181:502–3
86. Lake FR, Hamdorf JM. Teaching on the run tips 5: teaching a skill. Med J Aust. 2004;181:327–8
87. Lake FR, Ryan G. Teaching on the run tips 3: planning a teaching episode. Med J Aust. 2004;180:643–4
88. Lake FR, Vickery AW. Teaching on the run tips 14: teaching in ambulatory care. Med J Aust. 2006;185:166–7
89. Gordon J. ABC of learning and teaching in medicine. One to one teaching and feedback. Br Med J. 2005;326:543–5
90. Neher JO, Gordon KC, Meyer B, Stevens N. A five-step “microskills” model of clinical teaching. J Am Board Fam Pract. 1992;5:419–24
91. Gallagher P, Tweed M, Hanna S, Winter H, Hoare K. Developing the one-minute preceptor. Clin Teach. 2012;9:358–62
92. Baker K. Clinical teaching improves with resident evaluation and feedback. Anesthesiology. 2010;113:693–703
93. Tiberius RG, Sackin HD, Slingerland JM, Jubas K, Bell M, Matlow A. The influence of student evaluative feedback on the improvement of clinical teaching. J Higher Ed. 1989;60:665–81
94. Cohan RH, Dunnick NR, Blane CE, Fitzgerald JT. Improvement of faculty teaching performance: efficacy of resident evaluations. Acad Radiol. 1996;3:63–7
95. Glavin RJ. Excellence in anesthesiology: the role of nontechnical skills. Anesthesiology. 2009;110:201–3
96. Wachtel RE, Dexter F. Curriculum providing cognitive knowledge and problem-solving skills for anesthesia systems-based practice. J Grad Med Educ. 2010;2:624–32
97. Gaba DM. Crisis resource management and teamwork training in anaesthesia. Br J Anaesth. 2010;105:3–6
98. Schulz CM, Endsley MR, Kochs EF, Gelb AW, Wagner KJ. Situation awareness in anesthesia: concept and research. Anesthesiology. 2013;118:729–42
99. Gaiser RR. The teaching of professionalism during residency: why it is failing and a suggestion to improve its success. Anesth Analg. 2009;108:948–54
100. Hindman BJ, Dexter F, Kreiter CD, Wachtel RE. Determinants, associations, and psychometric properties of resident assessments of anesthesiologist operating room supervision. Anesth Analg. 2013;116:1342–51
101. Haydar B, Charnin J, Voepel-Lewis T, Baker K. Resident characterization of better-than- and worse-than-average clinical teaching. Anesthesiology. 2014;120:120–8
102. Wong A. Teaching, learning, and the pursuit of excellence in anesthesia education. Can J Anesth. 2012;59:171–81
103. de Oliveira Filho GR, Vieira JE. The relationship of learning environment, quality of life, and study strategies measures to anesthesiology resident academic performance. Anesth Analg. 2007;104:1467–72
104. Smith A, Goodwin D, Mort M, Pope C. Expertise in practice: an ethnographic study exploring acquisition and use of knowledge in anaesthesia. Br J Anaesth. 2003;91:319–28
105. Birnbach DJ, Santos AC, Bourlier RA, Meadows WE, Datta S, Stein DJ, Kuroda MM, Thys DM. The effectiveness of video technology as an adjunct to teach and evaluate epidural anesthesia performance skills. Anesthesiology. 2002;96:5–9
106. Ericsson KA. Necessity is the mother of invention: video recording firsthand perspectives of critical medical procedures to make simulated training more effective. Acad Med. 2014;89:17–20
107. Davis DA, Mazmanian PE, Fordis M, Van Harrison R, Thorpe KE, Perrier L. Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. JAMA. 2006;296:1094–102
108. Kruger J, Dunning D. Unskilled and unaware of it: how difficulties in recognizing one’s own incompetence lead to inflated self-assessments. J Pers Soc Psychol. 1999;77:1121–34
109. Ringsted C, Østergaard D, van der Vleuten CP. Implementation of a formal in-training assessment programme in anaesthesiology and preliminary results of acceptability. Acta Anaesthesiol Scand. 2003;47:1196–203
110. Gawande A. Top athletes and singers have coaches. Should you? October 3, 2011;9 New Yorker
111. Davis DA, Galbraith R. Continuing medical education effect on practice performance. Chest. 2009;135:42S–8S
112. Levine AI, Flynn BC, Bryson EO, Demaria S Jr. Simulation-based Maintenance of Certification in Anesthesiology (MOCA) course optimization: use of multi-modality educational activities. J Clin Anesth. 2012;24:68–74
113. McIvor W, Burden A, Weinger MB, Steadman R. Simulation for maintenance of certification in anesthesiology: the first two years. J Contin Educ Health Prof. 2012;32:236–42
114. Krampe RT, Ericsson KA. Maintaining excellence: deliberate practice and elite performance in young and older pianists. J Exp Psychol Gen. 1996;125:331–59
115. Young BW, Medic N, Weir PL, Starkes JL. Explaining performance in elite middle-aged runners: contributions from age and from ongoing and past training factors. J Sport Exerc Psychol. 2008;30:737–54
116. van de Wiel MW, Van den Bossche P, Janssen S, Jossberger H. Exploring deliberate practice in medicine: how do physicians learn in the workplace? Adv Health Sci Educ Theory Pract. 2011;16:81–95
117. Regehr G, Mylopoulos M. Maintaining competence in the field: learning about practice, through practice, in practice. J Contin Educ Health Prof. 2008;28(Suppl 1):S19–23
