A key principle of competency-based medical education is the premise that trainee proficiency progresses over the continuum of training. Educational milestones are believed to plot this timeline and benchmark residents’ progress throughout training as they move toward independent practice. In 2009, the Accreditation Council for Graduate Medical Education (ACGME) in the United States introduced the concept of competency-based assessments, or milestones, as part of the Next Accreditation System in partnership with the member boards of the American Board of Medical Specialties.1 Each specialty was charged with: (1) defining and refining the language of the competencies within the context of the specialty; (2) identifying expected levels of performance at the completion of each year of graduate medical education training; and (3) identifying and/or developing tools to assess milestones. Before that time, traditional training was time-based with global assessments tied to end of rotation evaluations. Each specialty was tasked with the creation of their own framework of competency assessments based on the 6 core competencies, namely; patient care, medical knowledge, practice-based learning and improvement, systems-based practice, interpersonnel and communication skills, and professionalism.2–4 The anesthesiology work group released 25 milestones in March 2014.5 Although some specialties have included Entrustable Professional Activities into assessments, anesthesiology reports milestones alone.6,7 The milestones are used in residency programs across the United States to chart the progress of individual residents, provide detailed feedback to trainees, and also determine curriculum efficacy. However, the anesthesiology milestones are generic and not subspecialty-specific. This led to several members of the Society for Neuroscience in Anesthesiology & Critical Care (SNACC) expressing the need for Neuroanesthesiology-specific milestones for resident education. As a result, SNACC appointed a task force to adapt the ACGME anesthesia milestones for use in Neuroanesthesiology rotations, create a milestone reporting worksheet, and provide recommendations for implementation of Neuroanesthesiology milestones. This report describes the process for the development of Neuroanesthesiology milestones, and presents the final agreed milestones following consultation with SNACC members and approval by the SNACC leadership.
SNACC appointed a 7-member task force and an 8-member advisory committee comprising senior Neuroanesthesiology educators in the United States. Task force members had served as institutional clinical competence committee chair/member and had experience in curriculum design/evaluation. The task force identified curricula and compiled the goals and objectives used by various residency programs in the United States. The programs represented by the members of the task force and the advisory committee shared their curricular documents. Following review and discussion, a final curriculum with clearly defined goals and objectives was drafted. Next, the task force developed the Neuroanesthesia milestones based on the 25 existing ACGME anesthesia milestones. The intent was to replace the nonspecific language of the anesthesia milestones, guided by the goals and objectives in the newly drafted Neuroanesthesia curriculum. There were extensive discussions whether all existing anesthesiology milestones should be included, and whether additional milestones should be added. The consensus was that milestones should be comprehensive, representing a minimum set of goals, but not exhaustive. After incorporating the suggestions from the advisory committee, the milestones were sent to all members of the task force, advisory committee and SNACC leadership to pilot and evaluate for 3 months in their respective programs. Those contributing to the pilot were asked to compare the newly developed SNACC milestones and reporting worksheet to their existing assessment tools for Neuroanesthesia rotations. Feedback was elicited from faculty as well as residents. The milestones were used in a variety of ways based on individual program preference. They were used by faculty for end of day debrief or end of rotation assessment, and by residents as a self-assessment tool before and following rotation, or as an end of day/end of rotation assessment. The milestones were further edited to incorporate the feedback from various programs, as deemed suitable by the task force. The ACGME was contacted for permission to adapt and publish the anesthesiology milestones, and after review the Neuroanesthesiology milestones, gave consent and issued the license in August 2017. The milestones were then reviewed and approved by the SNACC Board of Directors before soliciting feedback from all SNACC members through a link on the SNACC website. A final draft of the milestones was then prepared.
Sixteen institutions piloted the milestones, representing a wide range of program sizes with 5 to 30 residents per year. The programs had 4 to 15 faculty that routinely participated in evaluating residents on Neuroanesthesiology rotation. At the time of piloting only 11 programs had an evaluation reporting worksheet/tool specific for a Neuroanesthesia rotation. For the pilot, 8 (50%) programs used the SNACC milestones reporting worksheet for daily evaluation, 10 (62.5%) for end of rotation evaluation by each faculty, and 5 (31%) for end of rotation collective evaluation by faculty. A total of 142 residents (range, 1 to 30 per program) and 43 faculty (range, 1 to 8 per program) participated in the pilot. In total, 57% of programs believed that the SNACC milestones worksheet effectively captures the milestones specifically relevant to Neuroanesthesiology. The 44% of programs strongly agreed and 38% agreed that it was more accurate than their current method. The 50% of programs strongly agreed and 25% agreed that the SNACC milestones worksheet provides better feedback to residents than their current method. However, 69% found it more time-consuming and only 25% thought it was easier to use than their current method. The general feedback was that the SNACC milestones were “comprehensive,” “specific to Neuroanesthesiology,” “clear and unambiguous,” and “detailed and helpful in providing feedback.” However, the most common concerns were that they were “too long,” “time-consuming,” and “not very easy to use.” The majority of the suggestions for improvement revolved around making the milestone document more concise and user-friendly. All programs were willing to adapt the SNACC milestones, and some recommended developing electronic/cell phone application-based versions of the milestone worksheet.
The task force reviewed all the feedback and finally recommended 12 Neuroanesthesiology-specific milestones in 5 major ACGME domains that were identified as most pertinent to this subspecialty rotation. These pertain to patient care (7 milestones), medical knowledge (2 milestones), practice-based learning and improvement (1 milestone), and interpersonal and communication skills (2 milestones). An example of a milestone (patient care 7: technical skills: use and interpretation of monitoring and equipment) is shown in Table 1, and all are available in the supplementary material. Each milestone is described in detail with clear description of expectations at various levels of training.
- Preanesthetic patient evaluation, assessment, and preparation (Supplementary Table 1, Supplemental Digital Content 1, http://links.lww.com/JNA/A103).
- Anesthetic plan and conduct.
- Periprocedural pain management.
- Crisis anticipation and management.
- Management of the critically ill patient during transport.
- Technical skills: airway management.
- Technical skills: use and interpretation of monitoring and equipment (Table 1).
Medical Knowledge Care
- Knowledge of biomedical, clinical, epidemiological, and social-behavioral sciences as outlined in the American Board of Anesthesiology Content Outline (Supplementary Table 2, Supplemental Digital Content 2, http://links.lww.com/JNA/A104).
- Knowledge of neuroimaging and multimodality neurological monitoring in operating room and critical care.
Practice-based Learning and Improvement
- Analysis of practice to identify areas in need of improvement. Incorporation of feedback and new evidence-based information into practice improvement (Supplementary Table 3, Supplemental Digital Content 3, http://links.lww.com/JNA/A105).
Interpersonal and Communication Skills
- Communication with patients and families; informed consent, conflict management and disclosure of errors (Supplementary Table 4, Supplemental Digital Content 4, http://links.lww.com/JNA/A106).
- Communication with other professionals.
To facilitate assessment, the task force also recommends classifying common Neuroanesthesia procedures into 2 categories (Table 2).
This report describes the Neuroanesthesiology milestones developed by a SNACC task force. Following a 3-month pilot, the milestones were modified and subsequently incorporated inputs from SNACC members. The ACGME provided permission and license for this purpose. These milestones are expected to meet an important educational need for Neuroanesthesiology training.
With the expansion of medical knowledge and technology, changes in government regulations, and societal pressure for improved patient safety, it has become essential to show that graduating physicians are capable and competent. The ACGME milestones were initiated to provide a framework for programs to assess the trainee’s competencies over training toward the goal of ability to practice anesthesiology independently. The milestones also allowed training programs to assess curriculae and compare outputs with other programs. While each specialty was tasked with the creation of their own framework of competency assessments based on the 6 ACGME core competencies, there was no clear requirement to develop subspecialty-specific milestones. Nevertheless, publication of the core competencies was soon followed by initiatives to develop milestones for fellowship training in various subspecialties in Anesthesiology. In July 2015, milestones for anesthesiology fellowships in Adult Cardiothoracic Anesthesiology,8 Obstetric Anesthesiology,9 Pediatric Anesthesiology,10 and Critical Care11 were created by ACGME and ABA, and these became the guideline for ACGME-approved fellowship programs for education and evaluation of fellows’ competency in these subspecialties. Later, some Anesthesiology departments created subspecialty milestones for their individual programs. For example, Stanford University developed the Stanford Milestones for Head and Neck Anesthesia & Advanced Airway Management.12 As Neuroanesthesiology in the United States is not an ACGME-approved fellowship, there was no ACGME directive to develop Neuroanesthesiology milestones. However, educators struggled using the generic Anesthesiology milestones to evaluate residents during Neuroanesthesiology rotations, and this led SNACC to take the initiative to develop Neuroanesthesiology milestones. While the milestones described in this report are intended for use in residency programs, the continuum in competency progression suggests that may be adapted for use in fellowship training as well.
The SNACC milestones provide a tool to assess trainees during Neuroanesthesia rotations. They are succinct, with specific description of expectations to avoid ambiguity. The task force recommends that each resident should undergo 2 Neuroanesthesiology rotations—a basic rotation in the CA2 year and an advanced rotation in the CA3 year. The aim should be to accomplish level 3 or higher milestones at the end of the basic rotation, and level 4 or higher milestones at the end of the advanced rotation. However, it is for individual programs to decide the timing of these rotations. The task force recognizes that the power of the milestones is that the residents can be identified accomplishing a certain “level” of achievement irrespective of the timing of the rotation. The individual program should also decide the consequences of not meeting the desired level of training milestones. According to the ACGME, level 4 is designed as the graduation target but not a graduation requirement. The decision of readiness for graduation is the purview of individual programs, as is the implication of a resident not meeting expected milestones. A summative reporting of the performance during the Neuroanesthesiology rotation should involve the selection of a level for each milestone that the resident consistently and substantially demonstrates. In accordance with the general ACGME recommendations, the task force recommends the following general interpretation of levels for Neuroanesthesiology:
Level 1: The resident demonstrates milestones expected of a resident who has completed 1 postgraduate year of education in either an integrated anesthesiology program, or another preliminary education year before entering the CA1 year in anesthesiology.
Level 2: The resident demonstrates milestones expected of a resident in anesthesiology residency before significant experience in Neuroanesthesiology.
Level 3: The resident demonstrates milestones expected of a resident after having experience in Neuroanesthesiology (ie, after completion of the first Neuroanesthesiology rotation).
Level 4: The resident substantially fulfills the milestones expected of Neuroanesthesiology in an anesthesiology residency, and is ready to transition to independent practice. This level is designed as the graduation target.
Level 5: The resident has advanced beyond performance targets defined for residency, and is showing “aspirational” goals for a fellowship trained Neuroanesthesiologist or an individual who has been practicing Neuroanesthesiology independently for several years. It is expected that only a few exceptional residents will reach this level for selected milestones.
The SNACC task force recommends that the milestones reported here should be assessed for all residents during their Neuroanesthesiology rotation because they are the ones most pertinent to the subspecialty rotation. While the competencies captured by the remaining ACGME anesthesiology milestones may also be observed during Neuroanesthesiology rotations, they may not be specific/unique to this rotation. However, for specific residents and situations, individual programs may decide to include additional milestones described by the ACGME.
These milestones were developed using the ACGME framework and, hence, are primarily applicable to residency programs in the United States. However, since the milestones address competencies in Neuroanesthesiology that are more broadly applicable, they may be adapted and customized for use outside of the United States.
The major strength of these milestones is the rigorous process of their development. This included piloting by a large number of faculty and residents in Anesthesiology departments representing the diversity of training programs in the United States, as well as inputs from the SNACC membership representing content expertise. Moreover, the milestones were abridged after the pilot phase to make them more succinct and easier to use. Nevertheless, the uptake and effectiveness of the milestones will require further evaluation. The task force has not made any specific recommendations on how the milestones should be evaluated. While direct clinical observation and knowledge testing can evaluate some milestones, there may be a role for simulation.13,14 This decision is left to individual programs.
In summary, SNACC has provided the first set of milestones to evaluate resident training in Neuroanesthesiology using the ACGME framework. These milestones are expected to be adapted and assist in objective evaluation of individual residents as well as entire training programs.
The authors acknowledge the members of the advisory committee: Jack Buckley, MD (University of California Los Angeles, Los Angeles, CA), Catherine M. Christenson, MD (University of Vermont, Burlington, VT), Robert A. Peterfreund, MD (Mass General Hospital Harvard University, Boston, MA), Sergey V. Pisklakov, MD (Albert Einstein School of Medicine, New York, NY), Barbara M. Rogers, MD (The Ohio State University, Columbus, OH), David Schreibman, MD (University of Maryland, Baltimore, MD), Naveen Vanga, MD (University of Texas, Houston, TX), Matt Whalin, MD (Emory University, Atlanta, GA). Also, the authors acknowledge ACGME (Accreditation Council for Graduate Medical Education) for the permission to adapt milestones for Neuroanesthesiology and for issuing the license for the same to SNACC; and SNACC Board of Directors and members for their inputs and feedback on milestones.
1. Nasca TJ, Philibert I, Brigham T, et al. The next GME accreditation system—rationale and benefits. N Engl J Med. 2012;366:1051–1056.
2. Hicks PJ, Englander R, Schumacher DJ, et al. Pediatrics milestone
project: next steps toward meaningful outcomes assessment. J Grad Med Educ. 2010;2:577–584.
3. Leep Hunderfund AN, Reed DA, Starr SR, et al. Ways to write a milestone
: approaches to operationalizing the development of competence in graduate medical education
. Acad Med. 2017;92:1328–1334.
4. Meade LB, Borden SH, McArdle P, et al. From theory to actual practice: creation and application of milestones in an internal medicine residency
program, 2004-2010. Med Teach. 2012;34:717–723.
5. Schartel SA, Kuhn C, Culley DJ, et al. Development of the anesthesiology educational milestones. J Grad Med Educ. 2014;6:12–14.
6. Ten Cate O. Nuts and bolts of entrustable professional activities. J Grad Med Educ. 2013;5:157–158.
7. Ten Cate O. Entrustability of professional activities and competency-bases training
. Med Educ. 2005;39:1176–1177.
13. Isaak RS, Chen F, Martinelli SM, et al. Validity of simulation-based assessment for accreditation council for graduate medical education milestone
achievement. Simul Healthc. 2018;13:201–210.
14. Sinz E. Simulation for anesthesiology milestones. Int Anesthesiol Clin. 2015;53:23–41.