How Clinical Supervisors Conceptualize Procedural Entrustment: An Interview-Based Study of Entrustment Decision Making in Endoscopic Training : Academic Medicine

Secondary Logo

Journal Logo

Research Reports

How Clinical Supervisors Conceptualize Procedural Entrustment: An Interview-Based Study of Entrustment Decision Making in Endoscopic Training

Jeyalingam, Thurarshen MD, MSc1; Brydges, Ryan PhD2; Ginsburg, Shiphra MD, PhD3; McCreath, Graham A.4; Walsh, Catharine M. MD, MEd, PhD5

Author Information
Academic Medicine 97(4):p 586-592, April 2022. | DOI: 10.1097/ACM.0000000000004566

Abstract

Competency-based medical education (CBME) emphasizes achievement of performance outcomes rather than time in training. Entrustment, a central construct within CBME, has been conceptualized as an active choice by clinical supervisors regarding the degree of supervision to offer a trainee and reflects their underlying cognitive processing of the trainee’s clinical performance. 1,2 Entrustment decisions, which are the discrete decisions a supervisor makes in the training setting, can be made in the formative context, where supervisors decide what degree of supervision to offer a trainee in day-to-day clinical tasks. These decisions can also be made in the summative context, where supervisors contribute data toward signing off on a trainee for unsupervised practice. 3,4 In many CBME systems, both forms of entrustment are operationalized in specialty-specific tasks called entrustable professional activities (EPAs). 5,6

In assessing an EPA, a supervisor makes an in-the-moment judgment about whether a trainee is “entrustable” within a situated, workplace-based assessment context. In making this decision, supervisors have been shown to consider multiple factors, including those related to themselves, the trainee, the supervisor–trainee relationship, the task, and the context. 7–15 While such categories are likely common to the entrustment decision making for all clinical skills, the entrustment of procedural skills may be unique, given that procedures have a prominent technical element, have the potential to cause immediate patient harm, and may require supervisors to relinquish control of operating instruments to teach and assess. 16,17 Furthermore, procedures often occur in complex team settings, requiring trainees to draw upon multiple nontechnical skills. While several groups have already developed EPAs for specific procedural skills across specialties, 18–22 researchers have not explored how supervisors account for and synthesize trainees’ technical and nontechnical skills, as well as contextual factors, when making entrustment decisions about procedural EPAs.

Supervisors engage certain observational and decision-making processes in making assessment decisions, collectively referred to as their rater cognition. 23 Within the literature on rater cognition, researchers have questioned how supervisors synthesize various factors when forming their frame of reference about a construct. In this process, the rater’s frame of reference may develop to emphasize the person level (e.g., impressions formed on the basis of perceived trainee traits), the performance level (i.e., how a particular effort at a task manifests), or a unique combination of both. 23–25 Without a common frame of reference, supervisors may also develop their own impressions of which performance dimensions have the most value, leading to idiosyncratic decision making. 26,27

In this study, we aimed to characterize how supervisors conceptualize entrustment and make decisions about trainees during EPAs related to gastrointestinal endoscopy. We chose endoscopy because it is a skill performed by many specialists (gastroenterologists, surgeons, and primary care physicians); it includes technical, cognitive, and nontechnical elements; it encompasses both simple diagnostic (e.g., colonoscopy) and complex therapeutic procedures (e.g., endoscopic mucosal resection); and it is performed in both adult and pediatric contexts. 28,29 Within these unique layers of complexity and variability, we aimed to identify the factors supervisors consider in their entrustment decision making in formative, day-to-day training interactions in workplace-based contexts. Further, we aimed to understand how supervisors integrate these factors together, which may help programs support their supervisors in assessing entrustment and their trainees in achieving entrustment.

Method

We conducted an interview-based study using a constructivist grounded theory approach to explore how clinical supervisors conceptualize endoscopic entrustment and how they perceive making formative, in-the-moment entrustment decisions in the workplace. The interviews followed a semistructured guide (Supplemental Digital Appendix 1 at https://links.lww.com/ACADMED/B219) intended to elicit supervisors’ perceptions of and experiences with endoscopic entrustment, the factors they considered in making their entrustment decisions, and how they integrated these experiences and factors to generate entrustment decisions. Interviews were conducted by the primary investigator (T.J.), who has expertise in endoscopy. For clarity, we piloted the interview guide with 2 endoscopy supervisors in our institution.

We purposively sampled endoscopy supervisors from multiple specialties (adult and pediatric gastroenterology, surgery, and primary care), geographic regions, and experience levels, with the goal of enrolling a broad cross-section of endoscopists. We purposively sampled some participants who held educational roles within gastroenterology or endoscopy-focused organizations (e.g., the American Society for Gastrointestinal Endoscopy and the Canadian Association of Gastroenterology), and we specifically selected others not in education leadership roles to explore alternative viewpoints. We recruited study participants using an invitation email, which we sent to 49 potential participants. We continued recruitment until theoretical sufficiency (i.e., the point at which we deemed our depth of understanding was sufficient and where additional interviews did not result in further modification to our framework). 30

Analysis

All interviews were conducted by phone, audio-recorded, and transcribed verbatim between April and November 2019. Consistent with the tenets of constructivist grounded theory, we began data analysis alongside interviews. 31,32 We analyzed the interview transcripts using an iterative, constant comparative approach, whereby 2 team members (T.J. and G.A.M.) read groups of 2 to 3 transcripts and identified preliminary codes. We coded inductively, where codes were generated from the transcripts, while also attending to existing frameworks of entrustment decision making as our sensitizing concepts. 9,14 The entire study team met approximately every 3 months to discuss and refine the evolving coding structure, themes, and framework, as well as to revise the interview guide. Before these meetings, we provided senior members of the study team (R.B., S.G., and C.M.W.) with transcripts to read as well as the most recent coding tree, which allowed them to offer specific feedback on the analysis. The primary coders (T.J., G.A.M.) then revised the interview guide, coding structure, and evolving themes on the basis of this feedback. Finally, after all transcripts were coded, the full author team grouped codes together around themes and generated conceptual connections between these themes to develop an explanatory framework. We completed our coding using Dedoose version 8.0.35 (SocioCultural Research Consultants, Los Angeles, California).

Reflexivity

Our study team comprised scientists with expertise in assessment and feedback interactions (R.B., S.G., C.M.W.), as well as qualitative research (S.G.). Three of the researchers were also clinicians (T.J., S.G., and C.M.W.). The coders possessed clinical training in gastroenterology and graduate training in health professions education research (T.J.), as well as graduate training in psychology (G.A.M.), which produced generative divergences that led to a comprehensive analytic process. The primary coder (T.J.) kept memos throughout data collection and analysis and wrote additional memos after team meetings to clarify emerging ideas.

This study was approved by the Hospital for Sick Children Research Ethics Board (Toronto, Canada).

Results

We recruited 29 participants total, including the 2 pilot supervisors. Given that the pilot interviews did not generate data significantly different from the remainder of the cohort, they were included in the analysis. Eight participants (27.6%) were women and 21 (72.4%) were men; 18 (62.1%) practiced in the United States and 11 (37.9%) in Canada. Their experience in supervising endoscopic trainees ranged from less than 1 year to 36 years. Participants practiced (and supervised trainees) in a range of specialties, including adult gastroenterology (10, 34.5%), pediatric gastroenterology (9, 31.0%), general surgery (7, 24.1%), and primary care (3, 10.3%). Interviews were between 21 and 60 minutes long and generated 369 pages of transcript data. Our analysis identified 3 major themes, which we outline below.

Participants describe entrustment in degrees and perceive their level of entrustment as fluctuating over time

Participants described their view of entrustment as varying in degrees. Although the sense of entrustment having levels or degrees was universal, how participants conceptualized entrustment was variable.

Some participants described progressively entrusting rudimentary components of an endoscopic procedure before being able to entrust more advanced skills and, ultimately, the procedure in its entirety:

My review of EPAs is that [entrustment] is a progressive metric. Progressive metric means that if you take a [procedure], if you break it down into steps, and you say [these are the constituent steps] ... that allow people to understand where they fall in the continuum of procedure development. (Participant [P]19)

One participant described a system with multiple binary decisions regarding whether the participant entrusted a trainee to perform a procedure under direct, indirect, or absent supervision (“... each of those decisions has to be a yes/no” [P4]). Another participant proposed a similar system including the additional levels of whether they entrusted trainees to perform procedures on “their long-standing patients” or “[their] family member” (P14).

Other participants perceived their degree of entrustment as manifesting in their level of preparedness to take over from a trainee under their supervision:

There are sometimes in cases, if it’s a third-year resident that’s done 100 of them or is really proficient at it, I may not put gloves on or a gown on until they seem like they’re struggling. I’ll just say, “I’ll see how far they can get on their own.” Then, there are others that there’s no way, I would put the gown on right away and kind of be right there at the bedside. (P22)

These ways of conceptualizing entrustment appeared to function on relatively sliding scales, given many participants also described their judgments of a particular trainee as fluctuating over time, without clear relationships to a trainee’s training level. These fluctuations were perceived to occur due to situational factors related to the patient, procedure, or trainee:

I see [entrustment] as a continuum that has sort of a sinusoidal pattern. It’s a learning process, meaning that there may be moments where the resident, you feel you could entrust them. And then when they are challenged cognitively again by either a higher level or new circumstance or another twist that maybe overloads them, then maybe they need a little bit more support.... And then they can be entrusted again depending on the circumstances, so that over time they build a sort of overall improvement in the skill or the activity that is being entrusted. But it can vary from situation to situation. So it’s not like every single time they do it they improve, and they improve, and they improve, and once they become entrustable they are permanently entrustable. I think that it can vary. (P7)

As demonstrated in these quotes, entrustment appears to be conceptualized as a matter of degree and as a decision that fluctuates back and forth, dependent on numerous factors in the situated assessment setting.

Participants perceive entrustment decisions in one context as influencing their decisions within and across contexts

Some participants noted that within the realm of endoscopy, their entrustment decisions about a trainee in one instance influenced their entrustment decision about another endoscopic skill. Take, for example, the following representative discussion about entrustment decisions in colonoscopy and polypectomy (2 discrete endoscopic skills):

I think in someone who’s really shown me that they have those sort of baseline [endoscopic] skills that I’m much more likely to allow them to take the next step and be comfortable with [other endoscopic skills]. But if I see someone who can’t advance the endoscope or can’t loop reduce, I’m a lot more likely to take over and probably not allow them to do a polypectomy. (P2)

Some participants also perceived entrustment as transferrable across different types of procedural contexts. Speaking of the surgical and endoscopic context, one participant reported “a tremendous degree of overlap between technical skills” and that “a trainee that’s entrusted in one technical skill likely would be entrusted in another” (P3).

Certain participants also described transferring entrustment decisions across medical and procedural contexts, such as in this discussion of entrusting a trainee to endoscopically manage a bleeding ulcer:

I knew that they were quite competent outside of the endoscopy unit in regard to their knowledge and how to appropriately approach the situation we were in. So I felt completely comfortable allowing them to do the entire procedure. (P2)

Other participants expressed that a decision to not entrust a trainee in the medical context could extend to the endoscopic context based on a perceived lack of reliability, which they felt they could infer from a trainee’s previous suboptimal medical management of a patient:

I think it just goes down to those domains that we’ve talked about, which are reliability and truthfulness. I think someone who I can’t trust to get a good enough history, who I can’t trust to manage a patient appropriately ... that would definitely impact my entrustment [decision] for the endoscopic procedure. (P15)

In this vein, participants rationalized the influence of their entrustment decision in one context on another context as being due to a “preconceived notion” from one setting “pass[ing] over into another arena” (P25), reflecting trainee-related personal or professional factors that manifest in both settings (e.g., “lack of insight” [P25]).

Despite the prior theme suggesting that entrustment may fluctuate, participants perceived that their previous entrustment decisions and other interactions could influence future entrustment decisions both within procedural domains and across clinical contexts. Some indicated that this influence may relate to certain trainee factors (e.g., reliability or insight) manifesting with relative consistency as trainees perform different tasks. The particular factors that participants considered and how these become synthesized in entrustment decisions are described next.

Participants synthesize static and dynamic factors when making entrustment decisions

Participants reported synthesizing a medley of factors in making procedural entrustment decisions. Although the general categories of factors were common across participants, individual participants reported unique combinations of factors that were personally important to them. They also assigned different weights to their personal factors in their ultimate decisions.

The list of factors that participants considered could be categorized as related to the trainee, supervisor, supervisor–trainee relationship, patient, procedure, and environment. They could also be divided into static and dynamic categories. By static, we refer to factors that persisted between training encounters over long periods of time and appeared to influence a supervisor’s baseline frame of reference for entrustment decisions. By dynamic, we refer to factors situated within any specific assessment setting that appeared to change from one training encounter to the next. Participants described synthesizing a unique combination of dynamic factors against their personal baseline informed by static factors in making entrustment decisions (Figure 1). As illustrated in Figure 1, static factors informing supervisors’ baseline frame of reference for entrustment can be conceptualized as the location of the fulcrum along a seesaw, which influences the baseline propensity of the seesaw to fall to one side or another, whereas the synthesis of dynamic factors can be conceptualized as the application of various forces along the seesaw board. Combined, these factors influence a supervisor’s decision to entrust or not, as represented by the side to which the seesaw falls.

F1
Figure 1:
Entrustment decision-making seesaw. Static factors were perceived to influence a supervisor’s baseline frame of reference with respect to entrustment (represented by the location of the seesaw fulcrum). Salient dynamic factors were then incorporated with this baseline (represented by the application of various forces along the seesaw board), resulting in an ultimate entrustment decision. Participants described variably synthesizing static and dynamic factors related to the trainee, supervisor, supervisor–trainee relationship, patient, procedure, and environment in making endoscopic entrustment decisions. Static factors include the supervisor’s own competence, disposition, and prioritization of trainee learning; the culture of the institution and specialty; and legal considerations. Dynamic factors include level of training, professional attributes, and skills of trainee; mental workload of supervisor; supervisor–trainee familiarity; complexity and risk of procedure; composition and quality of team; time constraints; and patient age, acuity, comorbidity, comfort, pathology, and safety.

Static factors.

Participants frequently considered static factors. For instance, participants’ own self-assessed competence in the procedure was noted as influencing their decision making:

When I started doing colon procedures, and I had not done very many or I was still myself on my learning curve, I wasn’t really in a position where I felt comfortable entrusting any part of that procedure to a resident because I wasn’t comfortable with it myself. (P4)

Some participants noted their frame of reference was related to their own ability to recognize and manage complications, so that they were confident they could “get the trainee out of trouble” (P9):

I think the more you are comfortable that you can rescue a situation, quite frankly, and maybe that’s recognizing it before it happens, the more comfortable you are with your own ability to see that something could go wrong, [the more likely you are to entrust]. (P21)

Some participants also described how their baseline willingness to prioritize trainee learning relative to 1 or more competing demands influenced the likelihood they would entrust. For instance, some supervisors were less likely to entrust procedures that they perceived “[don’t] matter” to trainee learning due to the perception that the trainee did not require competency in that procedure for their future practice and, therefore, the risk of making an entrustment decision was not justified (P4). In a similar fashion, others reported being less likely to entrust due to competing legal considerations:

For example, the pregnant patient who needs an [endoscopic procedure], and in those situations, it just comes down to liability, and just recognition that there are certain situations that we just need to do as attendings. No matter how good the trainee might be, I’m potentially putting them in a difficult situation. (P27)

Participants who practiced in the pediatric setting reported feeling less likely to entrust trainees, due to a “hands-on” and “paternalistic” culture, which they attributed to being “cautious or protective of the patients” (P13). Several participants also reported that their institutional culture of having to “supervise the entire colonoscopy from insertion to removal of the scope” (P4) left them unable to entrust in practice:

I think in a sense we’re told to baby them more and not consider them “the” provider. And so, by definition, that makes me the provider.... I think culturally at our institution, if an error happened or a complication occurred, that would more be my doing.... I would not blame the fellow. I would say I was the one who missed that I needed to step in, or change something, or not trust them to do it. (P21)

Several participants also alluded to a supervisor’s general tendency to entrust; they typically attributed this to the “temperament, personality, and personal characteristics of the [supervisor] that go into whether they let [a trainee] continue to try something” (P21).

Dynamic factors.

While static factors seemed to influence participants’ baseline frame of reference for entrustment decisions, participants also reported attending to multiple performance dimensions dynamically. The particular factors supervisors prioritized and combined in decision making varied across supervisors and supervision encounters. Participants described considering the net influence of these dynamic factors (as represented by the application of a force to the seesaw board in Figure 1) against their baseline frame of reference (as represented by the location of the fulcrum along the seesaw in Figure 1), which led to their ultimate decision to entrust or not (as represented by which side the seesaw ultimately falls to in Figure 1).

As an example, one participant reported considering multiple dynamic factors (i.e., a trainee’s personal and professional attributes, training level, technical and nontechnical skills, and the nature of the supervisor–trainee relationship) against a static factor (i.e., their own competence):

Factors specific to the trainee are their attitude ... their level of experience, their knowledge of their own limitations, and their willingness to ask for help and kind of respond to feedback, and also their technical abilities and clinical judgment. And I think with respect to the trainees, also the relationship of the supervisor and the trainee.... And then with respect to the supervisor.... It’s their own kind of comfort level of the supervisor in terms of the actual procedure being performed and their level of experience. (P4)

Another participant reported considering a different combination of dynamic factors, including a particular trainee’s technical skills, the quality and composition of the care team (e.g., presence of an anesthetist), and the patient’s acuity, in addition to a different static factor (i.e., institutional culture):

The environment in the room does make a difference in the way you entrust because there’s often a whole variety of other behaviors, and performances, and strengths, or weaknesses in the room, and they influence what you’re entrusting. You think it’s a very focused [technical] issue, but if you’re entrusting them to supervise as well as perform the procedure, there’s a lot of distraction and interruption that a [trainee] needs to be prepared to deal with. And so, the environment of who is there, the acuity of the patient, the support staff available, presence or not of anesthesia, a lot of these other components influence how much you’re entrusting someone when you give them that privilege, so that would influence a decision. Within a department, I don’t think there’s a lot more outside other than I’m sure some departments have a laissez-faire attitude towards fellows’ independence, and others, perhaps like ours, have a ... much more conservative approach. (P8)

To summarize, participants considered static factors, which were relatively constant between training encounters and appeared to influence their baseline frame of reference in making entrustment decisions. Further, they described integrating performance dimensions dynamically in a fashion that was variable between supervisors and training encounters. They conceptualized their entrustment in a trainee as a matter of degree and perceived that it could fluctuate over time. Despite this framework, participants also described entrustment decisions in one context as influencing their entrustment decisions in another context, potentially owing to certain trainee factors manifesting across contexts.

Discussion

In our study, participants reported conceptualizing entrustment as fluctuating in degrees across trainees and assessment scenarios. With the same trainee, they also reported that one entrustment decision can influence future decisions both within and across procedural and clinical contexts. In forming their final entrustment decisions, they described considering a unique combination of static factors (i.e., those that tend to persist across training encounters and often years) that appeared to set their frame of reference, and they also described attending to performance dimensions that fluctuated dynamically between training encounters.

Our participants each engaged in an idiosyncratic cognitive process where different static and dynamic factors were synthesized in entrustment decisions. These factors were related to the trainee, supervisor, supervisor–trainee relationship, patient, procedure, and environment—categories consistent with what has been reported in the literature. 3,5,7,9,33 Participants’ experience with synthesizing these competing factors also aligns with previous findings. 5,13,34,35 However, in our study, the particular factors individual participants used to inform their baseline frame of reference and the performance dimensions they considered appeared to be different. This subjectivity in their entrustment decision making seemed situated in the context in which they assessed. For example, one participant reported liability issues as the most influential factor in decision making, while another reported the participant’s own procedural competence as most important. Such rater idiosyncrasies have been described in other assessments and mirror the concept of differential salience, where raters emphasize different elements in their assessments of the same task. 23,36,37

Our finding that supervisors conceptualize entrustment as varying in degree and evolving over time aligns with entrustment in the formative setting, where supervisors make in-the-moment decisions while weighing multiple, often competing, factors. 2 This conceptualization contrasts with a competency committee’s decision regarding summative entrustment, which is supposed to be binary in nature. 2 Our participants’ reported fluctuations in entrustment may be related to multiple contextual factors, as well as expected variations in trainees’ performance over time. 38 We propose that these fluctuations reveal participants’ tendency to think of entrustment in a longitudinal fashion and is consistent with work by Melvin et al that EPA assessments rarely reflect true point-in-time checks, given they often occur in the context of longitudinal supervisor–trainee relationships. 39

Our participants’ justification of why their entrustment decisions could transfer between contexts (e.g., clinical, endoscopic, surgical) centered on certain trainee characteristics (e.g., technical skills, knowledge, reliability) that they believed manifested across contexts. This finding echoes descriptions of initial trust, where entrustment decisions are based on first impressions rather than prolonged observations, as well as the assimilation effect, where prior trainee performance influences current rater judgment. 3,40 We noted heterogeneity in the trainee factors participants cited in explaining how their entrustment decisions might transfer, however, further highlighting the idiosyncrasy in their decision making. It is worth noting that participants’ perception that entrustment decisions may transfer due to trainee factors contradicts the prevailing belief that these seemingly stable trainee traits are actually variable states that can change across contexts. 41,42 Furthermore, the idea that entrustment decisions may transfer implies that these decisions exist in a relatively steady state, which is in tension with our participants’ description of entrustment as fluctuating. In addition, our participants’ longitudinal conceptualization of entrustment is at odds with how formative EPAs are usually assessed (i.e., as single, point-in-time assessments), which may cause further cognitive tensions for raters. Such rater inconsistencies and cognitive tensions warrant further study in the context of entrustment decision making.

Our findings have several implications for academic medicine. Despite how quickly CBME training and assessment systems have been adopted, a recent critical narrative review demonstrated that much of the evidence pertaining to the core assumptions about CBME was mixed and derived from a limited range of research designs (i.e., quantitative analyses). 43 With respect to EPAs in particular, the research agenda has largely focused on EPA development and not on understanding implementation or validation of assessment practices. 44 Relating to validation, our findings that participants idiosyncratically selected and weighed factors, that they reported variability in how they formed frames of reference and attended to performance dimensions, and that they described entrustment as fluctuating each represent important validity implications for the current use of EPAs in CBME (i.e., as decontextualized, point-in-time entrustment decisions). Taken together, our findings support the perspective that individual EPA assessments from a single supervisor, context, or time point do not represent a trainee’s abilities fully. Thus, we suggest that competence committees should strive to obtain multiple assessments across supervisors, contexts, and time points before rendering summative decisions.

Our findings also have implications for procedural training specifically. Given that procedural experience facilitates trainee learning and that entrustment facilitates procedural experience, optimizing training conditions to favor entrustment may also facilitate learning. 45,46 The static factors described by our participants can present lasting barriers to entrustment. Focused efforts may be needed to change the system in which these factors operate toward promoting entrustment and subsequent opportunities for learning. For example, institutions could change policies to allow for more trainee involvement in procedures, or supervisors could be more deliberately selected on the basis of their procedural competencies and abilities to manage complications. Thus, our framing of these factors as static or dynamic is potentially useful because it allows programs to explore which factors can be modified to optimize conditions for trainee entrustment.

Our study has limitations worth mentioning. First, participants were not situated within an actual assessment setting and thus could have been describing hypotheticals prone to recall biases. To mitigate this limitation, we asked participants to select and reflect on specific, real-life training experiences during our study interviews. Second, we used only 1 set of procedural skills (i.e., endoscopy) to study procedural entrustment more broadly, and therefore, our findings may not generalize to other procedural skills.

Conclusions

Procedural entrustment appears to be a complex process in which supervisors idiosyncratically synthesize static and dynamic factors to produce situated entrustment decisions. In selecting who assesses EPAs and in which context they are assessed, programs might consider the list of static and dynamic factors we identified, with the goal of optimizing them in ways that facilitate trainees’ learning and ultimate achievement of entrustment. Further studies of how these factors interact to influence supervisors’ decision-making processes have the potential to inform CBME faculty development practices, as well as competency committee practices when aggregating faculty entrustment decisions.

References

1. ten Cate O. When I say … entrustability. Med Educ. 2020;54:103–104.
2. ten Cate O, Schwartz A, Chen HC. Assessing trainees and making entrustment decisions: On the nature and use of entrustment-supervision scales. Acad Med. 2020;95:1662–1669.
3. ten Cate O, Hart D, Ankel F, et al.; International Competency-Based Medical Education Collaborators. Entrustment decision making in clinical training. Acad Med. 2016;91:191–198.
4. Pack R, Lingard L, Watling C, Cristancho S. Beyond summative decision making: Illuminating the broader roles of competence committees. Med Educ. 2020;54:517–527.
5. Hauer KE, ten Cate O, Boscardin C, Irby DM, Iobst W, O’Sullivan PS. Understanding trust as an essential element of trainee supervision and learning in the workplace. Adv Health Sci Educ Theory Pract. 2014;19:435–456.
6. ten Cate O. Entrustability of professional activities and competency-based training. Med Educ. 2005;39:1176–1177.
7. Tiyyagura G, Balmer D, Chaudoin L, et al. The greater good: How supervising physicians make entrustment decisions in the pediatric emergency department. Acad Pediatr. 2014;14:597–602.
8. Hauer KE, Oza SK, Kogan JR, et al. How clinical supervisors develop trust in their trainees: A qualitative study. Med Educ. 2015;49:783–795.
9. Sterkenburg A, Barach P, Kalkman C, Gielen M, ten Cate O. When do supervising physicians decide to entrust residents with unsupervised tasks? Acad Med. 2010;85:1408–1417.
10. Khan KZ, Ramachandran S, Gaunt K, Pushkar P. The Objective Structured Clinical Examination (OSCE): AMEE guide no. 81. Part I: An historical and theoretical perspective. Med Teach. 2013;35:e1437–e1446.
11. Saxon K, Juneja N. Establishing entrustment of residents and autonomy. Acad Emerg Med. 2013;20:947–949.
12. ten Cate O, Scheele F. Competency-based postgraduate training: Can we bridge the gap between theory and clinical practice? Acad Med. 2007;82:542–547.
13. Dijksterhuis MG, Voorhuis M, Teunissen PW, et al. Assessment of competence and progressive independence in postgraduate clinical training. Med Educ. 2009;43:1156–1165.
14. Choo KJ, Arora VM, Barach P, Johnson JK, Farnan JM. How do supervising physicians decide to entrust residents with unsupervised tasks? A qualitative analysis. J Hosp Med. 2014;9:169–175.
15. Holzhausen Y, Maaz A, Cianciolo AT, ten Cate O, Peters H. Applying occupational and organizational psychology theory to entrustment decision-making about trainees in health care: A conceptual model. Perspect Med Educ. 2017;6:119–126.
16. Dunkin B, Adrales GL, Apelgren K, Mellinger JD. Surgical simulation: A current review. Surg Endosc. 2007;21:357–366.
17. Sawyer T, White M, Zaveri P, et al. Learn, see, practice, prove, do, maintain: An evidence-based pedagogical framework for procedural skill training in medicine. Acad Med. 2015;90:1025–1033.
18. Pugh D, Cavalcanti RB, Halman S, et al. Using the entrustable professional activities framework in the assessment of procedural skills. J Grad Med Educ. 2017;9:209–214.
19. Lindeman B, Brasel K, Minter RM, Buyske J, Grambau M, Sarosi G. A phased approach: The general surgery experience adopting entrustable professional activities in the United States. Acad Med. 2021;96:S9–S13.
20. Rose S, Fix OK, Shah BJ, Jones TN, Szyjkowski RD. Entrustable professional activities for gastroenterology fellowship training. Gastroenterology. 2014;147:233–242.
21. Watson A, Leroux T, Ogilvie-Harris D, et al. Entrustable professional activities in orthopaedics. JB JS Open Access. 2021;6:e20.00010.
22. Woodworth GE, Marty AP, Tanaka PP, et al. Development and pilot testing of entrustable professional activities for US anesthesiology residency training. Anesth Analg. 2021;132:1579–1591.
23. Gauthier G, St-Onge C, Tavares W. Rater cognition: Review and integration of research findings. Med Educ. 2016;50:511–522.
24. Govaerts MJ, Van de Wiel MW, Schuwirth LW, van der Vleuten CPM, Muijtjens AM. Workplace-based assessment: Raters’ performance theories and constructs. Adv Health Sci Educ Theory Pract. 2013;18:375–396.
25. Gingerich A, van der Vleuten CPM, Eva KW, Regehr G. More consensus than idiosyncrasy: Categorizing social judgments to examine variability in Mini-CEX ratings. Acad Med. 2014;89:1510–1519.
26. Kogan JR, Hess BJ, Conforti LN, Holmboe ES. What drives faculty ratings of residents’ clinical skills? The impact of faculty’s own clinical skills. Acad Med. 2010;85(suppl 10):S25–S28.
27. Hodwitz K, Kuper A, Brydges R. Realizing one’s own subjectivity: Assessors’ perceptions of the influence of training on their conduct of workplace-based assessments. Acad Med. 2019;94:1970–1979.
28. Hitchins CR, Metzner M, Edworthy J, Ward C. Non-technical skills and gastrointestinal endoscopy: A review of the literature. Frontline Gastroenterol. 2018;9:129–134.
29. Walsh CM. In-training gastrointestinal endoscopy competency assessment tools: Types of tools, validation and impact. Best Pract Res Clin Gastroenterol. 2016;30:357–374.
30. Charmaz K, Belgrave L. Gubrium J, Holstein J, Marvasti A, McKinney K, eds. Qualitative interviewing and grounded theory analysis. In: The SAGE Handbook of Interview Research: The Complexity of the Craft. 2012;2nd ed. Thousand Oaks, CA: Sage Publications Inc;347–366.
31. Charmaz K. Constructivist grounded theory. J Posit Psychol. 2017;12:299–300.
32. Watling CJ, Lingard L. Grounded theory in medical education research: AMEE guide no. 70. Med Teach. 2012;34:850–861.
33. Teman NR, Gauger PG, Mullan PB, Tarpley JL, Minter RM. Entrustment of general surgery residents in the operating room: Factors contributing to provision of resident autonomy. J Am Coll Surg. 2014;219:778–787.
34. Kashner TM, Byrne JM, Chang BK, et al. Measuring progressive independence with the resident supervision index: Empirical approach. J Grad Med Educ. 2010;2:17–30.
35. Gingerich A, Daniels V, Farrell L, Olsen SR, Kennedy T, Hatala R. Beyond hands-on and hands-off: Supervisory approaches and entrustment on the inpatient ward. Med Educ. 2018;52:1028–1040.
36. Yeates P, O’Neill P, Mann K, Eva K. Seeing the same thing differently. Adv Health Sci Educ. 2013;18:325–341.
37. Tavares W, Young M, Gauthier G, St-Onge C. The effect of foregrounding intended use on observers’ ratings and comments in the assessment of clinical competence. Acad Med. 2020;95:777–785.
38. Labbé M, Young M, Mascarella M, Husein M, Doyle PC, Nguyen LHP. How consistent is competent? Examining variance in psychomotor skills assessment. Acad Med. 2020;95:771–776.
39. Melvin L, Rassos J, Stroud L, Ginsburg S. Tensions in assessment: The realities of entrustment in internal medicine. Acad Med. 2020;95:609–615.
40. Humphrey-Murto S, LeBlanc A, Touchie C, et al. The influence of prior performance information on ratings of current performance and implications for learner handover: A scoping review. Acad Med. 2019;94:1050–1057.
41. Eva KW. On the generality of specificity. Med Educ. 2003;37:587–588.
42. van der Vleuten CPM. When I say … context specificity. Med Educ. 2014;48:234–235.
43. Brydges R, Boyd VA, Tavares W, et al. Assumptions about competency-based medical education and the state of the underlying evidence: A critical narrative review. Acad Med. 2021;96:296–306.
44. O’Dowd E, Lydon S, O’Connor P, Madden C, Byrne D. A systematic review of 7 years of research on entrustable professional activities in graduate medical education, 2011–2018. Med Educ. 2019;53:234–249.
45. Bannister SL, Dolson MS, Lingard L, Keegan DA. Not just trust: Factors influencing learners’ attempts to perform technical skills on real patients. Med Educ. 2018;52:605–619.
46. Sandhu G, Thompson-Burdine J, Nikolian VC, et al. Association of faculty entrustment with resident autonomy in the operating room. JAMA Surg. 2018;153:518–524.

Supplemental Digital Content

Copyright © 2021 by the Association of American Medical Colleges