Secondary Logo

Journal Logo

How Entrustment Is Informed by Holistic Judgments Across Time in a Family Medicine Residency Program: An Ethnographic Nonparticipant Observational Study

Sagasser, Margaretha H., PhD; Fluit, Cornelia R.M.G., MD, PhD; van Weel, Chris, MD, PhD; van der Vleuten, Cees P.M., PhD; Kramer, Anneke W.M., MD, PhD

doi: 10.1097/ACM.0000000000001464
Research Reports
Free
SDC

Purpose Entrustment has mainly been conceptualized as delegating discrete professional tasks. Because residents provide most of their patient care independently, not all resident performance is visible to supervisors; the entrustment process involves more than granting discrete tasks. This study explored how supervisors made entrustment decisions based on residents’ performance in a long-term family medicine training program.

Method A qualitative nonparticipant observational study was conducted in 2014–2015 at competency-based family medicine residency programs in the Netherlands. Seven supervisor–resident pairs participated. During two days, one researcher observed first-year residents’ patient encounters, debriefing sessions, and supervisor-resident educational meetings and interviewed them separately afterwards. Data were collected and analyzed using iterative, phenomenological inductive research methodology.

Results The entrustment process developed over three phases. Supervisors based their initial entrustment on prior knowledge about the resident. In the ensuing two weeks, entrustment decisions regarding independent patient care were derived from residents’ observed general competencies necessary for a range of health problems (clinical reasoning, decision making, relating to patients); medical knowledge and skills; and supervisors’ intuition. Supervisors provided supervision during and after encounters. Once residents performed independently, supervisors kept reevaluating their decisions, informed by residents’ overall growth in competencies rather than by adhering to a predefined set of tasks.

Conclusions Supervisors in family medicine residency training took a holistic approach to trust, based on general competencies, knowledge, skills, and intuition. Entrustment started before training and developed over time. Building trust is a mutual process between supervisor and resident, requiring a good working relationship.

M.H. Sagasser was educationalist and researcher, Department of Primary and Community Care, Radboud University Medical Center, Nijmegen, the Netherlands, at the time of the study. The author is now educationalist and researcher, Huisartsopleiding Nederland, Utrecht, the Netherlands.

C.R.M.G. Fluit is head, Center on Research in Learning and Education, Radboud University Medical Center Health Academy, Nijmegen, the Netherlands.

C. van Weel is professor emeritus of general practice, Department of Primary and Community Care, Radboud University Medical Center, Nijmegen, the Netherlands, and honorary professor of primary health care research, Department of Health Services Research and Policy, Australian National University, Canberra, ACT, Australia.

C.P.M. van der Vleuten is professor of education and director, School of Health Professions Education, Faculty of Health, Medicine, and Life Sciences, Maastricht University, the Netherlands.

A.W.M. Kramer is general practitioner, professor of general practice, and head, Family Medicine Residency Program, Department of Public Health and Primary Care, Leiden University Medical Center, Leiden, the Netherlands.

Funding/Support: None reported.

Other disclosure: None reported.

Ethical approval: The ethical review board of the Netherlands Association for Medical Education (Nederlandse Vereniging voor Medisch Onderwijs [NVMO]) approved the study (no. 368). Anonymity was guaranteed, and participation was voluntary.

Previous presentations: Poster presentation, NVMO conference, Rotterdam, the Netherlands, November 12–13, 2015.

Correspondence should be addressed to Margaretha H. Sagasser, Huisartsopleiding Nederland, PO Box 20072, 3502 LB Utrecht, the Netherlands; telephone: +31 (0) 30-2823460; e-mail: g.sagasser@huisartsopleiding.nl.

In residency training, residents mainly practice medicine under the supervision of a supervisor. While residents learn by doing, supervisors have to secure patient safety.1–7 Depending on their trust in residents’ performance, supervisors increasingly allow residents to perform unsupervised tasks, and they adapt their supervision accordingly.6,8,9 Trust is thus an important precondition for learning, as it allows residents to experience increasing levels of autonomous responsibility in the workplace.10,11 This study explored the process of how entrustment develops in the context of competency-based family medicine residency programs.

The growing literature on entrustment illustrates the need to analyze the decision process regarding trust and patient safety. The literature describes five factors that contribute to building trust—namely, the resident, the supervisor, their relationship, the context, and the task.8 It follows that in a given situation these factors influence the trust-building process and may cause variability therein. The process of trust building is thus a complex phenomenon. To make transparent and justified decisions of trust for specific tasks, entrustable professional activities (EPAs) are increasingly mentioned in competency-based medical education (CBME).12–15 EPAs are observable and measurable units of professional practice. The designation of EPAs implies that the resident initially performs these activities under supervision, and the supervisor subsequently entrusts the resident to independently perform the activities after witnessing its successful performance.

To develop trust, supervisors must observe residents perform in practice and estimate their level of competence, while residents, in turn, must give insight into their performance.6,8,9 It is therefore advised that supervisors and residents have sufficient contact time over a prolonged period.16–18 The family medicine residency program is such a long-term training program where resident and supervisor have a one-on-one working relationship. In family practice, residents see patients with a variety of health problems, ranging from singular to complex. Soon after residents start their family medicine residency training, they are allowed to independently manage patient encounters, backed by their supervisors when necessary.19 Prior studies have revealed that residents’ learning from patient encounters is not always visible to their supervisors.20,21 This means that being trusted by their supervisors already is important before the residents have seen the full range of health problems. This concept led us to ask how supervisors come to trust residents with activities they have not performed under their supervision. The aim of our study was to explore the process of entrustment and to understand how supervisors make entrustment decisions based on residents’ performance in the context of a long-term training program where supervisor and resident have a one-on-one working relationship and where entrustment decisions are made without EPAs. Our research question was “How does the process of entrusting residents with independent patient care take place?” We designed a qualitative study in the context of a family practice residency program.

Back to Top | Article Outline

Method

From a constructivist perspective, we designed an ethnographic, nonparticipant observational study. Ethnography is a qualitative research method by which data are collected through observations, interviews, and documents to produce rich, holistic insights and comprehensive accounts of people’s views and actions.22–25 In nonparticipant observation, the researcher observes but does not participate in the activities being observed.22,25,26

This ethnographic nonparticipant observational study focused on two relevant and distinct concepts in workplace learning—namely, entrustment and self-regulation.

Each concept renders its own results, which are best presented in separate articles. In this article, we present the findings regarding entrustment.

Back to Top | Article Outline

Context

We conducted the study during the first year of the three-year competency-based family medicine residency program in the Netherlands. Eight university medical centers offer this program in their departments of family medicine/primary care.19 In the first and third years, each resident provides patient care in family practice under the supervision of a single designated supervisor. The second year comprises clinical rotations in hospitals, nursing homes, and psychiatric outpatient clinics with different supervisors. A few months before the program commences, supervisors receive application letters and curricula vitae from six residents. All supervisors alternately hold one-on-one interviews and establish a ranking. The university decides on the best match. When matched, the supervisor and resident have a get-acquainted interview. The program consists of four days of practice a week and a day release program at the university on the fifth day.27 The practice part ideally exposes residents to singular minor health problems in the beginning, which increase in complexity over time. Residents can consult their supervisors if needed. Daily debriefing sessions serve to discuss residents’ encounters, whereas educational meetings serve to discuss medical themes and residents’ development. Reflection and feedback on experience, assessment, and personal development planning are methods used in supervision.28–30 All supervisors are experienced family physicians. They attend a compulsory long-term training program on educational and coaching skills at the university.

Back to Top | Article Outline

Participants

We invited supervisor–resident pairs to participate. Inclusion criteria were first-year residents who had been in practice for at least two months and supervisors who had supervised at least three residents. We invited supervisors in writing and by telephone. Upon their acceptance, we approached their residents. When they concurred too, we made appointments for observation. Before collecting data, supervisors and residents gave written informed consent.

Back to Top | Article Outline

Design, data collection, and triangulation

We obtained data from observations, interviews, and documents, thereby promoting data triangulation.31,32 We noted the age and gender of all participating residents and supervisors. One researcher (M.S.) visited each practice on two separate days with an interval of one to three days. She observed residents’ patient encounters, daily debriefing sessions, and educational meetings, while encouraging supervisors and residents to act as they normally would. Each practice location had previously informed patients verbally and in writing about her presence. All observed patients had given prior written consent. M.S. took field notes on health problems presented, residents’ and supervisors’ activities and interaction, and the duration of all interactions. These notes were typed out shortly afterwards. The patient debriefing sessions and educational meetings were audiotaped and transcribed verbatim. After the observations, M.S. interviewed supervisors and residents separately. She asked supervisors to reflect on how entrustment was given with respect to the observed encounters and how the observed debriefing sessions and educational meetings contributed to the actual trust, and on how supervisors came to trust residents from the start of the training year. Residents were asked about their perceptions of the responsibilities entrusted in relation to the observed encounters, the development of trust from the beginning of their training year, and their role therein. The interviews were also audiotaped and transcribed verbatim. Supervisors and residents received a gift coupon for their participation. Anonymity was guaranteed, and participation was voluntary. The ethical review board of the Netherlands Association for Medical Education (Nederlandse Vereniging voor Medisch Onderwijs) approved the study (no. 368). Our research team included one educationalist/PhD student (M.S.) and four experienced researchers and educators with differing backgrounds: two family physicians (A.K., C.v.W.), a psychologist (C.v.d.V.), and an educationalist/medical doctor (C.F.).

Back to Top | Article Outline

Data analysis

We performed a qualitative inductive analysis guided by a phenomenological approach.22,33,34 Phenomenological analysis focuses on the lived experience and on the meaning that a person assigns to that phenomenon. The phenomenon under study is the process of building trust. The analysis searches for themes and patterns and allows for interpretation during analysis. We iteratively collected and analyzed the data, thereby adhering dependability.31,32 First, three researchers (C.F., A.K., M.S.) read and reread the field notes and transcripts of the debriefing sessions, educational meetings, and the interviews of three practices; marked and discussed relevant text fragments; identified codes for relevant themes; and developed an initial codebook. Two researchers (C.F., M.S.) used this codebook for the analysis of two more practices and developed a final codebook for the remaining two practices. The researchers critically reflected that their differing backgrounds brought various perspectives to the data, promoting reflexivity and confirmability.31,32,35 M.S. kept a reflective diary. We used Atlas.ti 7.1 qualitative software, version 7.1.5 (Atlas.ti Scientific Software Development GmbH, Berlin, Germany) to organize the data. To promote credibility, we conducted member checks by sending preliminary results to the participants.31,32

Back to Top | Article Outline

Results

We collected data from November 2014 to March 2015. Of 12 eligible pairs, 7 supervisor–resident pairs participated. Six supervisors were male, and all residents were female. Supervisors’ and residents’ mean age was 53 (range 44–64) and 29 years (range 26–40), respectively. M.S. did not know the participants before. M.S. observed 112 patient encounters, varying from 5 to 40 minutes. She observed that residents managed most of the encounters independently, that they occasionally consulted their supervisors, and that most of the encounters were discussed during debriefing sessions. The mean duration of patient debriefings was 26 minutes, and 41 minutes of educational meetings. The average interview time with residents was 45 minutes, and 38 minutes with supervisors. Our analysis of data showed that while supervisors and residents undertook a variety of activities, the process of entrustment could be summarized in three phases (Table 1). A large part of trust was built in the beginning of the residency year, based on general competencies and, sometimes, on specific resident skills. We will first describe the phases and then explain the variations we found.

Table 1

Table 1

Back to Top | Article Outline

Phase 1: Forming expectations about trust

Before the commencement of the training, supervisors developed a first idea regarding entrustment. The letter of application, curriculum vitae, one-on-one interviews, first impression, and acquaintance all informed supervisors of residents’ prior experience as medical doctors and established expectations, exemplified by this supervisor’s reflection:

From the letters I already got the idea that she had sound knowledge and experience. Experience may be more important than knowledge. In the one-on-one interview that idea was confirmed. (Interview supervisor 4)

Any signs indicating general medical competencies (such as the ability to relate to patients, collaborate with other health professionals, take responsibility, and act as a medical professional) contributed to the building of a priori trust. Supervisors also referred to basic medical knowledge and physical examination skills, and sometimes to specific skills, such as surgical skills.

Back to Top | Article Outline

Phase 2: Confirming expectations about trust

Upon commencement of practice, the expectations on trust were confirmed. This already occurred in the first two weeks (or less). One supervisor noted:

And here in practice those first two weeks as well, she won’t mess things up … she has to become a GP … but she knew the doctor’s role … taking position, putting patients at ease, she could do that already. (Interview supervisor 4)

Supervisors assured themselves that residents indeed had sufficient medical knowledge and were able to perform a physical examination, relate to patients, and take responsibility. They, however, acknowledged that residents still had to develop the knowledge and skills specific to the family physician. To see residents perform, supervisors and residents worked closely together in the beginning. All supervisors first let residents observe their patient encounters, and then progressively involved residents by letting them ask questions during the encounter, perform physical examinations (or parts thereof), and participate in clinical reasoning and decision making. They also discussed the encounters. Gradually during these two weeks, residents managed the entire encounter alone while supervisors observed. Supervisors were particularly alert to residents’ reflections and to their potential unawareness of lack of competence. Supervisors needed confirmation that residents recognized alarm signals and acted on these appropriately or, whenever they felt uncertain or they lacked knowledge, that they would consult them. Hence, residents’ questions were essential indicators in the entrustment process; they informed supervisors of residents’ learning needs and how they coped with uncertainty, as illustrated by one supervisor’s observation:

In the beginning residents may call me whenever they want. When they keep calling me after a few months, I start to have doubts, as that’s too much. When they do not ask me anything, I have doubts as well, because how, then, will I hear their learning needs? So too much and too little is not right. (Interview supervisor 3)

Over time, residents’ questions gradually referred to more complex topics. Witnessing residents perform successfully on repeated occasions bolstered supervisors’ trust. When supervisors were confident that residents were able to adequately manage patient encounters, recognized alarm signals, and consulted them when needed, they allowed them to manage encounters independently in their own consultation room, thereby enlarging the physical distance. Meanwhile, they made sure they were easily accessible and provided supervision during or after the encounter. Supervisors kept track of residents’ development through the many questions concerning varying health problems they received from residents. These fragmented perceptions helped form an overall impression of residents’ competence. The debriefing sessions and educational meetings helped them monitor residents’ medical knowledge and reasoning. Finally, their gut feeling guided entrustment decisions. For example, one supervisor remarked:

It’s a cluster of things, if you ask me to specify I could not give an answer to that … of course there’s a lot of intuition, intuition, you do not exactly name it as such. (Interview supervisor 5)

Table 2 presents quotes from supervisors that illustrate the complex interplay of activities and information that contribute to building trust in this phase.

Table 2

Table 2

Residents took their role in the entrustment process by giving insight into their reasoning and decision making and showing supervisors to be careful with patients, especially when performing independently, as one resident’s comments illustrate:

In the beginning I reported on every patient … as in “What did the patient come for, what I did or what I should do,” and very often he would observe me. I can understand, if I were to trust someone with my patients I would also like to know what he’s doing. (Interview resident 8)

Residents felt trusted when supervisors allowed them to see patients independently. Some supervisors specifically expressed their trust in residents. One such comment was:

And I tell them that I trust them. I say, you’re a doctor. I really say that. Even when I notice uncertainty. You’re a doctor…. I really give it to them. (Interview supervisor 6)

Residents counted on their supervisors to be available when needed and to intervene when necessary. Both supervisors and residents experienced the importance of an open and honest working relationship, as demonstrated by one resident’s observation:

Very soon we both noticed “Well, that’s going fine,” and it feels safe to express things to each other … and to consult and discuss. Yes, that helps, that it feels safe. (Interview resident 11)

Back to Top | Article Outline

Phase 3: Monitoring trust

Once residents had been entrusted with independent patient care under appropriate supervision, supervisors continued to monitor performance through residents’ questions during or after encounters, patient debriefing sessions, (video or live) observations, shared out-of-hours duties and home visits, and residents’ reports in the electronic patient records (EPR). While in the beginning they would discuss all patient encounters, after a few months they mostly discussed the topics the residents introduced. Some supervisors incidentally viewed the EPR to check residents’ medical expertise and to guard patient safety, called “sneaky” by some. Although information about residents’ performance upheld entrustment decisions, it was mainly used for residents’ learning. One supervisor noted:

No, I don’t need that for entrustment. For me, live and video observations purely serve to help the residents improve their interviewing skills and [have] nothing to do with entrustment. (Interview supervisor 4)

Supervisors stayed alert to the frequency and content of questions; when residents acted as supervisors would do, their trust was reinvigorated, as illustrated by the field notes and quote from Example 1 (Table 3).

Table 3

Table 3

Supervisors noticed when residents acted unexpectedly and then expressed their surprise with such behavior, as illustrated by the field notes and quote from Example 2 (Table 3). When residents acted differently, supervisors would not necessarily deny them certain responsibilities as supervisors found it important that residents discovered their own solutions provided that it did not harm patients. Patients’ reactions to residents also influenced trust, as this supervisor’s observation indicates:

It’s a small village, I hear a lot from patients like “adequate, friendly, or not.”… You also notice if patients return to the resident instead of going to a colleague. (Interview supervisor 3)

Supervisors stressed the impossibility of tracking all residents’ actions and that a lack of competence could suddenly become apparent. One such illustrative comment was:

On a home visit I said, “Well, write the prescription,” … to which the resident replied, “What should I write?… Then I realized he didn’t know his medication that well. He had gratefully used the list of preferred drug choices in our computer system when prescribing medication. (Interview supervisor 1 about a previous resident).

Our observations stemmed from this third phase: Residents had been entrusted to independently manage encounters; they sometimes consulted their supervisors who were immediately available.

Back to Top | Article Outline

Variations in the development of trust

Supervisor, resident, and context accounted for variations in building trust. Some supervisors were more quick to entrust residents with independent patient care than others (phase 2). They either found it important that residents performed on their own or they found themselves to be very trusting. Entrustment decisions were also influenced by experiences with previous residents, as supervisors recognized certain behavior and relied on their previous decisions. Negative experiences would make them observe more frequently, although this was only occasionally reported. When residents were performing independently (phase 3), most supervisors allowed them to choose the patients to be discussed, while others also selected patients themselves. For instance, one supervisor noted:

When I see the names [in the EPR], I think, “Hey, that’s interesting that he came for a visit” or “Why did he come?” or “Hey, he came to me last week” or that I know “that’s always difficult” or whatever. Then I want to know, “Why did the patient come?” (Interview supervisor 2)

If residents had already acquired clinical skills in a certain field, supervisors were more prone to trust them in that domain, as this field note and quote from one practice illustrate:

During the debriefing session the resident receives a call for an emergency home visit.

Resident: Chest pain and radiating pain into the left arm. We [resident and supervisor] will go to [name patient].

Supervisor: Well, as for me you can do this by yourself. (Field note debriefing session, practice K)

She [resident] got this emergency call, and I said, you will succeed regarding your experience, although I have never made a cardiac emergency visit with her. But I think yes, she will succeed…. I can trust her in that. (Interview supervisor, practice K)

At the same time, both supervisors and residents were cautious about prior experience from other contexts, as one set of comments illustrate:

Hospital experience does not mean a lot to me, or that someone is a good family physician. We much more use probabilities … we do not automatically request all lab tests…. So there’s a difference…. Prior experience does not always indicate the knowledge and skills of a resident. (Interview supervisor 1)

In the emergency department I was quick-quick-quick having 10 people in the waiting room … you think in pathology. In the beginning [of the family medicine residency training] I saw pathology everywhere. All patients in the hospital were ill, but here they are not, the majority is not ill, they suffer from something, but they’re not ill. (Interview resident 11)

Finally, the context of the workplace influenced the entrustment process. Practices differed in the distribution of health problems in their practice populations, which had an impact on the priorities residents had to set in their approaches. Being able to respond to such demands contributed to entrustment.

Back to Top | Article Outline

Discussion

The supervisors in family medicine residency training in our study took a holistic approach in entrustment. They based their entrustment decisions on general competencies necessary for handling a variety of health problems (such as clinical reasoning, problem solving, or the ability to relate to patients and other health workers); on medical knowledge and specific professional skills (such as heart and lung examination); and on their intuition. They weighed these against residents’ development, and expressed an overall entrustment. This entrustment granted residents the independence to provide patient care under appropriate supervision while patient safety remains secured. We discuss the contribution of holistic and more specific judgments of trust, the speed in which trust is given, and the relationship between supervisor and resident.

Our findings regarding the contribution of holistic judgments and judgments of general competencies to entrustment find resonance in the literature.36,37 The roles of both holistic judgments and judgments derived by more standardized instruments, such as EPAs, align with developments in the field of assessment, where standardized measures are important for valid and reliable judgments, but where expert judgment is indispensable for combining and weighing information.38 As trust is a rather complex phenomenon, EPAs might be a valuable tool for making decisions of trust. Although EPAs increasingly are developed in CBME, expressing trust entails more than using a list of EPAs because of the complex, ambiguous, and uncertain character of medicine.39 Also, using EPAs may interfere with holistic judgments or distract from authentic learning and supervising processes in a continuity relationship. The contributions of both holistic judgments and EPAs to developing trust need further exploration.40–42

In our study, supervisors entrusted first-year residents to manage independent patient encounters within the first two weeks of the residency year. A similar speed was found in another context.36 Apparently, entrustment does not require previous judgment of all professional activities, although we know that higher-risk tasks demand higher levels of trust.6,8,9 In family medicine residency training, a desired patient mix, from singular to complex problems, cannot always be achieved,43 and acute situations may happen any time. Therefore, supervisors in our study made sure early in training that residents could manage high-risk situations or would consult them. Hence, it seems that some professional activities must be satisfactorily judged before certain responsibilities can be entrusted.

Trust is built through complex interrelated processes that supervisors must orchestrate, such as observing performance, providing space for development, assessing development, and giving feedback, all tailored to residents’ needs and patient safety. According to the participants in our study, this entrustment process requires an open and honest working relationship, a finding consistent with the literature.6,8,9 In our study, the supervisors trusted residents, and residents sensed that. They, too, have to trust their supervisors to make appropriate entrustment decisions. The one-on-one interview during the matching procedure, typical for family medicine residency training, seems to be key to this process, as it affords supervisors to appoint residents with whom they expect to build a good working relationship. Sufficient time for sharing expectations and insights into professional growth is beneficial for developing such a relationship.16–18 Long-term training satisfies this criterion, allowing for mutual trust to be built,43 for legitimate increases in responsibilities,44,45 and for meaningful feedback and assessment.18,45,46 In other contexts, a lack of contact time appeared to be a barrier to the development of trust.36

In interpreting the outcomes, we should consider that this study concerned the first months of a long-term residency program and a one-on-one working situation. In unraveling the process of building trust, we identified that trust develops from forming expectations about trust, through confirming expectations about trust, towards monitoring trust. This knowledge might be helpful in the training of supervisors. Further research might refine this process and identify more phases, or steps within phases, or explore negative experiences with trust. Research on trust in other contexts, such as settings that work with multiple supervisors or EPAs, for instance, might render other insights in the development of trust.

A strength of our study is the ethnographic approach of using observations and interviews to study activities and interactions in family practice. Another strength is the inclusion of pairs of residents and supervisors. Because their participation was voluntary, they might have been more interested in the study’s topics than others. In terms of limitations, we failed to include more female supervisors and male residents. Although this male supervisor–female resident distribution largely reflects reality, we should be aware that gender potentially influences participants’ behavior in learning and supervision. Furthermore, the supervisors in our study did not report negative experiences in building trust with their current residents, while some could report some negative experiences with prior residents. Finally, the researcher’s presence may have influenced the results as participants may have acted differently than they otherwise would have.

Back to Top | Article Outline

Conclusions

The entrustment process starts before training and develops over time. Supervisors take a holistic approach to trust, and base their trust on general competencies, knowledge, skills, and intuition. Trust is heavily resident-, supervisor-, and context-specific. Future research might investigate in more detail how a holistic, intuitive approach to entrustment relates to entrustment based on specific professional activities, in different contexts, and during the first days of training. Knowledge about trust development is valuable for supervisors and residents as it offers a language for exchanging experience that may support resident development, supervision, and assessment. Knowledge about trust development adds to the debate about EPAs in CBME.

Acknowledgments: The authors wish to thank the residents, supervisors, and family practices that participated; the patients who accepted the researcher’s presence; Simone Vermeulen and Eefje Linnenbank for transcribing; and Angelique van den Heuvel for editing the manuscript.

Back to Top | Article Outline

References

1. Sibbald M, de Bruin AB, van Merrienboer JJTwelve tips on engaging learners in checking health care decisions. Med Teach. 2014;36:111–115.
2. Zwart DL, Heddema WS, Vermeulen MI, van Rensen EL, Verheij TJ, Kalkman CJLessons learnt from incidents reported by postgraduate trainees in Dutch general practice. A prospective cohort study. BMJ Qual Saf. 2011;20:857–862.
3. Boendermaker PM, Schuling J, Meyboom-de Jong BM, Zwierstra RP, Metz JCWhat are the characteristics of the competent general practitioner trainer? Fam Pract. 2000;17:547–553.
4. Kilminster S, Cottrell D, Grant J, Jolly BAMEE guide no. 27: Effective educational and clinical supervision. Med Teach. 2007;29:2–19.
5. Wearne S, Dornan T, Teunissen PW, Skinner TGeneral practitioners as supervisors in postgraduate clinical education: An integrative review. Med Educ. 2012;46:1161–1173.
6. Kennedy TJ, Lingard L, Baker GR, Kitchen L, Regehr GClinical oversight: Conceptualizing the relationship between supervision and safety. J Gen Intern Med. 2007;22:1080–1085.
7. Babbott SCommentary: Watching closely at a distance: Key tensions in supervising resident physicians. Acad Med. 2010;85:1399–1400.
8. Hauer KE, Ten Cate O, Boscardin C, Irby DM, Iobst W, O’Sullivan PSUnderstanding trust as an essential element of trainee supervision and learning in the workplace. Adv Health Sci Educ Theory Pract. 2014;19:435–456.
9. Sterkenburg A, Barach P, Kalkman C, Gielen M, ten Cate OWhen do supervising physicians decide to entrust residents with unsupervised tasks? Acad Med. 2010;85:1408–1417.
10. Lave J, Wenger ESituated Learning: Legitimate Peripheral Participation. 1991.Cambridge, UK: Cambridge University Press;
11. van der Zwet J, Zwietering PJ, Teunissen PW, van der Vleuten CP, Scherpbier AJWorkplace learning from a socio-cultural perspective: Creating developmental space during the general practice clerkship. Adv Health Sci Educ Theory Pract. 2011;16:359–373.
12. ten Cate OAM Last Page: What entrustable professional activities add to a competency-based curriculum. Acad Med. 2014;89:691.
13. ten Cate O, Chen HC, Hoff RG, Peters H, Bok H, van der Schaaf MCurriculum development for the workplace using entrustable professional activities (EPAs): AMEE guide no. 99. Med Teach. 2015;37:983–1002.
14. ten Cate O, Scheele FCompetency-based postgraduate training: Can we bridge the gap between theory and clinical practice? Acad Med. 2007;82:542–547.
15. ten Cate OTrust, competence, and the supervisor’s role in postgraduate training. BMJ. 2006;333:748–751.
16. Bernabeo EC, Holtman MC, Ginsburg S, Rosenbaum JR, Holmboe ESLost in transition: The experience and impact of frequent changes in the inpatient learning environment. Acad Med. 2011;86:591–598.
17. Hirsh DA, Ogur B, Thibault GE, Cox M“Continuity” as an organizing principle for clinical education reform. N Engl J Med. 2007;356:858–866.
18. Watling CJ, Kenyon CF, Schulz V, Goldszmidt MA, Zibrowski E, Lingard LAn exploration of faculty perspectives on the in-training evaluation of residents. Acad Med. 2010;85:1157–1162.
19. Van Berkestijn LGMLearning to learn. Curricular changes in the postgraduate training for general practitioners [in Dutch]. Huisarts Wet. 2002;45:248–252.
20. Sagasser MH, Kramer AW, van Weel C, van der Vleuten CPGP supervisors’ experience in supporting self-regulated learning: A balancing act. Adv Health Sci Educ Theory Pract. 2015;20:727–744.
21. Sagasser MH, Kramer AW, van der Vleuten CPHow do postgraduate GP trainees regulate their learning and what helps and hinders them? A qualitative study. BMC Med Educ. 2012;12:67.
22. Angrosino MDoing Ethnographic and Observational Research. 2007.Los Angeles, CA: SAGE Publications;
23. Atkinson P, Pugsley LMaking sense of ethnography and medical education. Med Educ. 2005;39:228–234.
24. Reeves S, Kuper A, Hodges BDQualitative research methodologies: Ethnography. BMJ. 2008;337:a1020.
25. Reeves S, Peller J, Goldman J, Kitto SEthnography in qualitative educational research: AMEE guide no. 80. Med Teach. 2013;35:e1365–e1379.
26. Liu F, Maitlis SMills AJ, Durepos G, Wiebe ENonparticipant observation. In: Encyclopedia of Case Study Research. 2010:Thousend Oaks, CA: SAGE Publications; 610–612.
27. van Roermund TC, Tromp F, Scherpbier AJ, Bottema BJ, Bueving HJTeachers’ ideas versus experts’ descriptions of “the good teacher” in postgraduate medical education: Implications for implementation. A qualitative study. BMC Med Educ. 2011;11:42.
28. Pelgrim EA, Kramer AW, Mokkink HG, van der Vleuten CPReflection as a component of formative assessment appears to be instrumental in promoting the use of feedback; an observational study. Med Teach. 2013;35:772–778.
29. Ram P, van der Vleuten C, Rethans JJ, Schouten B, Hobma S, Grol RAssessment in general practice: The predictive value of written-knowledge tests and a multiple-station examination for actual medical performance in daily practice. Med Educ. 1999;33:197–203.
30. Tromp F, Vernooij-Dassen M, Grol R, Kramer A, Bottema BAssessment of CanMEDS roles in postgraduate training: The validation of the Compass. Patient Educ Couns. 2012;89:199–204.
31. Mays N, Pope CQualitative research in health care. Assessing quality in qualitative research. BMJ. 2000;320:50–52.
32. Tavakol M, Sandars JQuantitative and qualitative methods in medical education research: AMEE guide no 90: Part I. Med Teach. 2014;36:746–756.
33. Creswell JWQualitative Inquiry and Research Design: Choosing Among Five Traditions. 1998.Thousand Oaks, CA: SAGE Publications;
34. Giorgi BAshworth PD, Chung MCCan an empirical psychology be drawn from Husserl’s phenomenology? In: Phenomenology and Psychological Science, Historical and Philosophical Perspectives. 2006:New York, NY: Springer Science+Business Media, LLC; 69–88.
35. Barry CA, Britten N, Barber N, Bradley C, Stevenson FUsing reflexivity to optimize teamwork in qualitative research. Qual Health Res. 1999;9:26–44.
36. Hauer KE, Oza SK, Kogan JR, et alHow clinical supervisors develop trust in their trainees: A qualitative study. Med Educ. 2015;49:783–795.
37. Wijnen-Meijer M, van der Schaaf M, Nillesen K, Harendza S, ten Cate OEssential facets of competence that enable trust in medical graduates: A ranking study among physician educators in two countries. Perspect Med Educ. 2013;2:290–297.
38. Ginsburg S, McIlroy J, Oulanova O, Eva K, Regehr GToward authentic clinical evaluation: Pitfalls in the pursuit of competency. Acad Med. 2010;85:780–786.
39. Kramer AWToo much control diverts from the essence of learning and teaching. Perspect Med Educ. 2015;4:272–274.
40. ten Cate O, Snell L, Carraccio CMedical competence: The interplay between individual ability and the health care environment. Med Teach. 2010;32:669–675.
41. Sklar DPCompetencies, milestones, and entrustable professional activities: What they are, what they could be. Acad Med. 2015;90:395–397.
42. Frank JR, Snell LS, ten Cate O, et alCompetency-based medical education: Theory to practice. Med Teach. 2010;32:638–645.
43. de Jong J, Visser MR, Mohrs J, Wieringa-de Waard MOpening the black box: The patient mix of GP trainees. Br J Gen Pract. 2011;61:e650–e657.
44. Hirsh DA, Holmboe ES, ten Cate OTime to trust: Longitudinal integrated clerkships and entrustable professional activities. Acad Med. 2014;89:201–204.
45. Bates J, Konkin J, Suddards C, Dobson S, Pratt DStudent perceptions of assessment and feedback in longitudinal integrated clerkships. Med Educ. 2013;47:362–374.
46. Dijksterhuis MG, Schuwirth LW, Braat DD, Teunissen PW, Scheele FA qualitative study on trainees’ and supervisors’ perceptions of assessment for learning in postgraduate medical education. Med Teach. 2013;35:e1396–e1402.
© 2017 by the Association of American Medical Colleges