Secondary Logo

Journal Logo

Exploring Integration in Action: Competencies as Building Blocks of Expertise

Mylopoulos, Maria PhD; Borschel, Debaroti (Tina) MD; O’Brien, Tara MD; Martimianakis, Sofia MA; Woods, Nicole N. PhD

doi: 10.1097/ACM.0000000000001772
Research Reports
Free
SDC

Purpose Competency frameworks such as the CanMEDS roles and the ACGME core competencies may lead to the implicit assumption that physicians can learn and practice individual competencies in isolation. In contrast, models of adaptive expertise suggest that the integration of competencies reflects the capabilities of an expert physician. Thus, educational programming aimed at teaching discrete roles or competencies might overlook expert physician capabilities that are central to patient care. To develop expertise, learning opportunities must reflect expert capabilities. To better understand the relationship between competency-based medical education and expert development, the authors sought to explore how integrated competencies are enacted during patient care by postgraduate medical trainees.

Method Using a cognitive ethnographic approach, in 2014–2015 the authors conducted observations and—to refine and elaborate these observations—ad hoc informal interviews with 13 postgraduate trainee participants. Data collection resulted in 92 hours of observation, 26 patient case portraits, and a total of 220 pages of field notes for analysis. Through analysis, the authors identified and examined moments when postgraduate trainees appeared to be simultaneously enacting multiple competencies.

Results The authors identified two key expert capabilities in moments of integrated competence: finding complexity and being patient-centered. They described two mechanisms for these forms of integration: valuing the patient’s narrative of their illness, and integrated understanding.

Conclusions Understanding integrated competencies as the building blocks of expert capabilities, along with recognizing the importance of mechanisms that support integration, offers an opportunity to use existing competency-based frameworks to understand and teach adaptive expertise.

M. Mylopoulos is assistant professor and scientist, Wilson Centre and Department of Paediatrics, University of Toronto, Toronto, Ontario, Canada.

D.(T.) Borschel is associate professor, Department of Medicine, University of Toronto, Toronto, Ontario, Canada.

T. O’Brien is assistant professor, Department of Medicine, University of Toronto, Toronto, Ontario, Canada.

S. Martimianakis is research assistant, Wilson Centre, University of Toronto, Toronto, Ontario, Canada.

N.N. Woods is assistant professor and scientist, Wilson Centre and Department of Family and Community Medicine, University of Toronto, Toronto, Ontario, Canada.

Funding/Support: This study was funded by the Royal College of Physicians and Surgeons of Canada, Royal College/AMS CanMEDS Research and Development Grant, 13/AMS-01. This work is supported by the Medical Psychiatry Alliance, a collaborative health partnership of the Centre for Addiction and Mental Health, the Hospital for Sick Children, Trillium Health Partners, and the University of Toronto, as well as the Ontario Ministry of Health and Long-Term Care and an anonymous donor.

Other disclosures: None reported.

Ethical approval: This study was approved by the Research Ethics Board at Women’s College Hospital in Toronto, Ontario, Canada, on November 11, 2013.

Correspondence should be sent to Maria Mylopoulos, Wilson Centre, 200 Elizabeth St., ES 1-565, Toronto, ON M5G 2C4; telephone: (416) 340-3615; e-mail: maria.mylopoulos@utoronto.ca; Twitter: @mmylopoulos.

Current models of expertise in the health professions have moved beyond traditional definitions that primarily consider the breadth and depth of biomedical knowledge. In medicine specifically, researchers and educators have worked to broaden conceptions of expertise to include a diverse set of facets that contribute to the competence of the physician in practice.1,2 CanMEDS, the Accreditation Council for Graduate Medical Education (ACGME), and other competency frameworks offer a means to operationalize this more inclusive understanding of expertise.3,4 The extensive uptake of these frameworks is in part due to a need to create objective outcomes for student assessment and licensing. Used as intended, competency frameworks provide a language for discussing facets of expertise, facilitate the identification of specific desirable behaviors, and enable the alignment of program objectives and student assessment.5

However, defining distinct competency domains for the purposes of teaching and assessment may inadvertently lead to an implicit assumption that individual competencies can be learned and practiced separately. Since the development of the ACGME and CanMEDS frameworks, numerous efforts have focused on developing instructional and teaching tools designed to develop specific competencies.6 Despite the proliferation of attempts, a systematic review of the literature on evaluating the ACGME competencies failed to identify an assessment tool that successfully measured performance of any one competency independent of the others.7 These findings support the interconnectivity of competencies and highlight the challenges involved in moving from labeling desirable competencies to assessing the performance of competencies in practice.6

In particular, by deconstructing the concept of the expert into separate components, competency frameworks run the risk of identifying the building blocks of expertise at the expense of emphasizing higher-order capabilities that incorporate the complexity of the clinical context.8,9 For example, models of adaptive expertise understand expert activity as the flexible use of knowledge in context,10,11 and studies of clinical reasoning in experts and novices have illustrated the value of integrating different forms of knowledge in the diagnosis and management of difficult cases.12–14 Moreover, renowned physicians view their own adaptive expertise as integrated competence that is deliberately cultivated and maintained.11 Taken together, these works suggest that it is the integration of knowledge and competencies within the context of clinical practice that defines the physician as expert. This emerging conception of expertise, which reflects the needs of the current health care climate,15 requires that competencies are learned, practiced, and assessed in an integrative manner rather than as separate skills.16

While existing competency frameworks have laid the groundwork for comprehensive assessment of competence by identifying a set of common domains, an authentic representation of expertise should reflect the interconnectivity of competencies and the influence of context on physician performance.17 Some medical educators have proposed entrustable professional activities (EPAs) as a potential solution because they provide detailed descriptions of observable and measurable clinical activities representing multiple competencies.18 The purpose of EPAs is, however, to provide clinical educators with a systematic approach to setting expectations and assessing trainee proficiency in specific units of clinical work (e.g., gathering a history or documenting a clinical encounter in the patient record).18

In contrast, our aim in this study was to investigate the integration of competencies linked specifically to the development of higher-order capabilities of expertise.9 We aimed to empirically explore the integration of competencies by observing and analyzing moments of activity during which postgraduate trainees can be described as simultaneously performing multiple physician competencies. In doing so, we sought to understand how trainees enact integration in practice and what they may learn from these experiences so that the integration of competencies can be better accounted for in models of expert development.

Back to Top | Article Outline

Method

We conducted an observational study exploring the integration of competencies. Our observations occurred while postgraduate physician trainees were providing patient care. We used a cognitive ethnographic approach, which allowed us to describe and analyze specific episodes of activity by integrating data across participant observations, ad hoc interviews, and the exploration of cognitive artifacts (physical objects that enhance or impede cognition, such as patient charts, health care technologies, and equipment).19,20 Cognitive ethnography focuses on the moment-to-moment development of activity and on the relationship between activity and broader discourses.21 As such, it was the ideal methodology to explore the provision of patient care by physician trainees as an activity situated within competency frameworks in medical education.

The site selected for the research is a complex care clinic (CCC) at Women’s College Hospital, a large urban Canadian ambulatory care center. Because the CCC serves patients with chronic and complex medical conditions, it provides a powerful opportunity to explore the phenomenon of interest. The CCC is an interprofessional teaching clinic that aims to provide excellent patient care as well as training in the best practices in ambulatory care for postgraduate medical trainees. The team comprises physicians, nurses, pharmacists, social workers, physiotherapists, occupational therapists, dieticians, and community care workers. The physician trainees at the CCC are postgraduate year three (PGY3) internal medicine residents from the affiliated university. They complete a one-year longitudinal weekly clinic rotation from July through the following June. Based on preliminary pilot work and existing relationships, the director (D.[T.]B.) of the CCC and a staff physician (T.O’B.) were invited to act as key informants for the duration of the project.

After obtaining Research Ethics Board approval from Women’s College Hospital, a research assistant with extensive experience in conducting observations and interviews (S.M.) conducted 92 hours of observation—and, to refine and elaborate the observations, ad hoc informal interviews (ranging from 5 to 15 minutes)—with a total of 13 PGY3 participants. Data collection took place from March 2014 to March 2015. After consulting with the key informants (D.[T.]B. and T.O’B.), observation times were carefully selected to optimize the attainment of rich data to best fulfill the objectives of the research. Observations of the PGY3 participants took place during triage and rounds with interprofessional staff; as participants prepared for patients (e.g., while they reviewed electronic patient records); during patient care, including both history taking and physical examinations; and after interactions with patients when residents presented the patient case to their attending physicians. The research assistant adopted a participant-as-observer role22 and took field notes capturing the entire narrative of each patient encounter. These observations resulted in 26 patient case portraits (CPs) and a total of 220 pages of field notes for analysis. Next, the research assistant anonymized the field notes, including assigning pseudonyms to patients and participants so as to protect their identities, before sharing these notes with the rest of our research team. The research assistant checked any ambiguous clinical data and, if necessary, asked the two key informants to clarify this information.

Data collection and analysis were conducted iteratively until new data provided no additional insights related to the integration of competencies. After each round of observations, the field notes were first coded independently by each member of our research team: the research assistant (S.M.), two education scientists (M.M. and N.N.W.), and the key informants (D.[T.]B. and T.O’B.). Because we sought to interpret observed patient care activities enacted within a dominant competency framework, this initial coding was deliberately informed by the seven CanMEDS 2005 roles. Next, we met and collectively analyzed the field notes, using NVivo qualitative data analysis software (Version 11, QSR International Pty Ltd., Victoria, Australia) to produce a summary of every instance for which data were coded as relevant to more than one role (e.g., medical expert or communicator). That is, after individually and then collectively coding the field notes using the CanMEDS 2005 framework, we queried the database for “role” and “at least one other role” (e.g., communicator and at least one other role). We ran this query for each role until every instance of multiple roles was documented.

Then, each of us individually conducted a further analysis of the full list of actions involving multiple roles. Next, our full team met again and collectively developed codes that represented these examples of integrated competence. As data collection and analysis progressed, these codes were eventually grouped into our final thematic structure representing “capabilities” and “mechanisms.” For example, the codes managing patient expectations and adaptively negotiating were grouped under the eventual capability of being patient-centered, and the codes using basic science knowledge and using knowledge from other professions were grouped under the mechanism integrated understanding. This process of careful, iterative analysis allowed us to understand how integration of competencies is enacted at the point of care.

Team meetings to analyze recent data and develop the coding framework continued for the duration of data collection and analysis, resulting in seven meetings over the course of 12 months. This process ensured, first, that all new data were consistently compared with existing data, resulting in an iterative and comprehensive analysis, and, second, that the analysis was continually informed by our entire research team. An audit trail was maintained throughout data collection and analysis, including each team member’s independent coding, notes from team meetings, dated versions of the NVivo database, and Excel files detailing the observation demographics, including participant pseudonyms, conditions being treated, and length of each observation.

Back to Top | Article Outline

Results

Our goal was not to provide a taxonomy describing which competencies were most likely to be integrated but, rather, to understand what can occur in moments of integration. Thus, we present our findings as two notable aspects of the enactment of integrated competencies that were made visible in situations drawn from across the CPs. Through observation and analysis of the CPs, we identified the simultaneous performance of two or more CanMEDS roles by our participants. First, we noted the ways in which integration of competencies could be understood as two expert capabilities that we defined as “finding complexity” and “being patient-centered.” Second, our analysis made visible two mechanisms that supported the integration of competencies in our data: “valuing the patient’s narrative of their illness” and “integrated understanding.”

Back to Top | Article Outline

Integration as capability: Finding complexity

In our analysis of integrated competencies, we identified finding complexity as a capability requiring that clinical reasoning be deliberately oriented toward the unexpected, which would allow or lead to the recognition of complexity. For example, in one instance of finding complexity (CP 11), the PGY3 participant (initially coded as fulfilling the CanMEDS roles of medical expert and communicator) responded to the unexpected patient perspective by adjusting the diagnosis and treatment plan:

The patient had previously tested positive for H. pylori and has recently finished a round of antibiotics. Before Dr. M meets the patient, she comments that “this referral was most likely made for reassurance, the patient has been in and out of the ER and is probably anxious but all of her test results are normal.” In the examination room, the patient explains to Dr. M that since her referral, she has been improving but she would characterize her past symptoms as “extreme.” Outside the examination room, Dr. M explains that this case was very different from what she was expecting because “Her test results are all good, but she’s still having symptoms.” Dr. M navigated the conflict between the test results and the patient’s own description of her experience of her illness by proposing a treatment plan that foregrounded the patient’s unexpected perspective. The patient noted “that in the past, doctors have just sent her away with ‘you probably just have the flu.’”

By remaining receptive to new information that surfaced during the encounter, in this case the patient’s perspective, Dr. M found complexity in a case that initially seemed straightforward.

In another example of finding complexity, the PGY3 physician proactively attended to the potential for complexity in the future. In this case, the patient was a young man struggling with fatigue (CP 3), and we initially coded the CanMEDS roles enacted by the caregiver as communicator, advocate, and manager:

The patient’s tests were inconclusive and he was started on iron. Dr. G comments, “I don’t have any good explanation for why he is so tired.” While a typical treatment plan would include a six-month follow-up, given the dramatically altered lifestyle of the patient who “used to be a skater and a hockey player and now finds himself lethargic on a daily basis,” Dr. G “advocates to get him a quicker follow-up” and proposes that signs of psychiatric distress should be monitored. The attending physician agrees “okay then maybe seeing him in six to eight weeks is better.”

Dr. G’s foregrounding of the unexpected potential for psychiatric multimorbidity, which emerged from the encounter through the lack of response to treatment and conversation with the patient, changed his reasoning and sense of acuity around future treatment.

Back to Top | Article Outline

Integration as capability: Being patient-centered

In the moments we identified as caregivers being patient-centered, we noted that trainees were integrating or adaptively shifting from directly applying “medical expert” knowledge to including patient needs, experiences, and expectations in their reasoning. In the following example of being patient-centered (CP 4), we initially analyzed the CanMEDs roles enacted by the PGY3 physician as medical expert, advocate, communicator, and collaborator:

Dr. L is giving the patient a diagnosis of Graves disease. Due to the medication prescribed and the nature of Graves disease, the patient will need to cut down on his alcohol use and smoking. Dr. L goes through the benefits of quitting smoking with the patient, focusing on long-term disease prevention, the risk of smoking reducing his life span, and the financial strain smoking can cause. Dr. L sympathizes with it being difficult for a young man to not drink when he’s at parties with friends. Dr. L reassures the patient that he will check with the pharmacist whether or not a small amount of alcohol would be allowed with the new medication. Dr. L leaves the room to ask the attending physician and pharmacist about the amount of alcohol that would be acceptable to suggest. The attending suggests 3 drinks per week, and the staff pharmacist agrees. Dr. L comments, “I was worried the patient would refuse the medication if he thought he could not drink at all.”

This example highlights the potential transformative effect that valuing patient needs can have on the management of illness. In particular, Dr. L determined that the patient was unlikely to comply with the optimal treatment. By working with the patient, pharmacist, and attending physician, Dr. L sought to find an acceptable solution that did not compromise care.

In another exemplar case of being patient-centered, Dr. Y adjusts the management plan of a patient (CP 21) to address the concerns and past experiences of the patient. We initially analyzed the roles Dr. Y played as medical expert, manager, and communicator:

The patient is an elderly man who has been followed by the clinic for four years for pleural effusion and atrial fibrillation. He is at the clinic with his daughter and being seen for the first time by Dr. Y. Because of a past adverse reaction to a medication, he was switched to a new prescription. Now Dr. Y wants him to switch back due to the patient’s chronic kidney disease. The first-line option is the original medication the patient was on because it’s safer, does not build up in the system, and can be reversed. His daughter acknowledges that adverse reaction was a result of a miscommunication with the family physician, so she would be more comfortable if the clinic manages the necessary monitoring of the medication. Dr. Y asks, “So am I understanding correctly that you are willing to try the medication again as long as you are monitored by us?” the patient says, “Yes,” and Dr. Y makes the necessary arrangements.

In this case, while Dr. Y determined the medication, he also made sure that the care provided was responsive to the past experiences of the patient and his family.

Our analysis showed that moments of integrated competence often reflected expert capabilities. Critically, this integration of competencies seemed to be supported by two key mechanisms: valuing the patient narrative and integrated understanding.

Back to Top | Article Outline

Mechanisms for integration: Valuing the patient’s narrative of their illness

The intentional valuing of the patient’s narrative of their illness to inform clinical reasoning often acted as a mechanism for integration. In CP 11, Dr. M truly heard and appreciated the patient’s own experience of her illness, which allowed Dr. M to find and consider the complexity of the case. The test results alone would have suggested that the patient should be discharged from clinic. Instead, Dr. M’s plan involved referring the patient for an endoscopy to check whether or not the H. pylori had been eradicated and testing her for Celiac disease. The patient’s comment that in the past none of her doctors have taken her dietary complaints seriously highlight Dr. M’s integration and valuing of the patient’s experience.

Similarly, in CP 4, Dr. L, enacting being patient-centered, makes an extra effort to find a treatment plan that works for a young man suffering from Graves disease, prioritizing the need for a minimal negative effect on the patient’s experience of his quality of life. In CP 3, Dr. G hears and incorporates the patient’s particular circumstances, particularly his fatigue. This ability to value the patient’s experience leads Dr. G to find complexity by identifying a possible comorbidity and adjusting treatment accordingly.

In other cases, PGY3 participants were careful to value how patients experienced the health care system and acted accordingly. For example, the resistance of the patient and his daughter to switching back to a previous medication (CP 21) was directly related to the previous complication which they believed was caused by ineffective monitoring and miscommunication by their family physician. Dr. Y heard the patient’s concerns as relayed by his daughter: “What he doesn’t like is this whole system of ‘if there is an issue we can fix it.’ He doesn’t want there to be an issue to begin with.” Recognizing the effect of the earlier, negative experience, Dr. Y reassures the patient and his daughter, “We will be taking responsibility for the [monthly] monitoring of your medication so things do not fall through the cracks. We will be communicating directly with your pharmacy on your behalf and you will be informed of any changes.”

Back to Top | Article Outline

Mechanisms for integration: Integrated understanding

Our analysis suggested that being patient-centered and finding complexity rely on an integrated understanding of both clinical and basic science knowledge, including skills for effective patient interactions and systems knowledge. Participants deliberately sought to develop a full understanding of the patient from the patient’s and family members’ narratives, records of past visits, test results, team members’ expertise, and available technology. For example, when presenting the case of CP 11 to the attending physician and pharmacist, Dr. M “gives a thorough history of the patient beginning with the reason for referral, the patient’s most recent concerns, and the previous positive H. pylori result as well as her physical exam.” The PGY3 participant asks the pharmacist if treating the patient with another round of antibiotics would be helpful to ensure that the H. pylori is completely eradicated. The pharmacist says the eradication rate, given the patient’s regimen, is 80%, and Dr. M asks the pharmacist to go over the options available for a second round of antibiotics. Dr. M also quickly does some research on Celiac disease. Ultimately, her integrated understanding of the patient’s situation, including the latter’s reluctance to take more antibiotics, leads her to decide to do the endoscopy first.

Likewise, Dr. Y has an integrated understanding of CP 21 and applies that understanding when considering the merits of attempting to convince the patient to go back on his previous medication. Dr. Y has learned from the pharmacist “that when the patient had a reaction to the previous medication it was because he took triple the dose while on vacation, but even after that, it was still the number one recommendation when taken correctly.” Later, when convincing the patient to give the medication a second chance, Dr. Y explains that the patient’s kidney function is in the range where “we would not prescribe your current medication at all.” Dr. Y hears and appreciates the patient’s concern that a miscommunication can occur “where there is a wrong reading or the information isn’t passed along properly to our family doctor and then to us.” Dr. Y suggests, therefore, as a solution that the monitoring be done by the clinic. Another reason Dr. Y considers home monitoring is because he “completely understands the inconvenience” that continually contacting the pharmacy/family doctor poses on the patient and his daughter. The patient does agree to switch, and Dr. Y, further integrating his understanding of the patient experience into the care he provides, consults with the pharmacist about how long it will take for the current medication to be completely out of the patient’s system. Based on the pharmacist’s recommendation, Dr. Y suggests that the patient can stop his current medication immediately and start the new medication the next day.

Back to Top | Article Outline

Discussion

Our exploration of integrated competencies has illuminated the ways in which integration occurs when physicians enact important capabilities. We observed the enactment of integrated competencies—that is, episodes of patient care during which, per our analysis, physicians demonstrated a skill from more than one competency domain. During these episodes, we identified two capabilities: finding complexity, and being patient-centered. We also described two mechanisms of integration for these capabilities: valuing the patient’s narrative of their illness, and integrated understanding. Our results have important implications for the development of adaptive expertise and the use of competency frameworks in physician training.

Our data provide empirical support for the premise that the competencies and roles outlined in frameworks such as CanMEDS may operate as foundational building blocks for higher-order capabilities that are critical for adaptive expertise.11,23 Our results reinforce the conclusion that competency must be understood as an integrated construct if the goal of competency-based medical education is the development of expertise. Our findings also offer some guidance on the ways in which medical educators might use competency frameworks to better train for expert capabilities. For example, capabilities such as finding complexity and being patient-centered demonstrate not only particular ways in which the flexible and situated use of knowledge is integral to effective patient care but also how integrated competencies underpin such capabilities. However, while integrated competencies appear critical for expert capabilities, it is important to note that the performance of individual competencies in tandem is insufficient. For example, having knowledge of a particular disease (being a medical expert) and effectively communicating (being a communicator) does not always equate to being patient-centered.

Our data suggest two mechanisms for integration: integrated understanding, and valuing the patient’s narrative of their illness. Taken together, these mechanisms, as enacted by our participants, emphasize the effect of integrated knowledge, including humanistic knowledge, as foundational for expertise. Our findings align with previous research conducted in experimental settings that shows the importance of the cognitive integration of basic and clinical science knowledge for diagnosing difficult cases and preparing for future learning.12,24 Importantly, our findings also expand on this previous work in two key ways. First, they demonstrate the ways in which individuals can seek to build an integrated understanding using the “affordances” (or possibilities) of their own context.20 Second, our findings highlight the potential value of including an expanded understanding of basic and clinical science that incorporates humanistic knowledge and system affordances.25

This study has a number of implications for medical education, especially in the current era of competency-based medical education. Most important, teaching and evaluating competencies independently is not only seemingly impossible,7 it is also not desirable if the goal of training is adaptive expertise.26 Our results show that expert capabilities can be understood as the integration of physician roles or core competencies within existing competency frameworks. Trainees must therefore have the opportunity to practice the expert capabilities associated with integrated competencies. For example, clinics, especially CCCs like the one in our study, offer powerful opportunities for trainees to engage in guided deliberate practice of the expert capabilities necessary for preparing them to effectively care for patients with chronic, complex illness.15 Similarly, educators must explicitly value these capabilities and the mechanisms that support them by incorporating opportunities for learning and assessment that are conducive to integration. To do this, educators and education systems must shift from an exclusive focus on competency domains (and their associated EPAs) to explicitly train for the expert capabilities that are represented in the interstitial space where they connect, align, and come together.

We note that this study has some limitations. We conducted the research in a CCC that offers patient care opportunities that are particularly conducive to developing expert capabilities. While we would argue that such opportunities provide great value for training, the nature of the patients and the interprofessional clinical environment is not representative of all postgraduate teaching clinics. Systematically exploring the enactment of integrated competencies in different settings would lead to a better understanding of the learning opportunities afforded by different educational contexts.

Back to Top | Article Outline

Conclusions

To better understand and foster expert capabilities in medical education is imperative. Our findings suggest that expert capabilities are not the aggregation of excellence in individual roles or competencies as defined by frameworks like CanMEDS and ACGME. Rather, excellence comes from the effective integration of competencies during patient care that best represents expertise. Understanding integrated competencies as the building blocks of expert capabilities as well as the mechanisms that support integration offers an opportunity to use existing competency-based frameworks to understand and teach adaptive expertise.

Back to Top | Article Outline

References

1. van der Vleuten CP, Schuwirth LW, Scheele F, Driessen EW, Hodges B. The assessment of professional competence: Building blocks for theory development. Best Pract Res Clin Obstet Gynaecol. 2010;24:703–719.
2. Eva KW, Lohfeld L, Dhaliwal G, Mylopoulos M, Cook DA, Norman GR. Modern conceptions of elite medical practice among internal medicine faculty members. Acad Med. 2011;86(10 suppl):S50–S54.
3. Frank JR. The CanMEDS 2005 Physician Competency Framework. 2005. Ottawa, Ontario, Canada: Royal College of Physicians and Surgeons of Canada; http://www.royalcollege.ca/portal/page/portal/rc/common/documents/canmeds/resources/publications/framework_full_e.pdf. Accessed March 2, 2017.
4. Leach DC. Evaluation of competency: An ACGME perspective. Accreditation Council for Graduate Medical Education. Am J Phys Med Rehabil. 2000;79:487–489.
5. Frank JR, Danoff D. The CanMEDS initiative: Implementing an outcomes-based framework of physician competencies. Med Teach. 2007;29:642–647.
6. Carraccio CL, Englander R. From Flexner to competencies: Reflections on a decade and the journey ahead. Acad Med. 2013;88:1067–1073.
7. Lurie SJ, Mooney CJ, Lyness JM. Measurement of the general competencies of the Accreditation Council for Graduate Medical Education: A systematic review. Acad Med. 2009;84:301–309.
8. Huddle TS, Heudebert GR. Taking apart the art: The risk of anatomizing clinical competence. Acad Med. 2007;82:536–541.
9. Neve H, Hanks S. When I say … capability. Med Educ. 2016;50:610–611.
10. Schwartz DL, Bransford JD, Sears D. Mestre JP. Efficieincy and innovation in transfer. In: Transfer of Learning From a Modern Multidisciplinary Perspective. 2005:Greenwich, CT: Information Age Publishing; 1–52.
11. Mylopoulos M, Lohfeld L, Norman GR, Dhaliwal G, Eva KW. Renowned physicians’ perceptions of expert diagnostic practice. Acad Med. 2012;87:1413–1417.
12. Woods NN, Brooks LR, Norman GR. The role of biomedical knowledge in diagnosis of difficult clinical cases. Adv Health Sci Educ Theory Pract. 2007;12:417–426.
13. Woods NN. Science is fundamental: The role of biomedical knowledge in clinical reasoning. Med Educ. 2007;41:1173–1177.
14. Lesgold AM, Rubinson H, Feltovisch, Glaser R, Klofper D, Wang Y. Chi MTH, Glaser R, Farr MJ. Expertise in a complex skill: Diagnosing X-ray pictures. In: The Naure of Expertise. 1988:Hillsdale, NY: Lawrence Erlbaum Associates Inc.; 311–342.
15. Sklar DP. Integrating competencies. Acad Med. 2013;88:1049–1051.
16. Mylopoulos M, Regehr G. Putting the expert together again. Med Educ. 2011;45:920–926.
17. Whitehead CR, Kuper A, Hodges B, Ellaway R. Conceptual and practical challenges in the assessment of physician competencies. Med Teach. 2015;37:245–251.
18. Carraccio C, Englander R, Gilhooly J, et al. Building a framework of entrustable professional activities, supported by competencies and milestones, to bridge the educational continuum. Acad Med. 2017;92:324–330.
19. Hutchins E. Cognitive ecology. Top Cogn Sci. 2010;2:705–715.
20. Hutchins E. Cognition in the Wild. 1995.Cambridge, MA: MIT Press.
21. Williams RF. Using cognitive ethnography to study instruction. In: Proceedings of the 7th International Conference of the Learning Sciences. 2006. Mahwah, NY: Lawrence Erlbaum Associates; http://hci.ucsd.edu/102b/readings/williams-cogethno.pdf. Accessed April 7, 2017.
22. Gold R. Roles in sociological field observation. Soc Forces. 1958;36:213–.
23. Norman G. Research in clinical reasoning: Past history and current trends. Med Educ. 2005;39:418–427.
24. Mylopoulos M, Woods N. Preparing medical students for future learning using basic science instruction. Med Educ. 2014;48:667–673.
25. Lucey CR. Medical education: Part of the problem and part of the solution. JAMA Intern Med. 2013;173:1639–1643.
26. Sockalingam S, Mulsant BH, Mylopoulos M. Beyond integrated care competencies: The imperative for adaptive expertise. Gen Hosp Psychiatry. 2016;43:30–31.
© 2017 by the Association of American Medical Colleges