Secondary Logo

Journal Logo

RIME: Assessment of Performance

Renowned Physicians’ Perceptions of Expert Diagnostic Practice

Mylopoulos, Maria PhD; Lohfeld, Lynne PhD; Norman, Geoffrey R. PhD; Dhaliwal, Gurpreet MD; Eva, Kevin W. PhD

Author Information
doi: 10.1097/ACM.0b013e31826735fc
  • Free


Educators in the health professions community have made extensive efforts both to understand expert performance and to create appropriate competency frameworks that can be translated into developing more effective experts. In the field of medicine, these explorations of expertise have traditionally focused on physicians’ diagnostic acumen, with an emphasis on understanding the knowledge that underpins effective and efficient clinical reasoning. Whereas researchers have identified important expert–novice differences in physicians’ organization and use of knowledge,1 researchers also increasingly recognize that not all physicians achieve the same level of expert performance, with some merely “swelling the ranks of the mediocre.”2

Studies exploring facets of expert performance that range beyond physicians’ diagnostic acumen have reinforced this reality. This research has emphasized the ways in which “adaptive experts” not only use knowledge effectively but also are able to construct new knowledge in response to novelty in their practice, thereby engaging in a process of sustained practice improvement.3 Moreover, our own research has shown that physicians’ conceptions of exceptional medical practice increasingly reflect a diverse set of skills and professional aptitudes that extend beyond the knowledge that one possesses.4

To foster excellence in medical practice, we believe that it is imperative that researchers extend their studies of expert performance to uncover the particular traits, experiences, and habits of mind that likely shape the performance of exceptional physicians. In this study, we explored the ways in which physicians who were nominated by their peers as “exceptional” across a range of competencies conceptualized their own diagnostic expertise specifically and diagnostic excellence generally.


From 2008 to 2010, we conducted a constructivist grounded theory,5 interview-based study, intended to contribute to the development of a substantive theory of expert diagnostic practice. We conducted this study as phase two of a larger, three-phase research project. The first phase consisted of a peer nomination survey exploring modern conceptions of expert medical practice, which also provided us with our sample of prospective participants.4 The last phase was a “think aloud” exploration of six diagnostic cases with the same participants. We obtained ethical approval from the institutional review boards at six American and Canadian research sites (Mayo Medical School; University of Michigan Medical School; University of California, San Francisco, School of Medicine; University of Toronto Faculty of Medicine; McMaster University Faculty of Health Sciences; and University of Ottawa Faculty of Medicine).


Our purposive sampling approach combined both criterion and theoretical sampling strategies.6 In phase one of the project, we administered a peer nomination survey in the departments of medicine of the six participating institutions—Mayo Medical School; University of Michigan Medical School; University of California, San Francisco, School of Medicine; University of Toronto Faculty of Medicine; McMaster University Faculty of Health Sciences; and University of Ottawa Faculty of Medicine. We chose to limit our sample to academic faculty to ensure that those surveyed had contact with a large number of individuals within their specialty, thereby increasing the likelihood that they could identify a core group of exceptional physicians. Restricting our sample to internal medicine faculty ensured that any variability in their reasons for nominating an exceptional colleague could not be attributed simply to the variability across specialties.

We asked all internal medicine faculty via e-mail to complete a Web-based survey nominating peers whom they considered to be “outstanding practitioners.” To avoid biasing respondents to particular aspects of practice, we did not define the phrase “outstanding practitioner” any further. We then asked respondents to nominate “exceptional diagnosticians” and “exceptional professionals.”4 We did not elaborate further on the former term but did define the latter as being “based on ethical decision-making, communication skills, or relationships with patients.”

We invited the most frequently nominated physicians from both groups at each institution to participate in phase two of our study. We recruited an additional six participants from the University of California, San Francisco, School of Medicine, three of whom we identified as exceptional diagnosticians because they routinely give public demonstrations of diagnostic reasoning at national and international conferences, and three others who have firm reputations locally as exceptional diagnosticians.

Our saturation sample7 included 34 participants: 20 nominated as exceptional diagnosticians, 14 nominated as exceptional professionals. Twenty-six of the 34 (76%) participants were men. Participants had an average age of 54.5 years (standard deviation [SD] = 9.3), had been in practice for an average of 24.2 years (SD = 9.8), and worked an average of 61.3 hours per week (SD = 16.9). All participants claimed internal medicine as their specialty area of practice; some also self-identified a subspecialty—infectious disease (n = 4), critical care (n = 3), pulmonology (n = 3), and gastroenterology (n = 1).

Data collection

For our current study, phase two of the project, we conducted a one-hour, semistructured interview with each participant. We developed the interview protocol in a consultative process during a daylong meeting that included members of our research team and four research scientists and/or medical educators with strong knowledge of the general expertise literature. We first explored each participant’s conceptualizations of outstanding clinical practice, diagnostic acumen, expert problem solving, and personal life and career trajectory. We then asked the participants to describe diagnostic experiences that they found to be “easy,” “hard,” or “stumpers” (i.e., cases they were never able to solve). Using a constant comparative approach,5 we conducted our data analysis concurrently with the data collection process, which allowed us to adjust the semistructured interview protocol as well as the participant and site selection to more effectively explore emergent themes related to expert diagnostic practice. We audiotaped these interviews with each participant’s permission, then transcribed them and entered the data into NVivo Version 8 qualitative data analysis software (Doncaster, Australia) to facilitate our analysis.

Data analysis

We conducted a constructivist grounded theory analysis of the interview transcripts. Each member of the research team (five investigators from three of the research sites and a research assistant) individually identified the preliminary codes as they emerged during data collection to facilitate the constant comparative analysis.7 Consistent with our constructivist grounded theory approach, each member of the team used his or her own theoretical lens to explore the data, resulting in an analytic process that was both inductive—themes developed as they emerged from the data—and deductive—researchers purposefully used their prior data and theoretical perspectives to develop the emerging thematic structure.

We discussed the preliminary codes at team meetings until we reached an agreement on appropriate codes and their meaning. We adopted this team approach to data analysis to ensure that multiple perspectives were involved in the process,8 and we maintained this approach through three iterations of data collection and analysis until we collectively developed a stable thematic structure through the iterative relating and grouping of codes. During this process, we purposefully explored emerging themes and identified negative examples of the initial thematic structure to ensure representativeness of the results.8

In the final stage of analysis, the research assistant used NVivo to apply our thematic structure to the entire data set and determine the prevalence of each theme across the participants. We maintained an audit trail of the developing thematic structure, including our notes from team meetings, Excel spreadsheets of the codes at various stages of the analysis, and copies of the evolving NVivo database.


Four interconnected themes informing our understanding of how physicians conceptualize expert diagnostic practice emerged from our analysis of the interview transcripts: (1) possession of extensive knowledge built through purposeful, continuous engagement in clinical practice, (2) possession of the skills to effectively gather patient “stories,” (3) reflective integration of that knowledge and those patient stories during diagnosis, and (4) continuous learning through clinical practice. We found no differences in the prevalence of these themes across those nominated as exceptional diagnosticians versus those nominated as exceptional professionals.

Theme 1: Possession of extensive knowledge built through purposeful, continuous engagement in clinical practice

Participants emphasized the importance of acquiring and maintaining an extensive knowledge base. Notably, they saw the development and maintenance of knowledge as an active process of continued engagement in clinical practice:

See patients. Be interested in clinical work. You cannot be a good clinician, diagnostician just by reading. It’s like flying a plane or to be good at hockey—you have to practice. To become a golfer you have to hit the ball. You just can’t read about golfing and be a good golfer. So seeing more patients, see a variety of patients.

Whereas participants emphasized that the daily learning that takes place through clinical practice is integral to the maintenance of the extensive knowledge base underpinning excellence, they also commented on the importance of reading both for maintaining a deep knowledge in their specialty and for gaining breadth across the broader medical domain:

Even though that point or that article is seemingly actually quite irrelevant to anything I’m doing currently, six or eight months from now that might become relevant. And to tell you the truth I won’t remember a lot in great detail. But, it seems to give me that layering and background and ability to then have that additional little bit of knowledge that maybe influences my decision about some clinical point.

Theme 2: Possession of the skills to effectively gather patient stories

Participants noted the importance of acquiring knowledge that they described as patient stories during the diagnostic process. They highlighted the skills that enabled them to acquire these stories, including effective communication and technical skills related to physical examination:

Actually, I think most important is learning to listen. It’s the skill of actually hearing what the patient wants to tell you and not trying to get the patient to tell you what you want to hear. Those are two very different things. I think the less skilled clinician, less experienced clinician interrupts, keeps asking questions, directs the patient rather than actually sitting back and actually hearing what the patient has to say because the vast majority of diagnosis happens with your ears, not with your eyes.

A technical proficiency of the physical exam is crucial. You have to do it right; otherwise, you’ll never get correct information.

Participants also commented on the importance of understanding patient interactions through observing and collating all available knowledge into a patient story:

Being able to observe things as an active process is a really important part of an expert diagnostician. Elite clinicians actually do that. They have not only knowledge and technical skills, but they have the ability to pick up the physical things by hearing, looking at it, percussing it, and then have to put it all together, which is another talent they have.

Theme 3: Reflective integration of that knowledge and those patient stories during diagnosis

Participants understood the process of diagnosis to occur through the integration of past knowledge and experience with a current patient story:

Diagnostic expertise is based on a careful, thorough approach to the patient where you take into account all the available clinical data. By that I mean, their appearance, their behavior, the content of their history, and their physical examination combined with experience, which is of enormous importance because the first requirement I mentioned by itself doesn’t do the trick. It has to be linked with experience.

In addition, participants recognized the primacy of pattern recognition for integrating and using knowledge efficiently during the diagnostic process:

So I think you have to take your experiential database … to allow you to kind of make short cuts to have those heuristics, that pattern recognition. That only can come with experience. You have to have seen 20 different variations of the theme of heart attacks to be able to recognize the common as well as the subtle atypical presentations.

However, participants also articulated a necessary and complementary reflective stance toward the evolving diagnosis, thereby recognizing the possibility that pattern recognition can lead to misdiagnosis:

One of my traps or pitfalls that I fall into sometimes is that you kind of assume this is going to fall into the pattern and you kind of get anchored to a certain condition. And it’s hard to keep the flexibility of thought to say, “Okay, what else could be going on?” and “Why is this not fitting?” as opposed to trying to jam a square peg into the wrong hole.

Moreover, participants noted the importance both of using their experience to recognize when their existing knowledge was insufficient for the necessary diagnosis and of using their current patient assessment as an opportunity to cultivate a deep understanding of their clinical domain:

The main resource I used was my experience and knowledge of the natural history of this infection and the knowledge that this was not behaving as an uncomplicated infection should act.

To sort of get to the underlying cause of things, asking that why question like the two-year-old, “Why do zebras have stripes?” And kind of getting to that level of not just accepting things at face value, but really … keep going, drilling through, persevering and trying to get to the root cause. I think that sense of discovery is important.

Theme 4: Continuous learning through clinical practice

For participants, the reflective approach to the activity of diagnosis was mirrored by their broader approach to medical practice that shaped them as lifelong learners through the adoption of a skeptical stance toward their own knowledge as well as the need for humility as clinicians engaged in practice-based learning;

Not to the point of an anarchist or a rebel but to think that every time I do something that I’ve learned to do or learned for the data, you wonder how long this is going to be true and that’s what I mean by becoming comfortable with the absence of certitude.

I think being humble. I think humility means learning from your mistakes also because you remember when you didn’t do something fundamental so you don’t do it the next time. That’s part of being an expert too.


Our results reveal a model of excellence that emphasizes four interconnected dimensions of diagnostic practice. Participants described a process of acquiring and maintaining knowledge through continuous engagement in clinical practice, as well as reading for breadth and depth. They also emphasized the importance of developing the skills that allow them to effectively gather knowledge in the form of patient stories during diagnosis. They described the process of diagnosis as the merging of previous knowledge with a current patient story, overlaid by a reflective approach enabling the continuous construction of knowledge and learning through problem solving. Exploring our results within the context of current discourses in medical education brings to the forefront two key features of this construct of diagnostic excellence: (1) the integrated nature of the medical competencies that underpin the diagnostic process, and (2) the “optimally adaptive”3 nature of the diagnostic process.

The Accreditation Council for Graduate Medical Education (ACGME) core competencies9 and the CanMEDS physician roles10 emphasize the importance of physicians maintaining a broad skill set beyond a strong knowledge base. The model of excellence that emerged from our study highlights the ways in which physicians integrate different competencies both during the diagnostic process and in their broader approach to maintaining and improving their skills as diagnosticians. In particular, participants emphasized the skills that are necessary to effectively acquire knowledge—They articulated the ways in which excellence in the CanMEDS medical expert role, or the ACGME medical knowledge competency, is dependent on excellence in a number of other physician roles. For example, our findings suggest that the way in which reputed physicians conceive of diagnostic practice reflects a close entwining of the medical expert, communicator, and scholar CanMEDS roles, or the medical knowledge, interpersonal and communication skills, and practice-based learning and improvement ACGME competencies.

Taken together with our findings from phase one of this project, which showed that physicians nominated colleagues as exceptional practitioners because those physicians possessed any of a variety of skills,4 the results that we report here suggest that excelling in any one competency or role alone is insufficient to be an exceptional physician. Moreover, they demonstrate that physicians understand their own and others’ medical practice as an integrated construct, pointing to the possibility that, as medical educators, we might need both to focus more attention on understanding the ways in which exceptional practitioners make effective and appropriate use of a broad, interrelated skill set in their practice and to make explicit this integrated competency model for our trainees. Essentially, in helping students and residents strive for diagnostic expertise and outstanding medical knowledge, we must simultaneously cultivate their practice-based learning and communication skills and treat these skills as a crucial means toward their development of diagnostic excellence, rather than simply as independent components of good clinical performance. In this regard, assessment practices that deliberately focus on a particular competency may be counterproductive, bringing further into question the wisdom of assessing competencies independently.11

Participants described this integration of skills in their definition of diagnosis as a process of combining prior knowledge with patient stories while adopting a reflective12,13 stance toward a diagnosis as it evolves during a patient encounter. For example, participants described using knowledge not only as a means to efficiently diagnose a patient through a process of pattern recognition but also as a resource to construct new knowledge in challenging diagnostic cases. We understand this process to be at work when a physician is working within the “optimal adaptability corridor,”3 as described by adaptive expertise theorists. The optimal adaptability corridor refers to the ability of adaptive experts to maintain an effective balance between complementary efficient (use of past knowledge) and innovative (construction and learning of new knowledge) dimensions of their practice. In effect, these experts are able to use their past knowledge without being constrained or misled by its limitations, while also constructing new knowledge for effective problem solving when necessary. Working within the optimal adaptability corridor allows experts to continuously learn as they problem solve, a process that participants conceptualized as “continuous learning through clinical practice.”

Whereas most accounts3 of optimal adaptability focus on the balance of efficiency and innovation at the level of maintaining a high standard of clinical practice, our results are of particular interest because they demonstrate how physicians might engage in optimally adaptive problem solving while working on a single diagnostic problem. Specifically, the balance between efficiency and innovation can shift within a diagnostic episode—To perform at the highest level of proficiency, physicians must remain sensitive to the possibility that a seemingly straightforward diagnostic case may bring unanticipated challenges, therefore becoming an opportunity for learning. This articulation of optimal adaptability supports the assertion by adaptive expertise theorists that innovation is a pervasive part of expert diagnostic practice that can occur during problem solving and is complementary to efficient diagnostic practice.3

We limited this study, phase two of a larger project, to exploring physicians’ perceptions of the processes and activities that form the foundation of diagnostic excellence. In the subsequent phase of the project, we explored actual differences in diagnostic performance among participants, the results of which will further inform our evolving understanding of diagnostic excellence. The results of phase two demonstrate a consistent approach among participants toward diagnostic excellence, emphasizing important ways in which physicians conceptualize their own and others’ diagnostic practice, with implications for the development and maintenance of diagnostic excellence. Given the results from phase one of the project, which demonstrated that broadening conceptions of exceptional medical practice are emerging in the physician community,4 our finding in phase two that physicians’ constructions of excellence do not appear to deviate across participant groups is less surprising than it might have been otherwise.

The consistency between the data that we report here and the current literature on the integrated nature of medical competencies and the adaptive nature of expertise suggests that researchers may be able to translate the results of studies on physician expertise into practical and effective pedagogical strategies to guide trainees, clinicians, and medical educators who strive for excellence.

Acknowledgments: The authors would like to thank Betty Howey for her invaluable assistance with data collection and analysis.

Funding/Support: This project was funded by a grant from the Social Sciences and Humanities Research Council.

Other disclosures: None.

Ethical approval: This study was approved by the institutional review boards at Mayo Medical School; University of Michigan Medical School; University of California, San Francisco, School of Medicine; University of Toronto Faculty of Medicine; McMaster University Faculty of Health Sciences; and University of Ottawa Faculty of Medicine.


1. Norman G. Research in clinical reasoning: Past history and current trends. Med Educ. 2005;39:418–427
2. Bereiter C, Scardamalia M Surpassing Ourselves: An Inquiry Into the Nature and Implications of Expertise. 1993 La Salle, Ill Open Court
3. Schwartz DL, Bransford JD, Sears DMestre JP. Efficiency and innovation in transfer. In: Transfer of Learning: Research and Perspectives (Current Perspectives on Cognition, Learning, and Instruction). 2005 Charlotte, NC Information Age Publishing
4. Eva KW, Lohfeld L, Dhaliwal G, Mylopoulos M, Cook DA, Norman GR. Modern conceptions of elite medical practice among internal medicine faculty members. Acad Med. 2011;86(10 suppl):S50–S54
5. Charmaz K Constructing Grounded Theory: A Practical Guide Through Qualitative Analysis. 2006 London, UK Sage
6. Patton MQ Qualitative Research & Evaluation Methods. 1990 Newbury Park, Calif Sage Publications, Inc.
7. Corbin J, Strauss A Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. 1990 Newbury Park, Calif Sage Publications, Inc.
8. Lincoln YS, Guba EG Naturalistic Inquiry. 1985 Newbury Park, Calif Sage Publications, Inc.
9. Accreditation Council for Graduate Medical Education.. Common Program Requirements: General Competencies. 2007 Chicago, Ill Accreditation Council for Graduate Medical Education
10. . Royal College of Physicians and Surgeons of Canada. CanMEDS 2005 Framework. 2012 Accessed June 5
11. Lurie SJ, Mooney CJ, Lyness JM. Measurement of the general competencies of the Accreditation Council for Graduate Medical Education: A systematic review. Acad Med. 2009;84:301–309
12. Schon DA The Reflective Practitioner: How Professionals Think in Action. 1983 London, UK Basic Book, Inc
13. Mann KV. Reflection: Understanding its influence on practice. Med Educ. 2008;42:449–451
© 2012 Association of American Medical Colleges