Secondary Logo

Journal Logo

What’s Behind the Scenes? Exploring the Unspoken Dimensions of Complex and Challenging Surgical Situations

Cristancho, Sayra M. PhD; Bidinosti, Susan J.; Lingard, Lorelei A. PhD; Novick, Richard J. MD, MSc; Ott, Michael C. MD, MSc; Forbes, Tom L. MD

doi: 10.1097/ACM.0000000000000478
RIME
Free

Purpose Physicians regularly encounter challenging and/or complex situations in their practices; in training settings, they must help learners understand such challenges. Context becomes a fundamental construct when seeking to understand what makes a situation challenging and how physicians respond to it; however, the question of how physicians perceive context remains largely unexplored. If the goal is to teach trainees to deal with challenging situations, the medical education community requires an understanding of what “challenging” means for those in charge of training.

Method The authors relied on visual methods for this research. In 2013, they collected 40 snapshots (i.e., data sets) from a purposeful sample of five faculty surgeons through a combination of interviews, observations, and drawing sessions. The analytical process involved three phases: analysis of each drawing, a compare-and-contrast analysis of multiple drawings, and a team analysis conducted in collaboration with three participating surgeons.

Results Findings demonstrate that experts perceive the challenge of surgical situations to extend beyond their procedural dimensions to include unspoken, nonprocedural dimensions—specifically, team dynamics, trust, emotions, and external pressures.

Conclusions Findings show that analysis of surgeons’ drawings is an effective means of gaining insight into surgeons’ perceptions. The findings refine the common belief that procedural complexity is what makes a surgery challenging for expert surgeons. Focusing exclusively on the procedure during training may put trainees at risk of missing the “big picture.” Understanding the multidimensionality of medical challenges and having a language to discuss these both verbally and visually will facilitate teaching around challenging situations.

Dr. Cristancho is assistant professor, Department of Surgery, and scientist, Centre for Education Research & Innovation, Western University, London, Ontario, Canada.

Ms. Bidinosti is research associate, Centre for Education Research & Innovation, Western University, London, Ontario, Canada.

Dr. Lingard is professor, Department of Medicine, and director and scientist, Centre for Education Research & Innovation, Western University, London, Ontario, Canada.

Dr. Novick is professor, Department of Surgery, Division of Cardiac Surgery, Western University, London, Ontario, Canada.

Dr. Ott is associate professor, Department of Surgery, Division of General Surgery, Western University, London, Ontario, Canada.

Dr. Forbes is professor and division chair/chief, Department of Surgery, Division of Vascular Surgery, Western University, London, Ontario, Canada.

Funding/Support: This study was funded in part by the Canadian Institutes of Health Research through an Open Operating Grant and by the Royal College of Physicians and Surgeons of Canada through a Medical Education Grant.

Other disclosures: None reported.

Ethical approval: This study was approved by the Health Sciences Research Ethics Board of Western University.

Correspondence should be addressed to Dr. Cristancho, Health Sciences Addition, Room 110B, Schulich School of Medicine and Dentistry, Western University, London, Ontario, N6A 5C1, Canada; telephone: (519) 854-1790 ext. 89253; e-mail: Sayra.Cristancho@schulich.uwo.ca.

The current health care landscape requires physicians to care for extremely complex patients. As a consequence, physicians encounter challenging and/or complex situations in their practices regularly; in clinical training settings, they must help medical learners understand such challenges. The increasing complexity of both medical practice and medical teaching has prompted recent calls for more holistic approaches to medical education research,1,2 approaches that generate “rich understandings” of the individual’s interaction with the constantly evolving context in which he or she is practicing1 (emphasis added). Context, therefore, becomes a fundamental construct for seeking to understand both what makes a situation complex and how people respond to it. How context influences physicians’ responses to a situation has been the focus of many research efforts in medical education.3–5 Although responding to context is certainly important, the equally important question of how people perceive such context remains largely unexplored.

Several areas in medical education research have started to give explicit prominence to the notion of context.6–15 To illustrate, Durning and colleagues4 have suggested that the academic medicine community’s understanding of the term “context” should expand from, simply, (in our words) “static physical location or setting” to “a situation that is evolving.” Context therefore becomes a fundamental construct for seeking to understand not only what makes a situation complex but also how people respond to it. Investigators studying expertise-in-context, for instance, have uncovered and articulated procedural factors that can lead to errors,16 emotional and psychological reactions that result from adverse intraoperative events,17–21 and contextual factors that influence internists’ clinical reasoning.22 This work has advanced the community’s understanding of how experts respond to and make decisions under the influence of multiple competing contextual factors. However, an appreciation of how experts perceive those factors is necessary to explore why they respond as they do. If the goal is to train novices to effectively deal with complexity, the academic medicine community requires an understanding of what complexity really means for those in charge of the training.

In this report, we use a systems engineering (SE) approach23 to begin to address the question of how people perceive contextual influences. One key premise in SE is that of “perspectives,” the unique viewpoints from which particular dimensions of a situation can be understood.23 The notion of “perspectives” in SE is particularly useful for identifying multiple dimensions beyond those that are explicitly valued in a particular domain. The surgical literature suggests that a dominant dimension is the procedural one24,25; our research asks, What other dimensions exist, what are the relationships among these dimensions, and how do they influence expert surgeons’ perspectives of complex, challenging situations? We anticipate that such an understanding will not only further enrich conversations between trainers and trainees about complex situations in medicine but also illuminate how experts understand these situations and how best to train novices to navigate them effectively.

Back to Top | Article Outline

Method

Our SE research approach relies on visual methods.26,27 We have previously demonstrated the usefulness of combining visual tools with interviews to uncover the evolving and emerging nonprocedural dimensions of complex and challenging situations in expert practice.28 In this report, our goal is to describe those dimensions from the perspective of surgical experts. Here, we summarize our methods in order to provide necessary background for the analytical approach we designed and implemented.

The Health Sciences Research Ethics Board of Western University approved the following research design. In 2013, we sampled faculty surgeons purposively to include rich informants from four surgical specialties (hepatobiliary, colorectal, cardiac, and vascular) at two academic hospitals, who were willing to reflect on their surgical experiences. We collected 40 snapshots28 (i.e., sets of data points from a particular point in time) from five surgeons. In this study, we used these snapshots to capture particular moments during which surgeons reflected on their understanding of how a challenging situation evolved. Each situation consisted of four snapshots: (1) a preoperative one-on-one interview in which surgeons verbally shared their perspectives on the potential challenges that they anticipated would play a role in the situation; (2) an intraoperative observation through which we documented the evolution of the situation, including both the participating surgeon’s actions and any team interactions within the boundaries of the operating room; (3) a postoperative one-on-one interview in which surgeons verbally described how, from their perspective, the situation, including any unanticipated events, actually played out; and (4) a postoperative drawing, completed within a week of the surgical case, along with an interview about the drawing, through which surgeons visually represented and described, from their perspective, the complexity of the situation as it evolved. In this study we used a particular type of drawing or visual method called “Rich Picture”26 to attempt to capture surgeons’ perspectives in ways that interviews alone cannot.28,29 All of the semistructured interviews (contact authors for the interview protocol) were performed in person and lasted between 30 and 60 minutes. We audio recorded the interviews, and the transcripts were anonymized.

This report focuses on our analysis of the drawings or “rich pictures,” triangulated30 both by analysis of the preop and postop interviews and by the intraoperative observations. Two members of the research team with professional backgrounds in engineering (S.M.C.) and the arts (S.J.B.) led the data analysis using an aesthetic approach to visual analysis.28,31,32 The analytical process involved three phases: (1) individual aesthetic analysis of each drawing, followed by (2) a compare-and-contrast analysis of multiple drawings, and finally (3) a team analysis conducted in collaboration with participating key informants (key informants are experts; in our case, they were experienced surgeons, regarded as highly insightful by their peers, who were also participants in the study). The combination of outsider (i.e., researcher) and insider (i.e., surgeon participant) analytic perspectives was purposeful and reflective of the constructivist epistemology33 underpinning our research. Our analytical process was guided by a set of analytical questions, which we had previously developed as part of an innovative visual methodology.28

In Phase 1, we developed an inductive description of the individual drawings by, first, focusing on the concrete elements in each, and moving gradually to considering the meaning of the elements individually and in relation to other elements. Our goal was to understand the overall story the drawing presented.

In Phase 2, we engaged in a cross-comparison of multiple drawings to identify common visual elements and to begin exploring dimensions shared across the drawings (and stories). Those dimensions constituted the emergent themes of the study (described in Results, below).

We conducted Phase 3 in the form of a team exercise, reflecting our epistemological stance to visual analysis by which the visual is treated as a social construction, a product of the encounter between researcher and research participant.31 Three of the five participating surgeons participated in a return-of-findings session designed to explore their viewpoints on our analytical results from Phase 1 and Phase 2 (the remaining two surgeons had scheduling conflicts that precluded their participation). We held these follow-up sessions with participants in order to achieve consensus in the interpretation and analysis of the data.

Throughout the analysis, to ensure the quality of our study, we used various criteria of rigor, including reciprocity, reflexivity, resonance, triangulation of data among researchers and participants, and the formation of an audit trail of the analytical process.34–36

Back to Top | Article Outline

Results

Our data set consists of snapshots from different points in time that capture the perspectives of experienced surgeons on specific complex and challenging operations. Taken together, the data illustrate multiple dimensions of complexity: procedural, team dynamics, trust, emotions, and external pressures. Whereas the interviews were dominated by the procedural dimension, the drawings revealed primarily the nonprocedural dimensions. In their interviews about their drawings, the participating surgeons often described these dimensions as pivotal factors, the factors that actually determined the level of challenge. These nonprocedural dimensions were powerfully depicted in the drawings through visual metaphors, such as a castaway, a sports field, and William Tell and the apple. Below we discuss the four main nonprocedural dimensions identified in our study—team dynamics, trust, emotions, and external pressures—illustrating each with representative pictorial elements from the rich pictures (Figure 1A-1J) as well as with quotes from the interviews.

Figure 1

Figure 1

Back to Top | Article Outline

Team dynamics

Team dynamics—which we define as the factors that influence the behavior and performance of a group—was a common nonprocedural dimension. In fact, the dimension of team dynamics appeared in almost every rich picture. Some of the participating surgeons evoked sports analogies to compare the surgical team with sports teams. For example, the following quote, paired with Figure 1A, illustrates how one surgeon used a soccer analogy to show not only the roles on the operative team but also the position of each role in relation to the patient:

This is like a sport, like a game, where everybody is playing a different role in the field. And we have the anesthetist as a goalkeeper. You cannot do this without having an anesthetist. You have radiologists, you have other surgeons, students, fellows, residents, the surgeon who tries to be in the center and tries to score the goal … this is a team approach. And of course, there is going to be some rules that we have to follow and we know that there is only one shot. [Surgeon #4, drawing interview]

In situations where personal interactions among team members had a direct impact on the proceedings of the operation, relationships among team roles were more explicitly represented, along with the dynamics produced by those relationships and the attributions of responsibility for any problematic dynamics. Figure 1B and this comment illustrate the importance of interactions and relationships among team members:

The day started with negative gestalt so you see the anesthetist, above the patient, looks very unhappy. And then, what happened is there’s a second assistant over here, who’s not the most clued-in individual on the best of days. That individual, unfortunately, clued out this morning. For instance, the operation that this individual said we planned was [operation X] instead of [operation Y]. That then led to some very negative interactions, at the beginning, between this second assistant and the anesthetist who have fought in the past, so of course, it’s never just a one-off, there’s always a preamble. [Surgeon #3, drawing interview]

A key dimension of complexity in challenging surgical situations was team dynamics—specifically, the roles within the team and the influence of the interactions among team members.

Back to Top | Article Outline

Trust

Trust—which we define as an interpersonal aspect of relationships among team members that involves reliance or dependence on another person—was also identified in most instances as a critical dimension of a complex surgical situation. Trust was depicted both inside and outside the operating room, particularly when surgeons purposefully sought the best people to depend on, as illustrated by Figure 1C and this accompanying comment:

In time we moved forward and the operation finished and this is the surgeon now with a more happy face going to communicate with the wife […]. And on the other hand, [in] the back of the head of the surgeon is, I’m leaving the city so there has to be very good communication. So at this meeting, a lot of residents were attending that meeting so I have to identify, according to me, the best people who I’m going to trust to look after that patient. [Surgeon #4, drawing interview]

As surgeons reflected on the dimension of trust, they used metaphors such as the legend of William Tell (Figure 1D) to represent the multiple lines of trust that could run among him- or herself, the patient, and the resident:

I tried to figure out how I was going to draw the trust because when you let a resident do something, it’s a bit of a leap of faith. You’ve got to trust them especially if it’s a step where once it’s done there’s no coming back. I started to draw the apple on his head but that should be on my head and you’re letting somebody shoot the arrow … and then I put the apple on the patient’s head.… You know a patient comes knowing it’s a teaching center, knowing that there are residents involved with the care, but they don’t really know to what extent. [Surgeon #2, drawing interview]

The rich pictures revealed that the dimension of trust was bidirectional. Although most times surgeons described trust as something that they provide to others (e.g., residents), receiving trust (e.g., from patients) also became an important aspect they considered when reflecting on their drawings.

Back to Top | Article Outline

Emotions

Surgeons’ drawings captured a range of emotions, including stress, frustration, regret, hope, sadness, and happiness. The surgeons’ drawings featured not only their own emotions but also those of the patient, the patient’s family, and the surgeon’s family. Additionally, surgeons described and visually represented people’s responses to others’ emotions. For example, as variously illustrated by the following quotes and their paired drawings (Figure 1E and Figure 1F, respectively), the feelings of frustration and loneliness in surgeons who had to make potentially controversial decisions were sometimes related to feelings of hope in the patient’s family.

On one end, there’s this big amorphous thing called cancer which is pulling in one direction, and on the other is the family pulling in the opposite direction, not wanting to give up.… There is a battle. It’s when families and patients haven’t come to peace with what’s going on and they want you to do things that, in some ways you feel are not potentially beneficial and it becomes a very difficult thing. [Surgeon #1, drawing interview; Figure 1E]

This is a person standing on an island saying that they’re so alone, because when you’re in that situation you have to make these decisions. You have to read the consequences for your decisions and no one else is there, really, with you. And sometimes, patients and their families put you in that situation where they say, What do you think he should do? It’s up to them to decide. I see my job as giving them information and they’re supposed to decide. But often families in this situation will put you on that island because they don’t want to be the ones responsible for making the decision. [Surgeon #1, drawing interview; Figure 1F]

Emotions were not limited to the patient’s situation; some drawings highlighted emotional aspects of the surgeon’s personal situation as a factor in complexity. As described below and illustrated in Figure 1G, the attention that a patient requires sometimes conflicts with the surgeon’s private life, prompting reflection on the emotional complexity of this balance:

This is my house and this is my wife sleeping alone. There is no husband because the husband is sitting in the chair looking at the computer, talking on the telephone, to know how the patient is doing. But the wife is very important. It’s like a rubber band—you can expand it or not to have support—without that it wouldn’t be possible. [Surgeon #4, drawing interview]

Emotion was a prevalent dimension—present, like team dynamics, in almost all drawings. Although describing the emotional impact of a situation caused some uneasiness in the surgeons, many commented on the power of disclosing and sharing the emotional aspects of complexity in surgery. One surgeon admitted in the team analysis meeting (Phase 3), “In all honesty, I still feel a little bit like you’re letting someone see a little less-veneered version of yourself,” but another acknowledged in the same meeting, “most people will be almost semirelieved if [they sense] that there are other people [who] feel the same way.”

Back to Top | Article Outline

External pressures

Another recurrent dimension depicted in the drawings was external pressures, which we define as issues other than the technicalities of the operation that create additional responsibilities. Sources of external pressure include institutions, peers or other team members, leadership or teaching responsibilities, and the need to manage one’s reputation. External pressures were visually represented as groups of pictorial elements around the surgeon, as opposed to isolated pictorial elements. Figure 1H and the accompanying explanation illustrate:

[T]here are some external things that are pushing especially when we do an operation of this type where no one else is doing this in North America. There [are] a lot of people watching you and they want to know your results, sometimes for positive and sometimes for negative things, so the division, the department, the hospital institution [are] watching you for good things and for bad things. This is kind of a storm and there can be some lightning and it can be raining on top of your head. You are under the loop and they are watching you. [Surgeon #4, drawing interview]

As mentioned, the participating surgeons mentioned the need to protect their professional image as a source of pressure. One described that particular pressure as the feeling of “stand[ing] in the middle,” facing judgment (Figure 1I).

Based on my decision making, I’m going to stand in the middle there [in morbidity and mortality rounds], even though that doesn’t really happen but that’s what it feels like. And people may be standing around saying, why did you even bother operating on this person? What were you thinking when you did this, this is ridiculous? You’ve created a problem where there was no problem. [Surgeon #1, drawing interview]

Another source of pressure was the surgeons’ awareness of the growing impact of their work (Figure 1J):

I think pioneering this in the country is good for our city, for our university, for the patients, for myself, for my division and for my department. It’s a personal satisfaction and self-fulfillment. Nationally, we have been traveling and giving some presentations and lectures on this procedure in 10 cities in the country so far. So that’s the reason of doing it and so far, the more cases we do, the more solid that I feel. [Surgeon #4, drawing interview]

Although the rich pictures revealed different sources of external pressure, all surgeons, independent of specialty, experienced this dimension of complexity, especially when engaged in a high-risk situation.

Back to Top | Article Outline

Discussion

Our results demonstrate that experts perceive the complexity of a surgical situation to extend far beyond its procedural dimension. This finding offers an important refinement of the common belief that procedural complexity is what makes a surgery challenging for expert surgeons. The visual method of drawing rich pictures was a successful way of gaining insight into the unspoken, nonprocedural dimensions that surgeons perceive in complex surgical situations. Interviews, according to the surgeons, allowed them the opportunity to recall events, whereas drawing provided them with an opportunity to reflect on the overall situation and on new things they encountered or learned that may shape how they understand other complex situations in the future. To illustrate, one surgeon commented during the team analysis meeting (Phase 3) that “drawing is an opportunity to reflect on what happened; rather than to just list step one, step two, step three.”

Surgeons frequently apologized for their “not too perfect artwork,” and they reported being taken out of their comfort zones by the request to draw. Nonetheless, they all commented that having time for themselves, to think and to draw, was a liberating experience. Further, all participants characterized the sharing of their pictures as an opportunity to disclose what was really important to them.

We acknowledge that using this visual-based approach to conduct research on a larger scale may not be feasible because of the labor intensity for participants and researchers alike. The innovative methodology we developed28 requires participants to devote about two hours to interviewing and drawing per observed case, and an additional two hours for analytical group discussions. Given the nature of surgeons’ work, which includes a very full clinical agenda, the required amount of engagement time may prevent some potential participants from taking part in a similar study. Additionally, the analytical time spent by researchers has been estimated at around six hours per surgical case. These time requirements are substantial; however, as the SE literature shows, exploring complexity requires both a detailed and a contextually sensitive exploratory approach.37–39 Such an approach requires progressively deeper, and therefore longer-term, engagement by researchers and participants to foster a co-constructed understanding of a situation as it evolves. We consequently suggest that our SE research approach to better understanding complexity is appropriate for only small-scale studies around very focused questions.

Although the procedural dimension of surgery is well described in the literature, some authors have recently begun to explore the emotional dimension of surgery, particularly as it relates to professional identity and institutional culture.17–21 With its different research focus—around complexity—our study has also identified an emotional dimension, but has moved further by offering rich insights into three additional heretofore unexplored dimensions of complexity in surgical situations (team dynamics, trust, and external pressures). We suspect that these findings were possible because of the methodological innovation we implemented. To our knowledge, this is the first study to move beyond interviewing and observing expert surgeons, to having them draw. Visual methods, such as drawings, have only recently begun to be recognized as useful research tools when the goal is to “invite stories” rather than “request reports” from participants.40,41 Drawing affords participants the opportunity to express a broader range (or more dimensions) of a challenging or complex situation by allowing them to attend to whole-part relationships within the situation.41 In particular, drawings are a source of metaphors that help spark insights.41,42 In our study, surgeons extensively commented on how visual metaphors helped them articulate aspects of their stories that they found difficult to explain with words alone, particularly aspects that emerged from the interaction between the procedural and the nonprocedural dimensions. For example, the surgeon who used a William Tell metaphor revealed that it is not just a matter of making the correct procedural decision but that it is also about how trust influences what is the right procedural decision.

As in verbal communication,43 we anticipate that “silences” or “absences” will—and should—occur in visual communication. People hold particular reasons to decide what to draw or not to draw. In our study, for instance, the academic aspects of the surgeons’ lives were not extensively represented in the drawings. In painting, visual silence is used as a strategy to emphasize a lack of balance between contrasting ideas (e.g., clarity versus obscurity).44,45 Whether or not the same notion applies to rich pictures remains an important avenue for further exploration.

Learning how to deal with complex and challenging operations in surgery is, for the most part, an “education by random opportunity” experience for trainees.46 Given surgeons’ tendency to foreground procedural elements of complexity,24,25 we suspect that their communications with trainees may also focus mostly on the procedural dimension. We contend that this tendency, either conscious or unconscious, may put trainees at risk for missing the big picture when performing complex operations. In complex environments such as an operating room, learning to recognize the messiness of lived experiences is a fundamental aspect of becoming an expert.2 Understanding the multidimensionality of surgical complexity and having a language to verbally and visually discuss this rich multidimensionality will facilitate teaching around complex and challenging surgical operations. We anticipate that such an understanding will be useful to educators in their attempts to guide trainees as the latter navigate the multiple facets of complex and challenging situations.

Acknowledgments: The authors would like to thank Dr. Chris Watling for his valuable comments and insights during the preparation of this manuscript.

Back to Top | Article Outline

References

1. Regehr G. It’s NOT rocket science: Rethinking our metaphors for research in health professions education. Med Educ. 2010;44:31–39
2. Mennin S. Complexity and health professions education. J Eval Clin Pract. 2010;16:835–837
3. Koens F, Mann KV, Custers EJ, Ten Cate OT. Analysing the concept of context in medical education. Med Educ. 2005;39:1243–1249
4. Durning SJ, Artino AR Jr, Pangaro LN, van der Vleuten C, Schuwirth L. Redefining context in the clinical encounter: Implications for research and training in medical education. Acad Med. 2010;85:894–901
5. Pimmer C, Pachler N, Genewein U. Reframing clinical workplace learning using the theory of distributed cognition. Acad Med. 2013;88:1239–1245
6. Moulton CA, Regehr G, Lingard L, Merritt C, Macrae H. “Slowing down when you should”: Initiators and influences of the transition from the routine to the effortful. J Gastrointest Surg. 2010;14:1019–1026
7. Mylopoulos M, Regehr G. Cognitive metaphors of expertise and knowledge: Prospects and limitations for medical education. Med Educ. 2007;41:1159–1165
8. Lingard L, Espin S, Evans C, Hawryluck L. The rules of the game: Interprofessional collaboration on the intensive care unit team. Crit Care. 2004;8:R403–R408
9. Ginsburg S, Regehr G, Hatala R, et al. Context, conflict, and resolution: A new conceptual framework for evaluating professionalism. Acad Med. 2000;75(10 suppl):S6–S11
10. Watling C, Driessen E, van der Vleuten CP, Lingard L. Learning from clinical work: The roles of learning cues and credibility judgements. Med Educ. 2012;46:192–200
11. Varpio L, Schryer CF, Lehoux P, Lingard L. Working off the record: Physicians’ and nurses’ transformations of electronic patient record-based patient information. Acad Med. 2006;81(10 suppl):S35–S39
12. Monrouxe LV. Identity, identification and medical education: Why should we care? Med Educ. 2010;44:40–49
13. Teunissen PW, Stapel DA, Scheele F, et al. The influence of context on residents’ evaluations: Effects of priming on clinical judgment and affect. Adv Health Sci Educ Theory Pract. 2009;14:23–41
14. Pimmer C, Pachler N, Genewein U. Contextual dynamics in clinical workplaces: Learning from doctor–doctor consultations. Med Educ. 2013;47:463–475
15. Daelmans HE, Hoogenboom RJ, Donker AJ, Scherpbier AJ, Stehouwer CD, van der Vleuten CP. Effectiveness of clinical rotations as a learning environment for achieving competences. Med Teach. 2004;26:305–312
16. Gawande AA, Zinner MJ, Studdert DM, Brennan TA. Analysis of errors reported by surgeons at three teaching hospitals. Surgery. 2003;133:614–621
17. Moulton CA, Regehr G, Lingard L, Merritt C, MacRae H. Slowing down to stay out of trouble in the operating room: Remaining attentive in automaticity. Acad Med. 2010;85:1571–1577
18. Leung A, Luu S, Regehr G, Murnaghan ML, Gallinger S, Moulton CA. “First, do no harm”: Balancing competing priorities in surgical practice. Acad Med. 2012;87:1368–1374
19. Jin CJ, Martimianakis MA, Kitto S, Moulton CA. Pressures to “measure up” in surgery: Managing your image and managing your patient. Ann Surg. 2012;256:989–993
20. Luu S, Patel P, St-Martin L, et al. Waking up the next morning: Surgeons’ emotional reactions to adverse events. Med Educ. 2012;46:1179–1188
21. Luu S, Leung SO, Moulton CA. When bad things happen to good surgeons: Reactions to adverse events. Surg Clin North Am. 2012;92:153–161
22. Durning SJ, Artino AR, Boulet JR, Dorrance K, van der Vleuten C, Schuwirth L. The impact of selected contextual factors on experts’ clinical reasoning performance (does context impact clinical reasoning performance in experts?). Adv Health Sci Educ Theory Pract. 2012;17:65–79
23. Cristancho S, Lingard L, Regehr G. From problem-solving to problem-definition: Conceptualizing expert judgment through the lens of systems engineering.
24. Jacklin R, Sevdalis N, Darzi A, Vincent C. Mapping surgical practice decision making: An interview study to evaluate decisions in surgical care. Am J Surg. 2008;195:689–696
25. Sarker SK, Chang A, Vincent C. Decision making in laparoscopic surgery: A prospective, independent and blinded analysis. Int J Surg. 2008;6:98–105
26. Armson R Growing Wings on the Way: Systems Thinking for Messy Situations. 2011 Axminster, Devon, UK Triarchy Press
27. Boardman J, Sauser B Systems Thinking: Coping With 21st Century Problems. 2008 Boca Raton, Fl: CRC Press
28. Cristancho S, Bidinosti S, Lingard L, Novick R, Ott M, Forbes T. Seeing in different ways: Introducing “rich pictures” in the study of expert judgment. Qual Health Res.
29. Hoffman RR, Shadbolt NR, Burton AM, Klein G. Eliciting knowledge from experts: A methodological analysis. Organ Behavior Hum Dec. 1995;62:129–158
30. Thurmond VA. The point of triangulation. J Nurs Scholarsh. 2001;33:253–258
31. Rose G Visual Methodologies: An Introduction to Researching With Visual Materials. 2012 Thousand Oaks, Calif Sage Publications
32. Bell E, Davison J. Visual management studies: Empirical and theoretical approaches. Int J Manag Rev. 2013;15:167–184
33. Crotty M The Foundations of Social Research: Meaning and Perspective in the Research Process. 1998 Thousand Oaks, Calif Sage
34. Tracy SJ. Qualitative quality: Eight “big-tent” criteria for excellent qualitative research. Qual Inq. 2010;16:837–851
35. Flick U Managing Quality in Qualitative Research. 2007 Thousand Oaks, Calif Sage
36. Flick U. Qualitative research —State of the art. Soc Sci Inf. 2002;41:5–24
37. Gharajedaghi J Systems Thinking: Managing Chaos and Complexity: A Platform for Designing Business Architecture. 20113rd ed Burlington, Mass Morgan Kaufmann/Elsevier
38. Johnson CW. What are emergent properties and how do they affect the engineering of complex systems? Reliab Eng Syst Safe. 2006;91:1475–1481
39. Rouse WB. Engineering complex systems: Implications for research in systems engineering. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews. 2003;33:154–156
40. Ellis J. Researching children’s experience hermeneutically and holistically. Alberta J Educ Res. 2006;52:111–126
41. Ellis J, Hetherington R, Lovell M, McConaghy J, Viczko M. Draw me a picture, tell me a story: Evoking memory and supporting analysis through pre-interview drawing activities. Alberta J Educ Res. 2013;58:488–508
42. Weber SKnowles JG, Cole AL. Visual images in research. In: Handbook of the Arts in Qualitative Research: Perspectives, Methodologies, Examples, and Issues. 2008 Thousand Oaks, Calif Sage:41–54
43. Gardezi F, Lingard L, Espin S, Whyte S, Orser B, Baker GR. Silence, power and communication in the operating room. J Adv Nurs. 2009;65:1390–1399
44. Gray C, Malins J Visualizing Research: A Guide to the Research Process in Art and Design. 2004 Burlington, Vt Ashgate
45. Zhao YQ Emptiness as a Visual Strategy: An Exploration of Visual Absence in Contemporary Art Practice [doctoral dissertation]. 2009 Auckland, New Zealand Auckland University of Technology
46. Gubrud-Howe P, Schoessler M. From random access opportunity to a clinical education curriculum. J Nurs Educ. 2008;47:3–4
© 2014 by the Association of American Medical Colleges