Moulton, Carol-anne E.; Regehr, Glenn; Mylopoulos, Maria; MacRae, Helen M.
The nature and development of expertise has been a research cornerstone in the “basic science” of health professional education for decades. Over that time, many theories of expert performance in the clinical domain have been proposed, and these theories have taken several forms. One broad set of theories has attempted to understand the apparent differences between experts and nonexperts with regard to the effortful, analytic processes that experts use (and nonexperts do not) in addressing clinical problems. Theories of this form have assumed that experts apply a “better” set of processing strategies than nonexperts and have tried to deconstruct the nature of these expert cognitive processes, to teach them to novices. The processes themselves have been identified using several labels, such as problem solving,1–4 diagnostic reasoning,5–7 clinical judgment,8,9 decision making,10,11 and critical thinking.12–14 Yet, despite extensive research and writing, there continues to be difficulty in specifying the exact nature of these expert processes. The broader construct underlying these labels has remained largely ill defined, and the labels themselves are often used interchangeably. Further, research highlighting situational specificity15 and case specificity1,16 has begun to undermine confidence that expert analytic processes even exist, at least in a useful generic sense.
In the absence of evidence that experts have a “better” set of analytic processing strategies, another broad set of theories of expertise has grown from an interest in “what is in the expert’s head” and attributes the acquisition of automatic, nonanalytic resources (obtained through experience) as the source of expertise.16 These nonanalytic resources have several variations, such as instances,17–19 scripts,16,20 schemas,20,21 and heuristics,22 and they have been associated with processes such as pattern recognition,18,20,23 chunking,24 encapsulation,25 and the situationally driven restriction of solution sets.26 An explanation for the inherent value of automatic resources may be found in the literature on attention.27 In this literature, there is consensus that humans must coordinate a constant flux of environmental information and personal intentions with limited cognitive capacity. Humans have become very good at adapting to this limitation through the development of automatic processing that requires less intentional capacity and, thereby, frees up cognitive resources for additional activity.28 Therefore, the development of pattern recognition, the formation of scripts and schemas, the restriction of solution sets, the accumulation of instances in memory, and the processes of chunking and encapsulation could be considered mechanisms that experts use to carry out their daily activities with minimal demands on cognitive load. For the expert, activities that were initially effortful become mundane and routine. This leads to greater efficiency, with experts getting to the right answer more often, more quickly, and with less effort than novices. The extreme of this process can be appreciated when experts can no longer verbalize the thought processes involved in reaching a decision or cannot accurately explain details of actions involved in carrying out a procedure.29,30
Anecdotally, evidence for the automatization of once cognitively effortful processes of clinicians is everywhere. Surgeons perform standard operations while talking about their tennis game, internists diagnose a patient from the end of the bed, emergency physicians immediately initiate multiple tasks in a trauma victim, and family doctors recognize the chicken pox rash instantly. Yet, conceptualizing expertise exclusively as the accrual and efficient use of nonanalytic resources and load-reducing automatic processes is likely insufficient.31 As Bereiter and Scardamalia32 have described, the individual who makes exclusive use of nonanalytic resources and automatic processes is unlikely to manage novel situations or unusual cases appropriately. Such an individual will tend to adapt the presenting problem to known solutions rather than adapting new solutions to the presenting problem and, in the taxonomy of Bereiter and Scardamalia,32 represents not an expert, but an experienced nonexpert. Thus, although progressing toward automaticity is essential for the development of expertise, experts must also be able to engage more cognitively effortful processes when the automatic approach is not sufficient.31,32
Although gaining an understanding of both the cognitively effortful, analytic processes and the automatic, nonanalytic resources of expertise is important, it might be argued (consistent with Bereiter and Scardamalia32) that the true hallmark of expertise may be more importantly related to the effective interfacing between these two modes of processing.33 When a clinical presentation is atypical, a postoperative patient goes off course, an unusual reaction occurs from medication, or an anatomical anomaly is confronted, will the clinician, in automatic mode, take heed and recognize the intricacies and complexities of the case and leave automatic mode, or will that clinician plow through, oblivious to its uniqueness and unaware of its consequences? When a clinician does slow down and shifts into a more effortful mode of processing, what does that look like? How is the transition from automatic to effortful coordinated? And what are the cues that the expert is using to initiate the transition? Perhaps the best phenomenological description of the two processes side by side has been provided by Schön34,35 in his comparative descriptions of knowing-in-action versus reflection-in-action. Yet, further research is clearly needed if we are to properly understand this potentially critical aspect of expert performance: the act of slowing down when you should. Toward this end, we will review the literatures that might inform the way experts transition between the automatic and the effortful. The expertise literature examines the development and application of expertise, and the attention and effort literature provides a model for understanding the limits of cognitive capacity. Using the situation awareness literature, we will explore the factors that influence the moment-to-moment control of complex environments. Finally, we will present a new model for examining expert physician judgment: one that focuses on the interface of the automatic and the effortful.
Theories of Expertise
Our first review focuses on expertise literatures that have explored the daily activities in which experts engage that define them as expert, rather than explore how experts are different from novices. As Bereiter and Scardamalia32 point out, “no one is disturbed by the fact that experienced physicians are better at diagnosis than interns.” What we might be disturbed by is a practicing physician whose expert judgment is inadequate. The recognition that not all those who have the title of expert are truly functioning as experts is an important feature of this body of literature. It challenges us to revisit and redefine what we, the medical education world, should accept as our goal in creating an expert. To this end, we will examine the theories of Dreyfus and Dreyfus,29 Bereiter and Scardamalia,32 and Schön34,35 from the perspective of how an expert manages analytic and nonanalytic resources in daily practice.
Dreyfus and Dreyfus29 describe a staged theory to explain the acquisition of skill (or development of expertise). As each of these fives stages is traversed (novice, advanced beginner, competence, proficiency, and expertise), the individual exhibits qualitatively distinct features representative of that stage. The development and display of automaticity increases with each stage as the individual progresses from the initial rule-guided knowing that to the experience-based, intuitive knowing how. The medical student, when confronted with an apneic infant, may explicitly search a list of differential diagnoses and, in a calculated series of specific steps, stumble through the resuscitation. The anaesthetist, on the other hand, will simply “intuitively” resuscitate the infant. As expertise develops, features of individual cases are no longer considered independently but, rather, form a holistic pattern that is recognized effortlessly from years of experience and practice. As Dreyfus and Dreyfus29 describe, “when things are proceeding normally, experts don’t solve problems and don’t make decisions: they do what normally works.” A toddler trying to open a door, for example, is problem solving, whereas an adult doing the same is simply opening a door.
Dreyfus and Dreyfus29 depict their model of expertise as addressing the management of unstructured problem areas, which they define as “areas in which the goal, what information is relevant, and the effects of our decisions are unclear.” Though they state that interpretation plays a significant part in expert judgment, they predominantly view judgment as nonconscious and automatic. Their model specifically celebrates the nonanalytic nature of human expertise (the title of their book is Mind over Machine: The Power of Human Intuition …29) and focuses almost exclusively on the automatic mode of processing that experts are “normally” using, even in these unstructured problem areas.
What is less clear in their model is what happens when things do not proceed normally; what does effortful processing look like in the expert? They acknowledge that, at times, experts use critical thinking and reflect on the situation at hand when “time permits and outcomes are crucial.” However, this reflection seems to be more a monitoring or checking process of automatic and intuitive responses to the “routine” (if unstructured) problems of an expert’s daily practice, and it has little to say about the expert’s reactions when this checking process detects anomalies or cause for concern. Thus, although Dreyfus and Dreyfus29 acknowledge that experts may not spend all their time in the automatic mode, they seem to equate expertise with increasing use of the automatic mode. As a result, they do not bring us substantially closer to an understanding of what an expert’s effortful problem solving looks like or what the transition between the automatic and the effortful might involve. Rather, they equate expertise with the ability to increasingly rely on automatic resources.
By contrast, Bereiter and Scardamalia32 make a clear distinction between experts and experienced nonexperts, to explicitly set apart individuals who do not limit themselves to this unreflective, automatic mode. In Bereiter and Scardamalia’s model, experienced nonexperts are technicians who perform well on routine problems by unreflectively and automatically applying standard theories and techniques. However, they will not display creativity in finding solutions to ill-defined or unusual situations, the problems for which the standard techniques will not work. Experts, on the other hand, explicitly identify the subtle complexities of situations and, by addressing and readdressing the problems of daily practice through an effortful process, develop a very deep understanding of the particular systems they are working with. The experienced nonexpert emergency physician, having treated hundreds of patients with abdominal pain and diarrhea, may be quick to make a diagnosis of gastroenteritis. By contrast, the expert emergency physician may recognize the inconsistencies with the presenting signs and symptoms and will feel uncomfortable with fitting the common diagnosis of gastroenteritis onto this problem. Instead, through thoughtful reflection and by addressing and readdressing her concerns, she will consider the alternative diagnosis of intestinal ischemia.
Thus, whereas Dreyfus and Dreyfus29 suggest that experts almost never require the use of reflective analytic tools in daily practice, Bereiter and Scardamalia32 seem to suggest that the true expert is constantly using these analytic resources. We might interpret Bereiter and Scardamalia’s notion of the expert process of reflection even during routine cases as a form of ongoing practice in the integration and coordination of nonanalytic, automatic resources and more effortful reflective processes. Thus, experts are well prepared to transition from moment to moment between heavier reliance on automatic and heavier reliance on effortful processes to complete the task at hand.
It is worth noting that Bereiter and Scardamalia32 suggest that individuals who are experts may become experienced nonexperts in the same domain. Circumstances such as burn-out, disillusionment, and complacency can cause experts to begin to act in less thoughtful, nonreflective ways. Doctors may, for example, cease to engage in thoughtful reflection of the complexities of clinical cases and begin to “process” patients. Again, therefore, expertise is not a state of never having to engage in an effortful manner with the problems of daily practice. Rather, for Bereiter and Scardamalia,32 expertise is achieved when one constantly and intentionally engages with one’s environment during the routines of daily practice. Failing to do so represents a loss of expert status.
A third model for understanding the nature of expertise is represented in the work of Donald Schön,34,35 which is based on the premise that practitioners require practical knowledge to deal with the uncertainties of everyday practice, and that this practical knowledge cannot be taught in the classroom. To gain an understanding of what this practical knowledge looks like in action, Schön34,35 observed trainees and experts interacting together during the course of their working day, both in practice and in educational settings. Observing many different professions, he was able to find between them a common thread that he used to explain how experts think in action. He coined the terms knowing-in-action, reflection-in-action, and reflection-on-action to describe the thought processes a professional engages in during the challenges of daily practice.
Knowing-in-action refers to the “know-how” a professional displays when carrying out daily routines of practice. This knowing-in-action is generally routinized and enacted without reflection, using nonanalytic resources and knowledge that is “built in” from years of practice and experience. Examples may include a surgeon tying a knot or a dermatologist immediately recognizing a case of contact dermatitis. Detailed descriptions of the pathways the experts use to arrive at a particular solution often cannot be accurately reconstructed. Reflection-in-action, by contrast, describes an ability to improvise on the spot to unexpected events or surprises. It requires “thinking on our feet.” A jazz band improvising and reacting to each others’ improvisations,35 or a surgeon reacting to sudden bleeding, are examples of this in practice. Reflection-on-action completes the reflective cycle and describes an effortful process performed some time after an event that cannot influence the outcome of that particular event. It is done in an attempt to make sense of previous situations of uncertainty or uniqueness, either out of a curiosity or an effort to prepare for future cases. An unsuccessful resuscitation attempt of a ward patient, a death of a trauma victim, or an unexpected clinical outcome may lead to reflection-on-action, either formally in a debriefing session, or informally as the physician drives home and considers what took place.
One of Schön’s34,35 important contributions to our thinking on expertise was his insight into the different types of problems that professionals face in their daily practice and the effect these problems have on the expert’s use of automatic resources and effortful processes. Routine problems, according to Schön, are often dealt with by way of knowing-in-action where the deployment of automatic resources is sufficient. However, as Schön argues, problems of professional practice do not always present themselves as nice, neat packages that can be solved with standard theories or techniques. Many problems are messy, uncertain, ill defined, and ethically challenged, occupying what Schön referred to as the indeterminate zones of practice. To deal with these types of nonroutine, ill-defined problems adequately, the professional must effortfully attend to the situation and understand the intricacies and complexities of the particular case. Schön calls this practice the naming and framing of the problem. That is, the ill-defined problem must first be identified, or named, as a problem. Subsequently the problem must be situated in the larger picture, or framed, to address it effectively. A surgeon may fail to notice on the preoperative computed tomography scans that there is an abnormal vessel that will be of utmost relevance to the procedure. He or she fails to name the problem. On the other hand, the surgeon may recognize the abnormal vessels but fail to appreciate the impact it will have on the operation. He or she fails to frame the problem adequately. Or, the surgeon may recognize the problem and its relevance and, therefore, accurately name and frame the problem. It is this process of converting a messy, ill-defined problem into a well-formed problem that Schön identifies as central to the art of practice.
Interestingly, Schön34,35 recognized that problems were not stable with regard to their status as routine or nonroutine. In fact, he explicitly described the phenomenon whereby a problem may start as routine, but during the expert’s engagement with the problem, it evolves into a nonroutine, ill-defined problem. In doing so, he also provided a personal, anecdotal description of the transitional process of moving from knowing-in-action to reflection-in-action. This process involved stages of initially smooth functioning, followed by increasing struggles with the activity (possibly, he suggests, without explicit awareness of the struggle), followed by explicit awareness of the struggle and a transition into the explicit, effortful problem-solving process of naming and framing. We might speculate, therefore, that for Schön, expertise involves not the predominant use of automatic, nonanalytic resources or the predominant use of effortful, analytic processes, but that it depends critically on the ability to transition appropriately and effectively from heavier reliance on one set of resources to heavier reliance on the other. Crucially, clinicians may miss this step and proceed to manage the unnamed (and now missed) problem automatically.
Within the expertise literatures, emphasis is placed on the management of these ill-defined,32 unstructured,29 and indeterminate zones of practice.35 Each body of literature acknowledges that differences between various levels of expertise can most likely be accounted for in the way the experts manage these problems. How the expert responds to these situations not only contributes significantly to the outcome of the situation but also determines their level of expertise. If we can begin to understand the factors involved during this process—what the individual pays attention to and why, how the information obtained affects his understanding of the situation, and how he uses this information to make predictions about what will happen in the future—we may begin to obtain a greater understanding of the factors that influence expert judgment.
Attention and Effort
If we are to seriously consider a model of expert judgment that highlights the effective and timely transition from automatic, nonanalytic resources to effortful, analytic processing, it is important to understand the mechanisms by which such a transition might be triggered. Theories of attention focus on how individuals effectively use their cognitive resources to select the information to which they should attend, to process that information, and to act in response to it. To this end, the literature on attention and effort provides an important additional framework for understanding the performance of experts.
It is a basic premise in cognitive psychology that the human cognitive “space” available for mental activity is limited.27,36,37 Different mental activities place different demands on this limited capacity; easy tasks require little capacity, and difficult tasks require more capacity.27 Research has shown that our limited cognitive resources can be divided between simultaneous activities with considerable freedom.37 However, each additional task requires additional cognitive resources. If there is enough cognitive capacity available, the additional task can be accommodated, and divided attention is successful. If the resources are unavailable and cannot be recruited from other activities, performance falters.27,38
Kahneman,27 in his book Attention and Effort, points out that paying attention itself is an effortful activity. He suggests, in fact, that paying attention, exerting effort, and investing capacity are terms that can be used synonymously. It is not possible to pay attention without exerting effort and without using available cognitive capacity. Yet, at any one time, there are numerous environmental stimuli or information inputs to which one could potentially attend. To be attended to, each input must be “activated” through the use of additional attentional resource from the limited capacity.27 Attending to multiple stimuli simultaneously requires divided attention, and the extent to which a given stimulus can be attended to depends on attentional resources being available.39 By necessity, therefore, humans must limit what they attend to, and how attention is allocated becomes an important consideration in understanding how experts interact with their environment.
According to the cognitive capacity model of attention and effort, there are several factors that control the allocation of resources. The first factor that affects allocation of attention involves the rules of involuntary attention.40 That is, there are certain stimuli that are not expected but, nonetheless, grab our attention because of their salient features. Some of these features are fairly hardwired, such as fast moving or novel stimuli,41 whereas others are acquired by association over time, such as our name being mentioned in a peripheral conversation.42,43 A second factor that determines allocation of attention is anticipation of the stimulus.44,45 If, for example, we are asked to listen to the voices in our right ear, we are more likely to hear the voices in our right ear than the voices in our left.46 Similarly, if we are asked to look for a fracture in an x-ray, our chances of identifying it are increased. Through anticipation, we reduce the effort required for adequate identification of relevant features in the environment. As a third factor, there is an evaluation system such that incoming, simultaneous demands on capacity are assessed and are either given the necessary resources or not.27 When two stimuli arrive simultaneously, the stimulus that is judged as more important by the evaluation system gets priority. Attention to stimuli that are deemed less important or unimportant can be effectively depressed, allowing us to ignore many irrelevant items in the environment.27
Finally, at a more generic level, it is worth noting that alterations in arousal can affect the allocation of resources.27 For example, arousal levels alter the absolute capacity limit in the form of an inverted-U-shaped function. Increased arousal can increase cognitive capacity, but too much arousal can be problematic. Thus, very low or very high arousal levels may impede performance by decreasing the absolute cognitive capacity.47,48 Arousal not only affects the absolute amount of cognitive resource available, it also affects the effectiveness with which it is allocated.49 For example, in more stressful contexts, attention becomes highly focused on tasks we identify as being of dominant importance. Peripheral (or, more accurately, less dominant) stimuli are not attended to—referred to in the stress literature as tunnel vision.44,48 In addition, our ability to differentiate between relevant cues and irrelevant cues is impaired, often resulting in a perseveration of attention on a narrow, sometimes inappropriate set of stimuli.48 Therefore, in states of high arousal, our ability to allocate our attention effectively can deteriorate.49
The attention literature, therefore, provides an interesting perspective for understanding the development and enactment of expert performance. Both the expert and the experienced nonexpert have a limited cognitive capacity. Through experience, increasingly complicated and complex activities become automated, and, therefore, cognitive resources are freed up for engagement in other activities. In professional practice, these activities may include meta-cognitive monitoring, reflective activities, and attention to a variety of stimuli in the environment. Thus, the expert should be able not only to engage a set of effective automatic resources but, also, to use the resulting freed-up cognitive resources to maintain an attentional vigil on the environment, determine whether the automatic resources are functioning effectively in the particular circumstance, and slow down, engaging more effortful processes, when the situation requires it. In contrast, the experienced nonexpert may fail to reinvest the freed-up attentional resources into the situation at hand and, therefore, may not slow down when necessary.
Building on the attention literature, with its theoretical and conceptual frameworks for what commands our attention, is another body of literature—the situation awareness literature—grounded in the measurement of human performance within complex, real-world situations. Situation awareness most commonly is defined as the “perception of the elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future.”50 More simply, it is defined as a “constantly evolving picture of the state of the environment.”51 Because of its grounding within real-life complex environments, it is not a construct that exists for its own sake; rather, it provides the backdrop for human decision making and action selection within these environments.50,52 The airline industry, for example, has ascribed most incidents involving human error to inaccurate situation awareness.51 Often, the decision made in a critical situation was the correct decision given the parameters that the individual was considering, but the parameters themselves were wrong or incomplete because of failures of situation awareness. Poor situation awareness has also been considered a primary causal factor for errors in many diverse domains, including medical dispatch,53 medical diagnosis, and anesthesia.54
Several of the dominant models of situation awareness have their roots in early cognitive information-processing models, which presume a linear progression of information processing from data collection to data integration to interpretation and prediction. Jones and Endsley’s51 taxonomy of situation awareness errors, for example, classifies errors according to the level at which they occur. Level 1 errors result from a failure to perceive or from a misperception of elements in the environment, level 2 errors result from improper integration or comprehension of those elements into a coherent picture of the situation, and level 3 errors result from incorrect projection of future actions of the system. Where errors involve multiple levels, errors at the higher level are presumed to have resulted from errors at the lower level, and, therefore, the lowest level is presumed to be the “root” of the error. Within cognitive psychology, it is now largely recognized that human information processing is much less linear than this taxonomy would imply. Perception of elements in the environment, for example, can depend very much on what information is being sought, which can be affected by what one anticipates will happen in the future.55–57 As a dramatic example of inattentional blindness from the cognitive psychology literature, Simons56 demonstrated that when asked to watch a video and count how many times a basketball was thrown between players standing in a circle, half the subjects failed to notice that a black gorilla walked directly through the circle of players. Our perception of stimuli in any environment very much depends on what we regard to be stimuli and what we anticipate the relevant stimuli to be.55 As Endsley50 suggests, “people are not helpless recipients of data from the environment but are active seekers of data in light of their goals.” It is likely that a combination of bottom-up processes (driven by environmental stimuli) and top-down processes (guided by the operational goals) run simultaneously during the attainment of situation awareness within dynamic environments.58,59
The development and attainment of situation awareness in complex environments can be likened to the practice of naming and framing in the indeterminate zones of practice, introduced by Schön.35 The accurate perception of a problem that coincides with the accurate framing of the problem establishes accurate situation awareness; this, in turn, will lead to correct decision making, action selection, and ultimately satisfactory patient outcomes. The physician who will do well in the muddy, indeterminate zones of practice will be the physician who can attain accurate situation awareness. Further, the factors that have been demonstrated to affect situation awareness are mirrored well in the literature on attention and effort. Thus, the construct of situation awareness may be the cross-road where many relevant literatures—attention, effort, automaticity, and professional expertise, as well as issues related to human factors—intersect. The research questions being asked in the situation awareness literature may, therefore, provide interesting insights into the notion of expert judgment as a process of slowing down appropriately. Situation awareness not only needs to be attained, but maintained, involving constant attention to the pertinent cues of the environment, with an evolving understanding of what that means in light of the goals and objectives, and a prediction of where this course will lead to in the future. At any moment in time, attention needs to be allocated to monitoring the environment for unexpected and unanticipated cues, as well as for assessing results of actions already taken. Thus, questions of how situation awareness develops, how it functions, and where and why it fails may be critical to our understanding of expert performance in daily practice.60
There is difficulty in defining precisely what expertise means, because it comes with many “social and evaluative connotations.”32 From everyday experiences and interactions with people labeled experts, we have all developed our own ideas, perceptions, and prejudices of what expertise or what an expert means to us. An expert is often thought of simply as having achieved a certain stature—a person who performs at an elite level, a person who holds a particular position, a person from a particular occupation, a person with experience in a particular field, a person with a certain degree, or a person who is a specialist or a subspecialist. However, thinking of expertise in this way implies that it can be achieved through a series of steps or hurdles and that, once achieved, it is always attained. However, as Bereiter and Scardamalia32 argue, expertise is not an inevitable consequence of experience, and it is not synonymous with stature or titles. It would seem more accurate, sensible, and useful to consider expertise not from an achievement perspective but, rather, from a process perspective—when one is behaving in an expert manner, what differentiates that behavior from the one who is behaving in a nonexpert manner. The terms expert or expertise in this paper, therefore, refer to this process of expertise, and they are not restricted by questions related to achievement or job-related definitions of what or who is an expert. Although there are many aspects to this expert process—meeting new challenges, keeping on top of the knowledge base, evolving and transitioning within the field—there is one vital aspect we believe is essential to expert judgment: slowing down when you should. Considering expert judgment from this perspective allows fluctuations in individual performances and provides an explanation of why a resident may behave in an expert manner, exerting expert judgment—or slowing down when he or she should—whereas the staff physician may not always do so.
Together, the literatures on expertise, attention and effort, and situation awareness provide a valuable context for considering the process of expertise in daily practice and, in particular, the process of expert judgment. Although the term judgment has been used rather loosely in the past within the medical literature, we would advocate that it be reserved to describe physician performance when confronted by the indeterminate zones of practice. That is, we would propose that expert judgment be considered as an expert’s ability to respond effectively in the moment to the limits of his or her automatic resources and to transition appropriately to a greater reliance on effortful processes when needed. With adequate judgment, the expert will slow down when appropriate and take the time to ensure that the muddy problems of practice will be correctly named and framed.
If we consider expert judgment in this light, different research questions arise. Rather than concentrating on what is in the head of the expert, we would instead focus on how the expert coordinates these analytic and nonanalytic resources. If expert physicians spend most of their time applying automatic resources to routine “problems,” how do they recognize the unusual, more ill-defined areas that require a transition to the effortful processes? What are the cues from the environment that initiate reflection-in-action for the physician? What does this naming and framing process look like? Exploring these questions may provide a valuable first step towards achieving the ultimate goal of ensuring and enhancing physician performance.
As one example of a potential program of research, we might examine the meta-cognitive aspects of this form of expert judgment. We implied earlier that situation awareness is likely made possible, or at least enhanced, through the freeing of cognitive resources as automaticity is developed. But how effortful and consciously directed is the process of situation awareness, and how “reflective” is the decision to slow down?61 The transition from the automatic to the effortful, by definition, starts while the individual is within the automatic mode. Thus, the expert may be struggling or slowing down for some time, using additional cognitive resources to do so, before becoming aware of the fact that he or she is struggling. For example, a surgeon may be working in a relatively automatic mode, using freed-up cognitive resources for other activities such as monitoring the situation, chatting with a resident, and listening to music. At some point, however, the case may increase in complexity. The surgeon may start to struggle with exposing a vessel that is normally obvious, using more cognitive resources. During this process, she may disengage from the conversation, without being aware that he or she is doing so. This may continue for some time before it reaches awareness, at which point she may ask that the music be turned down so that she can focus on the task at hand. Such a scenario would imply that the surgeon did not know when to slow down; she simply slowed down when it was necessary. Thus, it becomes a matter of future research to determine how much awareness is involved in situation awareness, and how reflective reflection-in-action truly is.
To answer such questions, we must consider which methodologies to use. Schön34,35 conducted fieldwork of the experts in their workplace. He observed experts interacting with students during the course of their normal working activities. The teaching provided a valuable tool for identifying the indeterminate zones of practice. The teacher and student engaged in a verbal think-aloud session while they worked through the issues. By contrast, researchers studying situation awareness, with its roots in the aviation industry, use simulators to mimic real-life performances. With an ability to manipulate the environment, experts observe and monitor pilots during simulated flight sessions.62 The flights can be intermittently interrupted to seek out what the pilot is paying attention to and why, as well as to obtain an understanding of the pilot’s level of situation awareness.
What would such a research program in the medical field look like? It is difficult (with the exception of certain specialties such as anaesthesia63) to simulate real- life problems in the laboratory. As Schön34 describes, the interest is not in discovering what the expert does with well-packaged problems, but rather to observe what the expert does in the muddy, ill-defined zones of practice. Such work might best be accomplished with ethnographic observational studies of clinicians in their workplace to examine the “in vivo” recruitment of additional cognitive resources and reflection-in-action.63,64 Another possibility would be to make use of think-aloud protocols.3,30,65 Although this method has its limitations, such an approach may be useful for highlighting some of the ill-defined areas that occur in the natural setting and how the expert thinks about them. Conducting these in a teaching environment could provide a natural think-aloud session as the teacher and student discuss the issues at hand.6,66 As a third, complementary approach, interviews with expert teachers could explore their ideas of whether they are able to detect students who lack this judgment and compare them with those who demonstrate it.67 Physicians who are experienced with training residents should be a rich data source for exploring and understanding what this ability to slow down (or lack thereof) looks like, and what causes the expert to become nervous when supervising trainees. Finally, more controlled manipulation of stimuli such as videotapes of student performances might provide additional information through an exploration of the consistency amongst experts at what causes this nervousness.
By pursuing such questions and methodologies, we may be able to develop a deeper understanding of this construction of expert judgment for health care professionals. It may also provide us with the common language or taxonomy necessary to disseminate findings to the medical community at large and to build on with further research. And, finally, it may assist us with reorienting our research and efforts in medical education toward a better goal, one that is more closely aligned with Bereiter and Scardamalia’s32 notion of expertise. That is, the goal of medical education research should not be how to turn medical students into physicians more quickly or with less effort; instead, it should be to ensure that medical students develop into experts rather than experienced nonexperts. If we train with this as our goal, we will provide physicians with the tools to pursue their jobs as experts: “addressing and readdressing, with cumulative skill and wisdom, the constitutive problems of the job, rather than reducing the dimensions of the job to what one is already accustomed to doing.”32 To ensure this goal, we must understand better how the automatic and the effortful are coordinated. Until we are able to understand this form of expert judgment, we will not be able to formally teach it, assess it, or provide any remediation for those who are deficient in it.
Carol-anne Moulton was supported through a Fellowship in Surgical Education from the Royal Australasian College of Surgeons. Glenn Regehr is supported as the Richard and Elizabeth Currie Chair in Health Professions Education Research. Maria Mylopoulos is supported through a CHSRF/CIHR postdoctoral fellowship award.
1Elstein AS, Shulman LS, Sprafka SA. Medical Problem Solving. Cambridge, Mass: Harvard University Press; 1978.
2Harasym P, Baumber J, Bryant H, et al. An evaluation of the clinical problem-solving process using a simulation technique. Med Educ. 1980;14:381–386.
3Neufeld VR, Norman GR, Feightner JW, et al. Clinical problem-solving by medical students: a cross-sectional and longitudinal analysis. Med Educ. 1981;15:315–322.
4Norman GR, Feightner JW, Tugwell P, et al. The generalizability of measures of clinical problem-solving. Proc Annu Conf Res Med Educ. 1983;22:110–114.
5Elstein AS, Ravitch MM, Swanson DB, et al. Symposium: alternative approaches to research on clinical reasoning. Annu Conf Res Med Educ 1980;325–333.
6Barrows HS, Norman GR, Neufeld VR, et al. The clinical reasoning of randomly selected physicians in general medical practice. Clin Invest Med. 1982;5:49–55.
7Patel VL, Groen GJ. Knowledge based solutions strategies in medical reasoning. Cogn Sci. 1986;10:91–116.
8Elstein AS. Clinical judgment: psychological research and medical practice. Science. 1976;194:696–700.
9Redelmeier DA, Ferris LE, Tu JV, et al. Problems for clinical judgement: introducing cognitive psychology as one more basic science. CMAJ. 2001;164:358–360.
10Kushniruk A, Patel V, Fleiszer D. Analysis of medical decision making: a cognitive perspective on medical informatics. Proc Annu Symp Comput Appl Med Care. 1995;193–197.
11Gorry GA, Kassirer JP, Fineberg HV, et al. Symposium: the role of decision analysis in medical education and practice. Annu Conf Res Med Educ. 1979;18:427–437.
12Schull MJ, Ferris LE, Tu JV, et al. Problems for clinical judgement: 3. Thinking clearly in an emergency. CMAJ. 2001;164:1170–1175.
13Koch FT, Speers AT. It is time to move from the nursing process to critical thinking. AORN J. 1997;66:318–320.
14Dobrzykowski TM. Teaching strategies to promote critical thinking skills in nursing staff. J Contin Educ Nurs. 1994;25:272–276.
15Bereiter C. Situated cognition and how to overcome it. In: Kirshner D, Whitson JA, eds. Situated Cognition: Social, Semiotic, and Psychological Perspectives. Hillsdale, NJ: Erlbaum; 1997:281–300.
16Schmidt HG, Norman GR, Boshuizen HP. A cognitive perspective on medical expertise: theory and implication. Acad Med. 1990;65:611–621.
17Allen SW, Brooks LR, Norman GR, et al. Effect of prior examples on rule-based diagnostic performance. Res Med Educ. 1988;27:9–14.
18Norman GR, Rosenthal D, Brooks LR, et al. The development of expertise in dermatology. Arch Dermatol. 1989;125:1063–1068.
19Hatala RA, Norman GR, Brooks LR. The effect of clinical history on physicians’ ECG interpretation skills. Acad Med. 1996;71(10 suppl):S68–S70.
20Custers EJ, Regehr G, Norman GR. Mental representations of medical diagnostic knowledge: a review. Acad Med. 1996;71(10 suppl):S55–S61.
21Lemieux M, Bordage G. Propositional versus structural semantic analyses of medical diagnostic thinking. Cogn Sci. 1992;16:185–204.
22Eva KW, Norman GR. Heuristics and biases—a biased perspective on clinical reasoning. Med Educ. 2005;39:870–872.
23McGaghie WC, McCrimmon DR, Mitchell G, et al. Quantitative concept mapping in pulmonary physiology: comparison of student and faculty knowledge structures. Adv Physiol Educ. 2000;23:72–81.
24Chase WG, Simon HA. Perception in chess. Cogn Psychol. 1973;4:55–81.
25Rikers RM, Loyens SM, Schmidt HG. The role of encapsulated knowledge in clinical case representations of medical students and family doctors. Med Educ. 2004;38:1035–1043.
26Wiley J. Expertise as mental set: the effects of domain knowledge in creative problem solving. Mem Cognit. 1998;26:716–730.
27Kahneman D. Attention and Effort. Englewood Cliffs, NJ: Prentice-Hall Inc; 1973.
28Logan GD. Automaticity, resources, and memory: theoretical controversies and practical implications. Hum Factors. 1988;30:583–598.
29Dreyfus HL, Dreyfus SE. Mind Over Machine. New York, NY: The Free Press; 1986.
30Ericsson K, Smith J, eds. Toward a General Theory of Expertise. Cambridge, UK: Press Syndicate of the University of Cambridge; 1991.
31Eva KW. The aging physician: changes in cognitive processing and their impact on medical practice. Acad Med. 2002;77(10 suppl):S1–S6.
32Bereiter C, Scardamalia M. The need to understand expertise. In: Surpassing Ourselves: An Enquiry into the Nature and Implications of Expertise. Chicago, Ill: Open Court; 1993:1–24.
33Eva KW. What every teacher needs to know about clinical reasoning. Med Educ. 2005;39:98–106.
34Schön DA. The Reflective Practitioner: How Professionals Think in Action. New York, NY: Basic Books; 1983.
35Schön DA. Educating the Reflective Practitioner: Toward a New Design for Teaching and Learning in the Professions. San Francisco, Calif: Jossey-Bass; 1987.
36Cowan N, Elliot EM, Saults JS, et al. On the capacity of attention: its estimation and its role in working memory and cognitive aptitudes. Cogn Psychol. 2005;51:42–100.
37Moray N. Where is capactiy limited? A survey and a model. Acta Psychol (Amst). 2007;27:84–92.
38Posner MI, Rossman E. Effect of size and location of informational transforms upon short-term retention. J Exp Psychol. 1965;70:496–505.
39Moray N, O’Brien T. Signal-detection theory applied to selective listenting. J Acoust Soc Am. 1967;42:765–772.
40Remington RW, Johnston JC, Yantis S. Involuntary attentional capture by abrupt onsets. Percept Psychophys. 1992;51:279–290.
41Barcelo F, Escera C, Corral MJ, et al. Task switching and novelty processing activate a common neural network for cognitive control. J Cogn Neurosci. 2006;18:1734–1748.
42Corbetta M, Shulman GL. Control of goal-directed and stimulus-driven attention in the brain. Nat Rev Neurosci. 2002;3:201–215.
43Neisser U. Cognition and reality— principles and implications of cognitive psychology. San Francisco: WH Freeman; 1976.
44Simons DJ. Attentional capture and inattentional blindness. Trends Cogn Sci. 2000;4:147–155.
45Leblanc VR, Norman GR, Brooks LR. Effect of a diagnostic suggestion on diagnostic accuracy and identification of clinical features. Acad Med. 2001;76(10 suppl):S18–S20.
46Moray N. Attention in dichotic listenting: affective cues and the influence of instructions. Q J Exp Psychol. 1959;11:56–60.
47Yerkes RM, Dodson JD. The relation of strength of stimulus to rapidity of habit-formation. J Comp Neurol Psychol. 1908;18:459–482.
48Johnston JH, Driskell JE, Salas E. Vigilant and hypervigilant decision making. J Appl Psychol. 1997;82:614–622.
49Easterbrook JA. The effect of emotion on cue utilization and the organization of behaviour. Psychol Rev. 1959;66:183–201.
50Endsley MR. Toward a theory of situation awareness in dynamic systems. Hum Factors. 1995;37:32–64.
51Jones DG, Endsley MR. Sources of situation awareness errors in aviation. Aviat Space Environ Med. 1996;67:507–512.
52Klein GA, Calderwood R, Clinton-Cirocco A. Rapid decision making on the fire ground. Human Factors and Ergonomics Society. In: Proceedings of the Human Factors Society 30th Annual Meeting; September 29–October 3, 1986; Dayton, OH:576–580.
53Blandford A, Wong BLW. Situation awareness in emergency medical dispatch. Int J Hum Comput Stud. 2004;61:421–452.
54Gaba DM, Howard SK, Small SD. Situation awareness in anestheiology. Hum Factors. 1995;37:20–31.
55Treisman A. How the deployment of attention determines what we see. Vis Cogn. 2006;14:411–443.
56Simons DJ. Attentional capture and inattentional blindness. Trends Cogn Sci. 1999;4:147–155.
57Leblanc VR, Brooks LR, Norman GR. Believing is seeing: the influence of a diagnostic hypothesis on the interpretation of clinical features. Acad Med. 2002;77(10 suppl):S67–S69.
58Folk CL, Remington R. Top-down modulation of preattentive processing: testing the recovery account of contingent capture. Vis Cogn. 2006;14:445–465.
59Casson RW. Schema in cognitive anthropology. Annu Rev Anthropol. 1983;12:429–462.
60Hogan M, Pace D, Hopgood J, et al. Use of human patient simulation and the situation awareness global assessment technique in practical trauma skills assessment. J Trauma. 2006;61:1047–1052.
61Reason J. Absent-mindedness and cognitive control. In: Harris JE, Morris PE, eds. Everyday Memory, Action and Absent-Mindedness. London, UK: Academic; 1984:111–132.
62Direct measurement of situation awareness: validity and use of SA GAT. In: Endsley MR, Garland DJ, eds. Situation Awareness Analysis and Measurement. Mahwah, NJ: Lawrence Erlbaum Associates; 2000.
63Angus J. Ethnographic research applied to anesthesia. Can J Anaesth. 2005;52:899–903.
64Malterud K. Qualitative research: standards, challenges and guidelines. Lancet. 2001;358:483–488.
65de Groot A. Thought and Choice in Chess. The Hague: Mouton; 1978.
66Farrington-Darby T, Wilson JR. The nature of expertise: a review. Appl Ergon. 2006;37:17–32.
67Silverman D. Interpreting Qualitative Data. Methods for Analysing Talk, Text, and Interaction. London, UK: Sage Productions; 1994.