The Accreditation Council for Graduate Medical Education (ACGME) Outcome Project has shifted the focus of residents’ education to competency-based outcomes of learning experiences in six broad and diverse domains.1 The challenge of achieving meaningful teaching in each of these domains is surpassed only by the challenge of successfully assessing learner competence. Many educators have adopted the assessment rubric of the Dreyfus and Dreyfus model, a framework for skill acquisition that describes developmental stages beginning with novice and progressing through advanced beginner, competent, proficient, expert, and master.2 Originally applied to skills such as driving a car and playing chess, the model has been adopted by medical educators without a consensus on interpretation of the stages as they apply to the acquisition of clinical skills by physicians in training. Recognizing that no one model provides a panacea for clinical assessment of competence, the widespread adoption of the novice-to-master rubric calls for some thoughtful discussion and consensus around the use of those terms and the others associated with the model. Our purpose in writing this article is to establish a framework for the application of the Dreyfus model of skill development to the assessment of students and residents as they learn to practice clinical medicine. We apply this model in the context of generally accepted truths about how one learns, and we combine it with examples from the field to illustrate the application of these truths and the model in clinical settings. We also suggest strategies for teaching and learning at each stage.
Integration of Accepted Educational Truths About Learning and the Dreyfus Model
Eva3 describes clinical reasoning as being dependent on two processes: (1) the analytic method, and (2) the nonanalytic method, or pattern-based recognition. Integration of these processes provides a framework for practicing the science and the art of medicine. The analytic method refers to the hypothetico-deductive approach to problem solving characteristic of the scientific method, whereas pattern-based recognition relies on the learner’s ability to realize relationships between a repertoire of context-specific past clinical experiences (illness scripts) and the current clinical situation. The importance of the content and context (setting and circumstance) as well as personal influences (beliefs and attitudes) in creating these relationships makes each pattern unique to the individual.
Learners at all stages unconsciously use both forms of reasoning in clinical practice; more experienced learners have the advantage of a more extensive library of illness scripts and accordingly rely more heavily on pattern recognition unless presented with a complex case for which there has been no prior experience. A seminal article by Schmidt et al4 that speaks to the stages of clinical reasoning further illustrates the development and integration of these cognitive processes. During Stage 1, learners develop causal networks, akin to the development of a “concept map” in which learners create a written two-dimensional framework of concepts around a given topic.5 The relationship of signs and symptoms of disease to pathophysiology is an example of a causal network. The process involves identifying the concepts starting with most general at the top of the hierarchy and progressing to most specific ones, connecting related concepts by arrows that describe a proposition or statement to define the relationship, with some concepts extrapolated to specific examples. The ability to “cross-link,” or draw arrows between, separate concepts and label the arrows with a phrase that describes the relationship between the domains is a developmental task. Preliminary work in this area shows that the complexity of the maps, using specific scoring rubrics, increases when senior residents are compared with junior residents.5
The hallmark of Stage 2 is the learner’s ability to mentally organize these elaborate networks and synthesize them into a succinct, unified explanation of the problem, based on knowledge of the pathophysiology and clinical signs and symptoms of illness. In the clinical setting this is the equivalent of the one-or-two-sentence summary of the case in abstract terms, which educators refer to as the “problem representation.”6,7 The work of Bordage and Lemieux8 on semantic qualifiers is applicable here. Creating semantic qualifiers requires a mental abstraction in which the learner reframes specific information about, or attributes of, the case into more general categories. It is the exposure to patients and practice in solving clinical problems that allows this higher-order thinking to occur.
Stage 3 is characterized by the emergence of illness scripts. As students gain further clinical experience by repeatedly encountering patients with the same or similar problems, they begin moving from a causal representation of the clinical problem based on pathophysiology (which requires hypothetico–deductive reasoning) to one based on lists of diagnostic categories that explain the presenting signs and symptoms. These illness scripts represent characteristic features of clinical diagnoses and have a predictable structure of predisposing conditions, pathophysiologic insult, and clinical consequences.4 These scripts are unique to each physician and will ultimately include both relevant case-specific clinical information and unconsciously perceived information emanating from setting and circumstances. Whereas Stage 1 and Stage 2 are characterized by analytic reasoning, Stage 3 is characterized by nonanalytical reasoning (i.e., pattern recognition). Whereas this combination of analytic and nonanalytic reasoning allows for more efficient and accurate diagnosis, it should not be construed that pattern recognition is a higher-order cognitive process than analytic reasoning. Rather, it reflects the natural order of building clinical experience on baseline knowledge of medical science.
Stage 4 represents the storing of these illness scripts as individual “instance scripts.”4 In essence, when encountering a new patient, the physician unconsciously searches through a wealth of scripts and matches a prior illness script to the current patient. For common problems, it is relatively easy to retrieve a matching illness script. But the myriad of disease processes and clinical manifestations, particularly the less common and more complex disease processes, calls for relative, or inexact, matching and the adaptation of stored scripts to new, heretofore unseen clinical problems. Fraser and Greenhalgh9 describe the progression and outcome of this process as capability building. They note that “learning which builds capability takes place when individuals engage with an uncertain and unfamiliar context in a meaningful way. Capability enables one to work effectively in unfamiliar contexts.” Situations where the learning environment and the task are only moderately familiar, but not completely unfamiliar, are critical in enabling the learner to progress to Stage 4 reasoning.
There are at least five educational implications of Schmidt’s work.
- Basic science knowledge is an important prerequisite that allows the learner to form the initial causal networks between clinical presentations and basic pathophysiology.
- Clinical encounters representing a broad scope of patient problems are critical in moving the learner from the purely hypothetico–deductive reasoning used in developing the causal networks to the development of illness scripts.
- A repertoire of illness scripts allows for the development of nonanalytic reasoning, or pattern recognition, a skill used at all stages of learning but most characteristic of more seasoned practitioners.
- Learners require exposure to environments and clinical scenarios outside their comfort zone to stimulate the adaptive higher-level clinical reasoning that is critical for transforming the proficient practitioner into the expert practitioner.
- Meaningful evaluation of a learner requires longitudinal assessment of the developmental progression over time.
Case Studies: Clinical Skill Development Using the Dreyfus and Dreyfus Model
The Dreyfus and Dreyfus model of skill acquisition, originating in response to the recognition of limitations of artificial intelligence, emphasizes pattern recognition, intuition, and reflection as critical in the development of professional skills.10 Initial application of the model addressed skill development of fighter pilots and has been detailed in its application to driving a car and playing chess.11 Studies in nursing have extrapolated the application of the model to clinical medicine.12 With the advent of the ACGME Outcome Project, there has been a surge of interest in looking to the Dreyfus and Dreyfus model as a method of describing and, ultimately, assessing clinical competence of physicians as they progress from novice to master.1List 1 represents our interpretation of the model in the context of clinical medical practice. The case studies below are intended as “snapshots” along the continuum of learning and assessment. The stages (e.g., “novice”) are mere callouts in the evolution of that developmental continuum. The transition from stage to stage is a gradual one, and a learner may be at different stages for his or her different skill sets.
;)
List 1 Principles of the Dreyfus and Dreyfus Model of Skill Development Applied to the Development of a Physician’s Competence
Novice
S.L. is beginning her first clinical rotation of medical school, having already completed her introduction-to-clinical-medicine course. She interviews her patient, performing a history and physical examination using a set of rules or templates. Regardless of the chief complaint, S.L. methodically goes through each item on a generic template. There is no capacity to focus the information gathering on the basis of a likely differential diagnosis, because she does not yet comprehend the big picture. Each sign and symptom seems equally relevant. Using learned rules, she links the large data set of information gathered to her knowledge of the pathophysiology of disease. When preparing her written history and physical, the pertinent positives and negatives remain scattered throughout the review of systems rather than being discussed in the history of the present illness because of S.L.'s inability to filter relevant from irrelevant. S.L. still has limited ability to synthesize the myriad pieces of information into a unified summary.
Dreyfus and Dreyfus define the novice as one whose decision making is rule based. Integrating this model with Schmidt’s stages of clinical reasoning suggests that the rules in this case are derived from the knowledge and relationships of pathophsyiology to clinical signs and symptoms. The learner uses analytic reasoning and rules to create and test causal networks. Learners engage in analytical or hypothetico–deductive reasoning because knowledge of the causal relationships between signs and symptoms and the pathophysiology of disease predominates over the ability to call on illness scripts, the development of the latter being quite limited by lack of clinical exposure.
Implications for teaching and learning
The key teaching task is to recognize, or diagnose, the learners’ stage and then employ techniques to support the learners at that stage and facilitate their movement to the next stage. For the novice, basic science knowledge is important in providing the substrate for the analytic process of clinical reasoning. However, the integration of basic and clinical sciences through methods of teaching such as problem-based learning or introductory courses to clinical medicine in the first two years of medical school will facilitate the early building of simple illness scripts and the progression to “advanced beginner” in the Dreyfus and Dreyfus model. Medical school curricula that provide early introduction to clinical correlates and clinical medicine allow the learner to more effectively use both analytic and nonanalytic processes of clinical reasoning early in clinical practice. From this early stage, Bowen6 recommends that teachers help learners organize their clinical knowledge by using the following strategies of case-based integration of clinical and basic science: (1) pointing out meaningful diagnostic information in the history and physical examination, (2) eliminating irrelevant information, (3) highlighting discriminating features along with their relative importance to a diagnosis, and (4) encouraging learners to read about at least two diagnostic hypotheses at the same time so that they learn to compare and contrast similarities and discriminating features of diagnoses.
Advanced beginner
At the beginning of her intern year, when S.L. is taking a history and performing a physical examination, she begins to generate a differential diagnosis that drives the data gathering in a more focused direction. Because she is capable of filtering information and focusing on the relevant, she is able to formulate a unified summary of the case. When writing up her notes, she can abstract the pertinent positives and negatives from the review of systems and appropriately incorporate them into the history of the present illness.
According to Dreyfus and Dreyfus, the expansion of the repertoire of past patient care experiences provides the learner with some basis for filtering the relevant aspects of a clinical case from the wealth of information resulting from data-gathering activities. S.L.'s approach is still heavily dependent on analytical thinking, but she now has sufficient clinical experience to apply more appropriate rules or broaden the rules to apply them to new situations. Clinical experience also enables her to invoke nonanalytic clinical reasoning based on both a greater repertoire and more robust illness scripts that facilitate pattern recognition.
Implications for teaching and learning
Exposure of the learner to clinical cases is critical for building robust illness scripts. The strategy of exposure should be to work from the common to the uncommon so that it is easy for the early advanced beginner to match current patient encounters to previously created scripts. It is also helpful to focus the learner on Bordage and Lemieux’s8 semantic qualifiers. An example would be helping the learner to reframe a chief complaint of “pain for 24 hours prior to admission” to the more general category of “acute onset of pain.” This ability has been associated with a type of problem representation of greater diagnostic accuracy. Clinical teachers have the opportunity to help learners develop this skill by emphasizing the use of these qualifiers in both written and verbal case presentations. Clinician educators can also help students navigate this stage by encouraging them to formulate and verbalize their own differential diagnosis and treatment plan. Two models that have been described elsewhere, the Microskills Model13 and SNAPPS,14 are useful in encouraging students to work through this process.
The commonly used team structure is another adjunct to aiding students’ progress because this is the stage at which coaching becomes relevant. A good coach helps the learner become attentive to the meaningful pieces, or situational aspects, of a case. In fact, the “near peer” may be the best coach because the more experienced clinician often relies solely on pattern recognition rather than analytic reasoning, making it difficult for the clinician to understand where the learner is encountering difficulties and how to help overcome them.
Competent
S.L. is nearing the end of her residency training. Her clinical exposure over the course of training allows her to recognize many common patterns of illness on the basis of previous encounters. It also allows her to see the big picture, facilitating a more holistic approach to formulating assessments and plans. Given the increased number of patients and increased time with patients, she sees the consequences of her clinical decisions, which results in an emotional buy-in to learning. When S.L. encounters complex or uncommon problems that she does not recognize, she will methodically attempt to reason through each step of the case, sometimes successfully and sometimes getting stuck. She feels responsible for her decision-making process and is consciously aware of her own role in contributing to the clinical outcome.
Early in this stage, clinical reasoning is still more heavily weighted toward analytic hypothesis testing. While gaining in experience, the learner is operating in the context of a changing learning environment. For example, in the realm of patient care the learner experiences a greater breadth of patient problems and increased responsibility. As a result, for each new case for which there are no illness scripts, the learner must rely on analytic reasoning. At some point, learners typically become overwhelmed, realizing that there is not a rule for every clinical situation. In the example above, S.L. must make a decision and understand that there are risks in making that decision. At some point, her decision will likely result in an outcome that is less than optimal for the patient. This realization makes the learner anxious and triggers an emotional response resulting in true engagement with the patient, replacing the prior relationship of an observer with limited responsibility. When the learner makes a mistake at this stage, she goes beyond simple analysis to define better rules and reflects in a way that involves both cognition and emotion to improve future practice. The emotional engagement promotes a “deeper” learning that, in turn, facilitates building more robust illness scripts. The volume and diversity of patients seen in clinical practice significantly increases the number of scripts available for retrieval. This is where the balance tips in the direction of nonanalytical clinical reasoning favoring pattern recognition for common illnesses. The learner handles a greater volume and diversity of clinical encounters by matching new cases with stored illness scripts or “instance scripts,” a strategy that correlates with Schmidt’s Stage 4. Pattern recognition highlights the “big picture” and helps the learner to sort through a large data set, focus on the relevant, and prioritize that information to retrieve appropriate illness scripts. The combination of a greatly enhanced repertoire of illness scripts coupled with the emotional investment in patient outcomes will tip the balance, transitioning the learner to the proficient phase. This transition will also require the learner to demonstrate “capability,” defined by Fraser and Greenhalgh9 as “the ability to extrapolate from the known to the unknown clinical situation to handle unfamiliar clinical problems.”
Implications for teaching and learning
At this stage, clinical teachers need to balance supervision with autonomy in medical decision making and hold learners accountable for their decisions. If learners are “told” rather than “asked” what to do, they will never achieve the emotional buy-in that is necessary to assume appropriate responsibility for patient outcomes. In addition, it is critical that the learner see a breadth and depth of patient encounters to be able to construct and store in memory a robust repertoire of illness scripts. Both of these processes are necessary to tip the balance of clinical reasoning to pattern recognition. If this does not occur, the learner will be overwhelmed by the volume and complexity of patients. The learner will rely too heavily on analytic reasoning and be forced to struggle with each feature of an illness rather than assimilating the features into a “whole.” This is the learner that is typically a good intern, can take an accurate history and perform a complete physical examination, but is not a good senior resident because he or she is incapable of synthesizing the data gathered into a big-picture assessment. As a result, the learner is overwhelmed by any atypical or complicated patient and is, therefore, not capable of meeting expectations for efficiency. Although a learner might be expected to be at the competent stage on the basis of years of training, he or she may be only an advanced beginner. Care in identifying the learner’s developmental stage will enable the clinician educator to use teaching strategies suitable for the learner’s skill level.
Proficient
S.L. is a clinical instructor in her department. Very early in the course of data gathering during a patient encounter, she begins to match the constellation of findings with those encountered during past experience, and she quickly develops a differential diagnosis. This ability drives the data gathering in a way that is more effective and efficient. When listening to a student present a patient, she can see the patient though a different lens than the student can, based on a few key pieces from the history and physical. An example would be the patient who is much sicker than initial appearances might suggest to the more inexperienced learner. When the student presents the case, he or she does so in a way that reflects a stable patient. Early in the course of the presentation, S.L. has an intuitive sense from the constellation of findings that more immediate intervention is necessary. Seeing the patient confirms her suspicion. She methodically engages in a process of clinical reasoning to work through to finding the best intervention.
For proficient learners, intuition plays the predominant role in clinical reasoning. This practitioner is clear about goals and salient aspects of a situation without step-by-step filtering of individual data points. Situational discriminators and pattern recognition predominate over the use of rules. This learner has enough experience to recognize many clinical cases by unconsciously calling on matching illness scripts developed during previous patient encounters. The unconscious nature of the search for these scripts makes the reasoning appear intuitive. The advantage that the proficient learner has over the competent learner is the former’s comfort level in correcting or changing plans in response to an evolving clinical situation, and tolerance for ambiguity. Unlike the expert, the proficient learner does not necessarily have an intuitive sense for the appropriate response to every situation. Such a learner may have seen a number of patterns of illness that correspond to the diagnosis at hand but likely will not have had experience with the outcomes of the different interventions that are possible in each of these cases. In the instance of particularly complex cases, the proficient learner still needs to fall back on analytic reasoning to make a conscious decision about management plans.
Implications for teaching and learning
According to the Dreyfus and Dreyfus model, a learner at this stage needs to work alongside and be mentored by an expert. He or she must learn to trust his or her intuition for managing a clinical situation, so it is important that the learner not be constantly presented with complex unknowns that require tipping the balance back in favor of analytic clinical reasoning. At this stage in the model, some modification in thought that brings in the work of other cognitive researchers is warranted. Recently, Eva and Regehr15 and others16,17 have written about the critical self-regulation that defines experts. They know when to “slow down and look it up.” This calls for a more balanced look at the proficient learner, who needs to trust intuition in management when appropriate. Equally important, in transitioning to a higher level of expertise, is the capacity to know one’s limitations and automatically step back and call on additional resources when stretched beyond one’s capabilities.
Expert
After many years in practice, S.L. is an associate professor in her department. Her thoughts, feelings, and actions are all aligned into an intuitive approach to problem recognition and response. Her broad repertoire of robust illness scripts, based on clinical experience over time, allows her to act immediately for the majority of clinical encounters. Her favorite cases, however, are diagnostic dilemmas. As a true expert, she is mindful of her limits, and, when presented with a diagnostic dilemma, she will “slow down and look it up.”15,16
The expert’s broad experience allows reliance on pattern recognition except in cases that represent diagnostic dilemmas. On the basis of past experience and a robust set of illness scripts, the expert perceives “more subtle and refined discriminations” not apparent to those with lesser experience.18 The expert is also clever and can notice when the unexpected occurs or when the current pattern or constellation of findings does not quite align with any previously encountered pattern. The danger to the expert is that the ease of responding to the majority of clinical encounters can foster complacency, in which case the emotional involvement with each encounter dwindles.11 Bereiter and Scardemalia also express concern that this “automaticity” can negatively affect improvement in performance.19 If this occurs, the intense drive to continue to learn and improve will fade. Instead of transitioning from expert to master, in the Dreyfus and Dreyfus model, the learner becomes the “experienced nonexpert” described by Bereiter and Scardemalia.19 These authors make the critical distinction between an expert and an experienced nonexpert. The expert (or expert transitioning to master in the Dreyfus and Dreyfus model) takes the mental resources saved in applying pattern recognition to solve a problem and reinvests them in tackling the next layer of more complex problems. This habit of reinvesting mental resources into “progressive problem solving” that extends beyond one’s zone of comfort is the hallmark of the expert.
Implications for teaching and learning
The key here is to keep the expert challenged. The expert needs ongoing experience and exposure to interesting and complex cases to avoid complacency and to transcend beyond this level. Ideally, the expert should be apprenticed to a master who models the skills of the reflective practitioner and a commitment to lifelong learning.
Master
S.L. is now the clinician that everyone comes to with their problem cases. She recognizes subtle features of a current case as reminiscent of sporadic cases seen over the years of practice. She painstakingly revisits each of the cases in the hope of identifying a common thread or pathognomonic feature that will help her and others more quickly identify and treat this clinical problem or illness in the future. Her vision extends beyond individual practice; she contributes in the bigger context to improvements in the field. She has an intense internal drive to continue to learn and improve.
This stage is characterized by practical wisdom and the ability to effortlessly impart “pearls” to learners. The master has a broad and sensitive understanding of not only the “big picture” of the situation but the “bigger picture” of context and culture in which this situation occurs. The master’s deep level of commitment to the work triggers an automatic and ongoing concern for right and wrong decisions, which promotes emotional engagement in every encounter and acts as the intrinsic motivator for ongoing learning. The master is the practitioner who self-assesses and self-regulates and “reflects in, on and for action.”20,21
Implications for teaching and learning
By virtue of ongoing reflection on practice, the master is self-motivated to engage in lifelong learning and practice improvement. This can be likened to being habitually engaged in a plan–do–study–act cycle for both personal and professional development and practice improvement.22
Discussion
Despite efforts to identify and overcome the many sources of bias inherent in clinical performance assessment,23 the psychometric model on which current assessments are based may be inappropriate.24 Recently, Regehr et al25 demonstrated the limitations of anchored scales, even those incorporating the evaluator’s language. They found that using a standardized set of holistic, realistic vignettes improved discrimination of students’ performance. The use of descriptive assessment in the summative evaluation of medical students’ competence using the reporter, interpreter, manager, and educator (RIME) framework has also shown promise of improving reliability and validity while also providing a feasible and credible approach to formative evaluation.26 This approach combines a “portable vocabulary” (four terms), each with a widely accepted narrative description with which the evaluator can match their impressions gathered over time. The Dreyfus and Dreyfus model framework presented here may enable a similar approach to evaluation of learner competence that spans the medical education continuum. Many educators are using the model, but, to date, there has not been a consistent definition of how the “labels” are applied in assessing the professional formation of physicians, a necessary first step towards shared mental models. As with any model, there are inherent limitations, particularly around the definition of expert. In fact, the Dreyfus and Dreyfus definition of master is better aligned with what others have defined as the expert.15–17,19 Some authors have argued that expertise is an approach to problems that begins early in practice rather than a developmental phenomenon that occurs as a result of time and experience.17,19 Whether one ascribes to expertise as a developmental phenomenon or a more innate quality that is cultivated with time, Ericsson’s27 work in this area clearly emphasizes the expert’s critical need for “ongoing deliberate practice,” that is, “engaging in practice activities with the primary goal of improving some aspect of performance.”
One must also be clear that the six stages proposed by Dreyfus and Dreyfus are not discrete steps but increments along a continuum. The journey towards self-directed learning seems to be unique to an individual learner with different skills developing at different rates depending on content, context, and past experience.28,29 Likewise, the achievement of competence requires a need for more individualized time frames for learners to reach the desired outcomes of education and training.
Although our examples demonstrate application to patient care, the model may also be applied to the acquisition of other skills or competencies. The challenge in doing so arises when one deconstructs the competencies into such detailed elements that it becomes difficult to address developmental outcomes in a meaningful way, analogous to a statistical P value that has little biologic significance. For example, deconstructing the six ACGME competencies and assessing a learner’s competence in key elements is necessary but not sufficient for determining whether the learner is a “good doctor.” The latter requires the integration of all of these individual elements. Finding the right balance between deconstructing and reconstructing the competencies as well as the use of multiple models such as RIME, or Kegan’s model of identity development,30 in conjunction with Dreyfus and Dreyfus, provides the best strategy for assessing the developmental progress to becoming a good doctor. This more holistic approach to assessment of competence would address the concerns raised by ten Cate and Scheele’s31 article on bridging the gap between theory and clinical practice by reconstructing the competencies in the clinical assessment of the good doctor.
To pursue our interest in the development of clinical competence, we hope to eventually integrate the findings of the Medical School Objectives Project32 with the ACGME competencies1 and then apply the Dreyfus and Dreyfus model as well as other developmental models of assessment to provide a holistic approach to evaluating the development of competence across the educational continuum.
Acknowledgments
The authors acknowledge and thank Dr. Glenn Regehr for his careful read of the manuscript and his thoughtful editorial suggestions.
References
1 Accreditation Council for Graduate Medical Education. Outcome Project. Available at: (
http://www.acgme.org/Outcome). Accessed April 19, 2008.
2 Batalden P, Leach D, Swing S, Dreyfus H, Dreyfus S. General competencies and accreditation in graduate medical education. Health Aff (Millwood). 2002;21:103–111.
3 Eva K. What every teacher needs to know about clinical reasoning. Med Educ. 2004:39;98–106.
4 Schmidt HG, Norman GR, Boshuizen HPA. A cognitive perspective on medical expertise: Theory and implications. Acad Med. 1990:65;611–621.
5 West DC, Park JK, Pomeroy JR, Sandoval J. Concept mapping in medical education: A comparison of two scoring systems. Med Educ. 2002;36:820–826.
6 Bowen JL. Educational strategies to promote clinical diagnostic reasoning. N Engl J Med. 2006;355:2217–2225.
7 Nendaz MR, Bordage G. Promoting diagnostic problem representation. Med Educ. 2002;36:760–766.
8 Bordage G, Lemieux M. Semantic structures and diagnostic thinking of experts and novices. Acad Med. 1991;66:S70–S72.
9 Fraser S, Greenhalgh T. Complexity science: Coping with complexity: Educating for capability. BMJ. 2001;323:299–303.
10 Dreyfus HL, Dreyfus SE. Mind Over Machine. New York, NY: Free Press; 1988.
11 Dreyfus H. From novice to world discloser. Paper presented at the Accreditation Council for Graduate Medical Education Design Conference on the Learning Environment; September 9, 2006; Chicago, Ill.
12 Benner P. Using the Dreyfus model of skill acquisition to describe and interpret skill acquisition and clinical judgment in nursing practice and education. Bull Sci Technol Soc. 2004;24:188–199.
13 Neher JO, Gordon KC, Meyer B, Stevens N. A five-step “microskills” model of clinical teaching. J Am Board Fam Pract. 1992;5:419–424.
14 Wolpaw TM, Wolpaw DR, Papp KK. SNAPPS: A learner-centered model for outpatient education. Acad Med. 2003;78:893–898.
15 Eva K, Regehr G. Knowing when to look it up: New conception of self assessment ability. Acad Med. 2007;82(10 suppl):S81–S84.
16 Moulton CE, Regehr G, Mylopoulos M, MacRae HM. Slowing down when you should: A new model of expert judgment. Acad Med. 2007;82:S109–S116.
17 Myopoulos M, Regehr G. Cognitive metaphors of expertise and knowledge: Prospects and limitations for medical education. Med Educ. 2007;41:1159–1165.
18 Dreyfus HL. Overcoming the myth of the mental. Topoi. 2006;25:43–49.
19 Bereiter C, Scardamalia M. Surpassing Ourselves: An Inquiry Into the Nature and Implications of Expertise. La Salle, Ill: Open Court; 1993.
20 Schon D. Educating the Reflective Practitioner. San Francisco, Calif: Jossey- Bass, Inc.; 1987.
21 Plack M, Greenberg L. The reflective practitioner: Reaching for excellence in pediatrics. Pediatrics. 2005;116:1546–1552.
22 Berwick DM. A primer on leading the improvement of systems. BMJ. 1996;312:619–622.
23 Williams RG, Klamen DA, McGahie WC. Cognitive, social and environmental sources of bias in clinical performance ratings. Teach Learn Med. 2003;15:270–292.
24 Schuwirth LWT, van der Vlueten CPM. A plea for new psychometric models in educational assessment. Med Educ. 2006;40:296–300.
25 Regehr G, Bogo M, Regehr C, Power R. Can we build a better mousetrap? Improving the measures of practice performance in the field practicum. J Educ Soc Work. 2007;43:327–343.
26 Pangaro LN. Investing in a descriptive evaluation: A vision for the suture of assessment. Med Teach. 2000;22:478–481.
27 Ericsson KA. Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Acad Med. 2004;79(10 suppl):S70–S81.
28 Ten Cate O, Snell L, Mann K, Vermunt J. Orienting teaching toward the learning process. Acad Med. 2004;79:219–228.
29 Grow GO. Teaching learners to be self-directed. Adult Educ Q. 1991;41:125–149.
30 Forsythe G. Identity development in professional education. Acad Med. 2005;80(10 suppl):S112–S117.
31 ten Cate O, Scheele F. Competency-based postgraduate training: Can we bridge the gap between theory and clinical practice? Acad Med. 2007;82:542–547.
32 Association of American Medical Colleges. Learning Objectives for Medical Student Education. Available at: (
http://www.aamc.org/meded/msop/msop1.pdf). Accessed April 19, 2008.