Secondary Logo

Trends in Medical Education Research

Regehr, Glenn, PhD

Special Theme Article
Free

The medical education community is reflecting increasingly on the role and nature of research in the field. Useful sources of data to include in these reflections are a description of the topics in which we are investing our energies, an analysis of the extent to which there is a sense of progress on these topics, and an examination of the mechanisms by which any progress has been achieved. This article presents the results of a thematic review of the medical education research literature in four key journals since the turn of the 21st century. It describes four examples of areas in which the community appears to be investing its energies: curriculum and teaching issues, skills and attitudes relevant to the structure of the profession, individual characteristics of medical students, and the evaluation of students and residents. A discussion of the recent publications in these domains highlights a distinction between thematic categories of research, in which many members of the community are working on the same topic, and programmatic lines of research, in which members of the community are working together toward the shared goal of consensual understanding. The author suggests that community-level, programmatic lines of research are necessary to build knowledge and understanding of a domain and that, in the absence of such communal effort, the value of research is limited to the uncoordinated accrual of information.

Dr. Regehr is the Richard and Elizabeth Currie Chair in Health Professions Education Research, associate director, Wilson Center for Research in Education, and professor, Department of Surgery, Faculty of Medicine, University of Toronto, Ontario, Canada.

Correspondence should be addressed to Dr. Regehr, Wilson Center for Research in Education, 200 Elizabeth Street, Eaton South 1-565, Toronto, Ontario, Canada M5G2C4; e-mail: 〈g.regehr@utoronto.ca〉.

In recent years, the medical education community has been reflecting increasingly on the role of research in the field. Among the issues associated with this reflection is a concern that the field is not developing systematic or productive programs of research and, therefore, does not seem to be advancing on “big questions.” One of the reasons proposed by those who perceive this as a problem is the absence of research studies that are motivated and informed by useful theories. For example, Prideaux and Bligh1 have suggested that one consequence of not locating our work in a theoretical context has been the difficulty of aggregating findings into consistent themes. Making a slightly different argument, Colliver2 has suggested that the theories we are using in the field are too weak to be productive. Whether there is a lack of “advancement” in our field and whether the absence of “functional” theories is an important contributor to this phenomenon requires further reflection. One process for shedding additional light on this issue would be to look at the areas where we as a community have recently invested our research and development energies, to explore the extent to which these energies have led to a sense of “advancement” in the field, and to examine the mechanisms by which advancement has been achieved.

The first step in such a process is an examination of our literature to provide insight into the domains where we are investing our energies. This article, therefore, is an effort to call our attention, as a community, to the question of what we are doing. It is a preliminary effort to describe some of the areas of thematic research in our literature as they have manifested in the last several years, to explore how these thematic areas have faired in advancing the field, and to examine the extent to which theory has played a role.

Back to Top | Article Outline

Medical Education Research Themes and Trends

Many research articles have been generated in the field of medical education since the turn of the century. It would be impossible to cite or categorize them all, even if it were possible to find and read them all. Thus, this exercise focused on only four of the journals that are central to the medical education research enterprise: Academic Medicine, Advances in Health Sciences Education, Medical Education, and Teaching and Learning in Medicine. These journals were selected for a variety of reasons: they are explicitly dedicated to medical (or health professional) education questions, but are generalist within this mandate (i.e., they focus on all aspects of the medical education enterprise rather than specializing in an area such as continuing education alone); they are research focused; and the editors of each of these four journals have been active and vocal in their pursuit of promoting and shaping the face of medical education research.3–9 From my review of the research articles in these journals since 2000, many themes and trends emerged. The themes highlighted in the following sections are some of the most populated, suggesting that these are key domains in which the community has been investing its energies.

Back to Top | Article Outline

Applied Curriculum and Teaching Issues

Perhaps not surprising for an applied domain such as medical education, a substantial number of research articles are dedicated to discussing and exploring applied issues of curriculum and teaching. The questions in this theme address curriculum issues across a wide continuum, from large-scale curriculum evaluation, through the description and evaluation of specific aspects of curricula, to implementations of teaching strategies and techniques, and finally to descriptions and evaluations of highly specific content-based courses or workshops in the context of curricula.

At the level of large-scale curriculum evaluation, for example, there are continuing questions about the relative strengths and weaknesses of problem-based learning (PBL), with continuing efforts to understand how it influences learning and under what conditions it might be successful. As just a few examples, Curet and Mennin10 addressed the highly practical question of the influence of using long-term versus short-term tutors in the PBL tutorial setting, and De Grave et al.11 explored the impact of critical incidents in PBL tutorial groups, how students interpreted these events, and what they learned as a result. At a more theoretical level, van den Hurk et al.12 developed and tested a causal model for learning in a PBL curriculum as part of a research program designed to increase our understanding of the critical factors in making PBL successful. However, PBL is not the only target of large-scale curriculum evaluation studies. Other important questions include the integration of preclinical and clinical training in medicine (with a frequent focus on the integration of basic sciences13–15) and the shift from bedside teaching to ambulatory or community teaching settings in clinical training.16,17 Although the exact target of these curriculum-level studies varies, an important subset of the studies has attempted not only to provide examples of how a curriculum can be implemented, but also to inform our understanding of how a curriculum can be shaped by the interrelation among theoretical and practical factors. In doing so, they have informed both theory and practice. In an excellent illustration of theory informing practice and practice informing theory, Miflin et al.18 demonstrated and described how a facile interpretation and inappropriate implementation of the concept of self-directed learning can undermine the development of an effective PBL curriculum.

Other studies focus not on the broad curriculum as a whole, but on the integration and implementation of curriculum-level educational strategies or programs. One of the largest and fastest-growing issues at this level of curriculum design and evaluation is the integration of simulation into the curriculum as a mechanism for teaching without direct contact with patients. The types of simulation being addressed range widely, including standardized patients,19 bench models,20 and virtual reality simulators.21 However, all these studies are focused on understanding the relative strengths and weaknesses of simulation and its place in the larger curriculum. A second, similarly active topic in the domain of curriculum-level education strategies is the use of technology as a vehicle for curriculum delivery. Broadly, these technology-based instruction models involve the use of synchronous visual mechanisms such as videoconferencing,22 asynchronous Web-mediated collaborative learning environments,23 or individual-access CD-ROM or Web-based databases of educational information.24 But, again, an important subset of these studies has attempted to address this issue by asking not merely whether technology “works” as an educational medium, but also how technology can be integrated effectively with the larger goals and structures of the curriculum.

More specific but still theoretically driven programs of research in this area address more localized content delivery questions. Many of these content delivery questions are tied to specific skills to be learned but often have broader implications for the theory of learning and the practice of education. In the tradition of cognitive psychology, researchers have addressed issues such as the relative value of mixed versus blocked practice25 or the relative value of using complete clinical vignettes versus chief complaints26 when teaching diagnostic skills to students. In a tradition more grounded in the marriage of sociology and education, researchers have addressed issues such as the impact of integrating real (HIV) patients into preclinical learning groups27 and the use of team learning in the context of an evidence-based medicine course.28

More narrowly focused questions in this area provide examples of highly specific educational implementations for the delivery of highly specific content areas. Here the questions are not so much addressing a theoretical question of the best way to deliver content; rather they ask whether local efforts at a particular content delivery mechanism were successful in delivering a particular content. Studies of this type are quite ubiquitous, and a few examples include the delivery of content such as spiritual history-taking,29 domestic violence,30,31 geriatrics assessment,32 attitudes toward substance-abusing patients,33 cardiology skills,34 lesbian and gay health care,35 the thyroid examination,36 end-of-life care,37 deafness and hearing impairments,38 the pelvic examination,39 and microscopic urinalysis.40 In most studies of this type, the authors provide an example of how they teach a particular content domain, offering an implied invitation to readers to use a similar model. Thus, these studies are relevant to the larger community of educators and researchers to the extent that the particular content domain is seen as important and as either lacking or poorly taught in the reader's own curriculum.

Finally, related to this last category of questions is a set of studies designed to determine the general level of knowledge of students or residents in highly specific content domains. Here, there is often no implementation of a content delivery mechanism to be evaluated. Rather, these studies attempt broad-scale evaluations of curricula in general for their success in teaching specific skills or knowledge, such as interns’ prescribing for common clinical conditions,41 graduating nurses’ medication calculation skills,42 family medicine residents’ ability to interpret electrocardiograms,43 or medical students’ skill sets in addressing domestic violence.44 Again, the content domains being addressed in these large-scale needs assessments are often quite specific, and they function either implicitly or explicitly as a call to the community to focus greater attention on the delivery of this particular content. The relevance of the study's findings and conclusions for the larger community of educators is largely dependent on the relative importance that individuals within the community place on these content domains compared with the vast number of other content domains that could be examined in this way.

Back to Top | Article Outline

Skills and Attitudes Relevant to the Structure of the Profession

A second large theme of research questions in medical education addresses the underlying skills and attitudes that form the substrate of professional competence. These “core competencies” appear to be quite similar to, if not always exactly synonymous with, those elaborated by the Accreditation Council for Graduate Medical Education (ACGME) and by the Royal College of Physicians and Surgeons of Canada (RCPSC) in its CanMeds roles. These competencies have generally been seen as important for some time, but more recently they have been the focus of increasing study and have been accorded the concentration of the medical education community's resources.

One area where a substantial number of educators and researchers have focused their collective efforts is on developing an understanding of “professionalism” in its broadest terms. Many are asking questions such as, What do we think professionalism is?45 What do our trainees think it is?46–48 How do we teach it or instill it?49–51 How do we measure it?52–54 As a group, the researchers and educators who have chosen to work in this area appear to have formed a loose consortium, a microcommunity within medical education, who meet, share their ideas, pool resources, and work toward a conceptualization of professionalism that is coherent and broadly accepted as valuable and meaningful to the community at large.

Similar concerted community efforts are being directed at several specific skills and attitudes that compose the professional substrate. Although they appear somewhat less developed as microcommunities, there are growing pools of researchers, educators, and administrators who are focusing their time and effort on the understanding, teaching, and evaluation of domains such as self-directed learning,55,56 self-assessment,57–62 and interprofessional collaboration and communication.63–68

Back to Top | Article Outline

Students’ Characteristics

A somewhat less integrated but nonetheless identifiable area of research is in the domain of students’ characteristics and, frequently, the interactions of these characteristics with curricula and performance. From the psychological and learning perspective, the focus has tended to be on individual differences in learning styles69–72 and motivation.73 These studies generally are fueled by the theory-based assumption that there are stable characteristics (or aptitudes) in individuals, and that researchers are motivated by the search for evidence of “aptitude by treatment interactions,” such as the interaction between learning style and content delivery, that are predicted by the theory. Although the evidence for these interactions has been somewhat elusive, their demonstration would offer both support for the theory and practical implications for education.

From a more sociological perspective, researchers have addressed issues of systematic differences in medical school experiences as a function of many individual characteristics including age74,75 gender,74–79 and race or ethnicity.80,81 These studies often highlight the systemic barriers to developing and maintaining diversity in the medical curriculum broadly and/or within specific programs within medicine.

An issue obviously related to the general topic of students’ characteristics is the question of medical school admission policies.82,83 Again, the questions being asked in this domain generally separate into two broad categories. The first and larger of these addresses efforts to increase the reliability and predictive validity of admissions policies (that is, selecting students who will be successful).84–89 The second but equally important category addresses the continuing effort to increase access to medical school for underrepresented populations.90–92 Also related to these topics are the issues of career choice and practice patterns, with studies addressing the individual and social factors that influence decisions such as trainees’ selection of a specialty93–96 or practice setting.97–99

Back to Top | Article Outline

Evaluation of Individuals

Of course, no discussion of research themes in medical education could ignore the massive effort being exerted in the field of measurement and testing. Student evaluation has long been a staple of medical education, and, if anything, attention to it is being accelerated by political forces such as the ACGME's and RCPSC's shifts in policy from accreditation requirements based on education processes to requirements based on learners’ outcomes. In short, these organizations have made a clear statement that they will no longer be satisfied with evidence that programs are teaching certain content; they are now interested in evidence that the students are learning it. This mandate has not so much altered the direction of measurement in medical education as it has fueled changes that were already evolving.

Thus, increasingly, there is an emphasis on authentic, performance-based assessments of clinical competencies, with these assessments occurring both in the clinical setting and in the simulated environment. Within the clinical setting, continuing efforts are being made to redress past concerns about the reliability and validity of end-of-rotation clinical marks.100 Many researchers are attempting to understand the cognitive, social, and environmental influences that affect the reliability and validity of longitudinal clinical ratings.101,102 Many others are exploring mechanisms to improve these clinical ratings, such as the use of formal evaluation sessions103 and the use of multiple, independent perspectives in the evaluation process.104,105 Still others are attempting to address the problem by shifting the focus of “in vivo” clinical evaluation away from longitudinally based assessments to more localized, “in-the-moment” assessments of specific performances using tools such as the mini–clinical evaluation exercise.106–108 In addition, there are efforts to develop novel tools for assessing clinical competence,109,110 with portfolio assessments becoming increasingly popular.111

Perhaps the greatest amount of energy, however, is being invested in the use of simulation for the purposes of evaluation. As described earlier, these simulations use many media including the computer interface112,113 and both high-tech114 and low-tech115 bench models. But by far, the most common simulation technology in evaluation is the standardized patient-based examination format widely known as the objective structured clinical examination (OSCE). Literally hundreds of articles have been published on the OSCE, and this concerted effort has led to an extensive understanding of the technology. We now have a body of literature that provides an understanding of how to mark it,116–120 many of the potential sources of error in scores,121–126 methods for setting passing standards,127–129 potential innovations in its administration,130–134 the impact of its use on the educational milieu,135 as well as a sociological critique of its use.136

Finally, as in the context of educational innovation, there are individual examples of innovation in evaluation for the purposes of measuring specific aspects of clinical competence, such as student reflection,137,138 bedside neurology,139 and attitudes towards the homeless.140 Again, as in the context of educational innovation, these tools are useful to the extent that the community sees the skills as needing explicit assessment and to the extent that individual evaluators within the community lack their own tools to make these evaluations (as was clearly the case, for example, with the OSCE).

Back to Top | Article Outline

Discussion

The scope of topics that are addressed by medical education research is broad. The field's breadth has the advantage of providing many opportunities for research and discovery, but it also risks generating a sense that the field lacks coherence and communal effort toward the resolution of “big questions.” In addition, the work of researchers and educators in medical education is broadly published not only across many journals that would identify themselves as relevant to medical education but also across many clinical journals. This breadth and scope of publication venues reflects positively on the state of medical education scholarship. Clearly the messages of medical education are infiltrating many places, to great benefit. However, again, this breadth does have implications for the community's ability to locate and integrate the research into coherent, programmatic efforts. These concerns can only be redressed through constant communal reflection regarding our choices of topics to address and our success in addressing them. This article is intended to be one contribution to this process. The purpose of the exercise was not to generate a comprehensive literature review of all medical education research published in the last half decade, nor to impose a set of themes on the literature that should considered comprehensive, prescriptive, or definitive. Rather, this article is intended to function as the early stages of an exercise in community-level reflective practice, to provide some examples of the content areas where we are choosing to invest our energies, and to establish the basis of an explicit dialogue regarding the nature and relative success of these choices.

Although such dialogues are important, they should not be limited to debates or discussions of what areas we are studying, which areas are relatively more or less valuable, and who fits into what categories. Rather, it is equally important to address why the community might see such an exercise as valuable and how we could take advantage of the results. In this sense, taking the initial step of trying to develop this article has been an important personal learning exercise, in that it has highlighted for me an important distinction regarding the pursuit of knowledge-building in a community of scholars: the distinction between scholars who are working on the same topic and scholars who are working together toward a shared goal. It is, to use a common analogy, the difference between parallel play and interactive play.

During the process of reflecting on the literature, several themes jumped out as being very apparent. As examples, the lines of research related to the content area of professionalism and those related to the development of the OSCE were obvious, at least in part because of their coherence of purpose. For these themes, I suspect that participants would recognize themselves as part of a community of researchers with common goals and questions, and would see the “value-added” in discussing their various ideas with each other. By contrast, other themes were possible to develop, but seemed to be substantially less satisfying upon development, more theoretical than real, more abstract than functional. To individuals who find their work placed in these categories, I apologize, as I suspect they feel much like the conference presenter who has discovered herself to be in the session entitled “Potpourri.” Although it was possible to see studies within these themes as “thematic,” they were not in any practical sense “programmatic.” The problems being addressed by individual researchers were similar at some level, but as a whole the individual studies did not seem to inform each other. It is in this sense that these themes were marked by a quality of institutionalized parallel play. Perhaps this merely speaks to the weakness of the themes that I created as an outsider to these domains. However, as described in the introduction, I am not the first to make such observations.1

Returning to a discussion of the role of theory in determining a sense of coherence, it is not clear, based on this review, that locating our work in a theoretical framework as suggested by Prideaux and Bligh1 is either necessary or sufficient for coherence to emerge in a particular domain. For example, with a few notable exceptions,137 much of the OSCE work appears more driven by practical innovation than by theoretical exploration despite its apparent coherence as a domain of research and development. By contrast, there are some domains that have theoretical grounding and intent but nonetheless generate a relatively weak sense (at least to an outsider) of communal effort. The research on experts’ cognitive knowledge structures, for example, seems to have been marked by this quality of institutionalized parallel play, with many of the community's members arguing and engaging at conferences, but largely ignoring or dismissing each other's data as they develop their own theories about the domain. Interestingly, despite its previous prominence in the medical education literature, this field of work was not a strong presence during the last five years in the four journals reviewed. It appears that these researchers, to the extent that they are still engaged in this work, have found other communities of researchers with whom to interact.

It is possible therefore, that the problem being perceived by key members of our community is not merely the absence of effective, guiding theory. Although the absence of theory in our writings and considerations may be a problem, this absence may be epiphenomenal to, rather than the cause of, a larger issue. Perhaps the real issue is the absence of a sense of community effort to build understanding of the phenomena we care about, and the absence of a community where data and ideas are not merely described, but listened to, and not merely dismissed or ignored but addressed, incorporated, and improved upon by other members of the community.

Community-level consensually mandated directions in research and development arise from a variety of sources: political institutions, funding agencies, particularly successful lines of research, and like-minded groups of researchers from the community of practice who acknowledge each other's work and incorporate it into their thinking rather than working in isolation. But for mandated directions in research to become programmatically knowledge-building at the community level, all these sources must be present, each making important and interacting contributions to the effort. As a medical education community, we have the capacity to affect each of these aspects. As administrators and lobbyists, we can work jointly toward the accrual of political and economic support for key areas of research and development. But as a community of researchers and developers, we must also do our part to work programmatically, both individually and as a community. We must seek out and support individuals who are pursuing promising ways of thinking and promising lines of research. And we must work more collaboratively to build mutual understanding. It is only through such community-based efforts that we will create a vibrant and evolving field in which we are not merely accruing information, but building knowledge and understanding about the enterprise of medical education.

Back to Top | Article Outline

References

1.Prideaux D, Bligh J. Research in medical education: asking the right questions. Med Educ. 2002;36:1114–5.
2.Colliver JA. Educational theory and medical education practice: a cautionary note for medical school faculty. Acad Med. 2002;77:1217–20.
3.Whitcomb ME. Research in medical education: what do we know about the link between what doctors are taught and what they do? Acad Med. 2002;77:1067–8.
4.Whitcomb ME. The journal continues to evolve. Acad Med. 2002;77:3–4.
5.Norman G. Holding on to the philosophy and keeping the faith. Med Educ. 2001;35:820–1.
6.Bligh J. Research in medical education at the start of the century. Med Educ. 2002;36:1000–1.
7.Bligh J. Evaluating the effects of research in medical education. Med Educ. 2001;35:1094.
8.Colliver JA. The research enterprise in medical education. Teach Learn Med. 2003;15:154–5.
9.Colliver JA. Constructivism with a dose of pragmatism: a cure for what ails educational research. Adv Health Sci Educ Theory Pract. 1999;4:187–90.
10.Curet MJ, Mennin SP. The effect of longterm vs shortterm tutors on the quality of the tutorial process and student performance. Adv Health Sci Educ Theory Pract. 2003;8:117–26.
11.De Grave WS, Dolmans DH, Van Der Vleuten CP. Student perspectives on critical incidents in the tutorial group. Adv Health Sci Educ Theory Pract. 2002;7:201–9.
12.Van den Hurk MM, Dolmans DH, Wolfhagen IH, Van der Vleuten CP. Testing a causal model for learning in a problem-based curriculum. Adv Health Sci Educ Theory Pract. 2001;6:141–9.
13.Custers EJ, Cate OT. Medical students’ attitudes towards and perception of the basic sciences: a comparison between students in the old and the new curriculum at the University Medical Center Utrecht, The Netherlands. Med Educ. 2002;36:1142–50.
14.Prince KJ, van Mameren H, Hylkema N, Drukker J, Scherpbier AJ, van der Vleuten CP. Does problem-based learning lead to deficiencies in basic science knowledge? An empirical case on anatomy. Med Educ. 2003;37:15–21.
15.Rudland JR, Rennie SC. The determination of the relevance of basic sciences learning objectives to clinical practice using a questionnaire survey. Med Educ. 2003;37:962–5.
16.Carney PA, Eliassen MS, Pipas CF, Genereaux SH, Nierenberg DW. Ambulatory care education: how do academic medical centers, affiliated residency teaching sites, and community-based practices compare? Acad Med. 2004;79:69–77.
17.Bowen JL, Irby DM. Assessing quality and costs of education in the ambulatory setting: a review of the literature. Acad Med. 2002;77:621–80.
18.Miflin BM, Campbell CB, Price DA. A conceptual framework to guide the development of self-directed, lifelong learning in problem-based medical curricula. Med Educ. 2000;34:299–306.
19.Davidson R, Duerson M, Rathe R, Pauly R, Watson RT. Using standardized patients as teachers: a concurrent controlled trial. Acad Med. 2001;76:840–3.
20.Kneebone R. Simulation in surgical training: educational issues and practical implications. Med Educ. 2003;37:267–77.
21.Bearman M. Is virtual the same as real? Medical students’ experiences of a virtual patient. Acad Med. 2003;78:538–45.
22.Callas PW, Bertsch TF, Caputo MP, Flynn BS, Doheny-Farina S, Ricci MA. Medical student evaluations of lectures attended in person or from rural sites via interactive videoconferencing. Teach Learn Med. 2004;16:46–50.
23.Stromso HI, Grottum P, Hofgaard Lycke K. Changes in student approaches to learning with the introduction of computer-supported problem-based learning. Med Educ. 2004;38:390–8.
24.Treadwell I, de Witt TW, Grobler S. The impact of a new educational strategy on acquiring neonatology skills. Med Educ. 2002;36:441–8.
25.Hatala RM, Brooks LR, Norman GR. Practice makes perfect: the critical role of mixed practice in the acquisition of ECG interpretation skills. Adv Health Sci Educ Theory Pract. 2003;8:17–26.
26.Nendaz MR, Raetzo MA, Junod AF, Vu NV. Teaching Diagnostic Skills: Clinical Vignettes or Chief Complaints? Adv Health Sci Educ Theory Pract. 2000;5:3–10.
27.Solomon P, Guenter D, Salvatori P. Integration of persons with HIV in a problem-based tutorial: a qualitative study. Teach Learn Med. 2003;15:257–61.
28.Hunt DP, Haidet P, Coverdale JH, Richards B. The effect of using team learning in an evidence-based medicine course for medical students. Teach Learn Med. 2003;15(2):131–9.
29.King DE, Blue A, Mallin R, Thiedke C. Implementation and assessment of a spiritual history taking curriculum in the first year of medical school. Teach Learn Med. 2004;16:64–8.
30.Korenstein D, Thomas DC, Foldes C, Ross J, Halm E, McGinn T. An evidence-based domestic violence education program for internal medicine residents. Teach Learn Med. 2003;15:262–6.
31.Haist SA, Wilson JF, Pursley HG, et al. Domestic violence: increasing knowledge and improving skills with a four-hour workshop using standardized patients. Acad Med. 2003;78(10 suppl):S24–S26.
32.Tandeter H, Peleg R, Menahem S, Biderman A, Fried VA. Teaching geriatric assessment in home visits: the family physician/geriatrician attachment. Teach Learn Med. 2003;15:123–6.
33.Christison GW, Haviland MG. Requiring a one-week addiction treatment experience in a six-week psychiatry clerkship: effects on attitudes toward substance-abusing patients. Teach Learn Med. 2003;15:93–7.
34.Issenberg SB, McGaghie WC, Gordon DL, et al. Effectiveness of a cardiology review course for internal medicine residents using simulation technology and deliberate practice. Teach Learn Med. 2002;14:223–8.
35.McGarry KA, Clarke JG, Cyr MG, Landau C. Evaluating a lesbian and gay health care curriculum. Teach Learn Med. 2002;14:244–8.
36.Houck WA, Soares-Welch CV, Montori VM, Li JT. Learning the thyroid examination–a multimodality intervention for internal medicine residents. Teach Learn Med. 2002;14:24–8.
37.Lorenz KA, Steckart MJ, Rosenfeld KE. End-of-life education using the dramatic arts: the Wit educational initiative. Acad Med. 2004;79:481–6.
38.Lock E. A workshop for medical students on deafness and hearing impairments. Acad Med. 2003;78:1229–34.
39.Herbers JE Jr, Wessel L, El-Bayoumi J, Hassan SN, St Onge JE. Pelvic examination training for interns: a randomized controlled trial. Acad Med. 2003;78:1164–9.
40.Canaris GJ, Flach SD, Tape TG, Stierwalt KM, Haggstrom DA, Wigton RS. Can internal medicine residents master microscopic urinalysis? Results of an evaluation and teaching intervention Acad Med. 2003;78:525–9.
41.Pearson S, Smith AJ, Rolfe IE, Moulds RF, Shenfield GM. Intern prescribing for common clinical conditions. Adv Health Sci Educ Theory Pract. 2000;5:141–50.
42.Grandell-Niemi H, Hupli M, Leino-Kilpi H. Medication calculation skills of graduating nursing students in Finland. Adv Health Sci Educ Theory Pract. 2001;6:15–24.
43.Boltri JM, Hash RB, Vogel RL. Are family practice residents able to interpret electrocardiograms? Adv Health Sci Educ Theory Pract. 2003;8:149–53.
44.Miller AW, Coonrod DV, Brady MJ, Moffitt MP, Bay RC. Medical student training in domestic violence: a comparison of students entering residency training in 1995 and 2001. Teach Learn Med. 2004;16:3–6.
45.Cruess SR, Johnston S, Cruess RL. “Profession”: a working definition for medical educators. Teach Learn Med. 2004;16:74–6.
46.Rizk DE, Elzubeir MA. Self-reported assessment by medical students and interns of unprofessional practice. Teach Learn Med. 2004;16:39–45.
47.Ginsburg S, Regehr G, Lingard L. To be and not to be: the paradox of the emerging professional stance. Med Educ. 2003;37:350–7.
48.Goldie J, Schwartz L, McConnachie A, Morrison J. Students’ attitudes and potential behaviour with regard to whistle blowing as they pass through a modern medical curriculum. Med Educ. 2003;37:368–75.
49.Kenny NP, Mann KV, MacLeod H. Role modeling in physicians’ professional formation: reconsidering an essential but untapped educational strategy. Acad Med. 2003;78:1203–10.
50.Baernstein A, Fryer-Edwards K. Promoting reflection on professionalism: a comparison trial of educational interventions for medical students. Acad Med. 2003;78:742–7.
51.Hatem CJ. Teaching approaches that reflect and promote professionalism. Acad Med. 2003;78:709–13.
52.Fontaine S, Wilkinson TJ. Monitoring medical students’ professional attributes: development of an instrument and process. Adv Health Sci Educ Theory Pract. 2003;8:127–37.
53.Arnold L. Assessing professional behavior: yesterday, today, and tomorrow. Acad Med. 2002;77:502–15.
54.Gordon J. Assessing students’ personal and professional development using portfolios and interviews. Med Educ. 2003;37:335–40.
55.Lloyd-Jones G, Hak T. Self-directed learning and student pragmatism. Adv Health Sci Educ Theory Pract. 2004;9:61–73.
56.Harvey BJ, Rothman AI, Frecker RC. Effect of an undergraduate medical curriculum on students’ self-directed learning. Acad Med. 2003;78:1259–65.
57.Tousignant M, DesMarchais JE. Accuracy of student self-assessment ability compared to their own performance in a problem-based learning medical program: a correlation study. Adv Health Sci Educ Theory Pract. 2002;7:19–27.
58.Ward M, Gruppen L, Regehr G. Measuring self-assessment: current state of the art. Adv Health Sci Educ Theory Pract. 2002;7:63–80.
59.Biernat K, Simpson D, Duthie E Jr., Bragg D, London R. Primary care residents self assessment skills in dementia. Adv Health Sci Educ Theory Pract. 2003;8:105–10.
60.Reiter HI, Eva KW, Hatala RM, Norman GR. Self and peer assessment in tutorials: application of a relative-ranking model. Acad Med. 2002;77:1134–9.
61.Mattheos N, Nattestad A, Falk-Nilsson E, Attstrom R. The interactive examination: assessing students’ self-assessment ability. Med Educ. 2004;38:378–89.
62.Fitzgerald JT, White CB, Gruppen LD. A longitudinal study of self-assessment accuracy. Med Educ. 2003;37:645–9.
63.Hawryluck LA, Espin SL, Garwood KC, Evans CA, Lingard LA. Pulling together and pushing apart: tides of tension in the ICU team. Acad Med. 2002;77(10 suppl):S73–S76.
64.Lingard L, Reznick R, Espin S, Regehr G, DeVito I. Team communications in the operating room: talk patterns, sites of tension, and implications for novices. Acad Med. 2002;77:232–7.
65.Hawk C, Buckwalter K, Byrd L, Cigelman S, Dorfman L, Ferguson K. Health professions students’ perceptions of interprofessional relationships. Acad Med. 2002;77:354–7.
66.Farmer EA, Beard JD, Dauphinee WD, LaDuca T, Mann KV. Assessing the performance of doctors in teams and systems. Med Educ. 2002;36:942–8.
67.Reeves S, Freeth D, McCrorie P, Perry D. “It teaches you what to expect in future …”: interprofessional learning on a training ward for medical, nursing, occupational therapy and physiotherapy students. Med Educ. 2002;36:337–44.
68.Gardner SF, Chamberlin GD, Heestand DE, Stowe CD. Interdisciplinary didactic instruction at academic health centers in the United States: attitudes and barriers. Adv Health Sci Educ Theory Pract. 2002;7:179–90.
69.Lindblom-Ylanne S, Lonka K. Students’ perceptions of assessment practices in a traditional medical curriculum. Adv Health Sci Educ Theory Pract. 2001;6:121–40.
70.Engleberg NC, Schwenk T, Gruppen LD. Learning styles and perceptions of the value of various learning modalities before and after a 2nd-year course in microbiology and infectious diseases. Teach Learn Med. 2001;13:253–7.
71.Mattick K, Dennis I, Bligh J. Approaches to learning and studying in medical students: validation of a revised inventory and its relation to student characteristics and performance. Med Educ. 2004;38:535–43.
72.Martin IG, Stark P, Jolly B. Benefiting from clinical experience: the influence of learning style and clinical experience on performance in an undergraduate objective structured clinical examination. Med Educ. 2000;34:530–4.
73.Perrot LJ, Deloney LA, Hastings JK, Savell S, Savidge M. Measuring student motivation in health professions’ colleges. Adv Health Sci Educ Theory Pract. 2001;6:193–203.
74.Haist SA, Wilson JF, Elam CL, Blue AV, Fosson SE. The effect of gender and age on medical school performance: an important interaction. Adv Health Sci Educ Theory Pract. 2000;5:197–205.
75.Kick S, Adams L, O'Brien-Gonzales A. Unique issues of older medical students. Teach Learn Med. 2000;12:150–5.
76.Bienstock JL, Martin S, Tzou W, Fox HE. Medical students’ gender is a predictor of success in the obstetrics and gynecology basic clerkship. Teach Learn Med. 2002;14:240–3.
77.Levy BT, Merchant ML. Differences in clinical experiences based on gender of third-year medical students in a required family medicine preceptorship. Acad Med. 2002;77:1241–6.
78.O'Hara BS, Maple SA, Bogdewic SP, Saywell RM Jr, Zollinger TW, Smith CP. Gender and preceptors’ feedback to students. Acad Med. 2000;75:1030.
79.de Saintonge DM, Dunn DM. Gender and achievement in clinical medical students: a path analysis. Med Educ. 2001;35:1024–33.
80.Gartland JJ, Hojat M, Christian EB, Callahan CA, Nasca TJ. African American and white physicians: a comparison of satisfaction with medical education, professional careers, and research activities. Teach Learn Med. 2003;15:106–12.
81.Tekian A, Han Y, Hruska L, Krainik AJ. Do underrepresented minority medical students differ from non-minority students in problem-solving ability? Teach Learn Med. 2001;13:86–91.
82.Elam CL, Stratton TD, Scott KL, Wilson JF, Lieber A. Review, deliberation, and voting: a study of selection decisions in a medical school admission committee. Teach Learn Med. 2002;14:98–103.
83.Searle J, McHarg J. Selection for medical school: just pick the right students and the rest is easy! Med Educ. 2003;37:458–63.
84.Turnbull D, Buckley P, Robinson JS, Mather G, Leahy C, Marley J. Increasing the evidence base for selection for undergraduate medicine: four case studies investigating process and interim outcomes. Med Educ. 2003;37:1115–20.
85.Stratton TD, Elam CL, McGrath MG. A liberal arts education as preparation for medical school: how is it valued? How do graduates perform? Acad Med. 2003;78(10 suppl):S59–S61.
86.Albanese MA, Snow MH, Skochelak SE, Huggett KN, Farrell PM. Assessing personal qualities in medical school admissions. Acad Med. 2003;78:313–21.
87.Gilbert GE, Basco WT Jr, Blue AV, O'Sullivan PS. Predictive validity of the Medical College Admissions Test Writing Sample for the United States Medical Licensing Examination Steps 1 and 2. Adv Health Sci Educ Theory Pract. 2002;7:191–200.
88.Kulatunga-Moruzi C, Norman GR. Validity of admissions measures in predicting performance outcomes: the contribution of cognitive and non-cognitive dimensions. Teach Learn Med. 2002;14:34–42.
89.Eva KW, Rosenfeld J, Reiter HI, Norman GR. An admissions OSCE: the multiple mini-interview. Med Educ. 2004;38:314–26.
90.Thomson WA, Ferry PG, King JE, Martinez-Wedig C, Michael LH. Increasing access to medical education for students from medically underserved communities: one program's success. Acad Med. 2003;78:454–9.
91.Blakely AW, Broussard LG. Blueprint for establishing an effective postbaccalaureate medical school pre-entry program for educationally disadvantaged students. Acad Med. 2003;78:437–47.
92.Strayhorn G. A pre-admission program for underrepresented minority and disadvantaged students: application, acceptance, graduation rates and timeliness of graduating from medical school. Acad Med. 2000;75:355–61.
93.Ciechanowski PS, Russo JE, Katon WJ, Walker EA. Attachment theory in health care: the influence of relationship style on medical students’ specialty choice. Med Educ. 2004;38:262–70.
94.Owen JA, Hayden GF, Connors AF Jr. Can medical school admission committee members predict which applicants will choose primary care careers? Acad Med. 2002;77:344–9.
95.Ward AM, Kamien M, Lopez DG. Medical career choice and practice location: early factors predicting course completion, career choice and practice location. Med Educ. 2004;38:239–48.
96.Zun LS, Downey L. Is a third year clerkship in emergency medicine correlated with a career choice in emergency medicine? Teach Learn Med. 2004;16:14–7.
97.Wilkinson D, Laven G, Pratt N, Beilby J. Impact of undergraduate and postgraduate rural training, and medical school entry criteria on rural practice among Australian general practitioners: national study of 2414 doctors. Med Educ. 2003;37:809–14.
98.Woloschuk W, Tarrant M. Does a rural educational experience influence students’ likelihood of rural practice? Impact of student background and gender. Med Educ. 2002;36:241–7.
99.Brooks RG, Walsh M, Mardon RE, Lewis M, Clawson A. The roles of nature and nurture in the recruitment and retention of primary care physicians in rural areas: a review of the literature. Acad Med. 2002;77:790–8.
100.Speer AJ, Solomon DJ, Fincher RM. Grade inflation in internal medicine clerkships: results of a national survey. Teach Learn Med. 2000;12:112–6.
101.Williams RG, Klamen DA, McGaghie WC. Cognitive, social and environmental sources of bias in clinical performance ratings. Teach Learn Med. 2003;15:270–92.
102.Kreiter CD, Ferguson KJ. The empirical validity of straight-line responses on a clinical evaluation form. Acad Med. 2002;77:414–8.
103.Battistone MJ, Milne C, Sande MA, Pangaro LN, Hemmer PA, Shomaker TS. The feasibility and acceptability of implementing formal evaluation sessions and using descriptive vocabulary to assess student performance on a clinical clerkship. Teach Learn Med. 2002;14:5–10.
104.Norcini JJ. Peer assessment of competence. Med Educ. 2003;37:539–43.
105.Joshi R, Ling FW, Jaeger J. Assessment of a 360-degree instrument to evaluate residents’ competency in interpersonal and communication skills. Acad Med. 2004;79:458–63.
106.Durning SJ, Cation LJ, Markert RJ, Pangaro LN. Assessing the reliability and validity of the mini-clinical evaluation exercise for internal medicine residency training. Acad Med. 2002;77:900–4.
107.Holmboe ES, Huot S, Chung J, Norcini J, Hawkins RE. Construct validity of the miniclinical evaluation exercise (miniCEX). Acad Med. 2003;78:826–30.
108.Kogan JR, Bellini LM, Shea JA. Feasibility, reliability, and validity of the mini-clinical evaluation exercise (mCEX) in a medicine core clerkship. Acad Med. 2003;78(10 suppl):S33–S35.
109.Bulik RJ, Frye AW, Callaway MR, Romero CM, Walters DJ. Clinical performance assessment and interactive video teleconferencing: an iterative exploration. Teach Learn Med. 2002;14:124–32.
110.Olson L, Schieve AD, Ruit KG, Vari RC. Measuring inter-rater reliability of the sequenced performance inventory and reflective assessment of learning (SPIRAL). Acad Med. 2003;78:844–50.
111.Wilkinson TJ, Challis M, Hobma SO, Newble DI, Parboosingh JT, Sibbald RG, Wakeford R. The use of portfolios for assessment of the competence and performance of doctors in practice. Med Educ. 2002;36:918–24.
112.Lieberman SA, Frye AW, Litwins SD, Rasmusson KA, Boulet JR. Introduction of patient video clips into computer-based testing: effects on item statistics and reliability estimates. Acad Med. 2003;78(10 suppl):S48–S51.
113.Guagnano MT, Merlitti D, Manigrasso MR, Pace-Palitti V, Sensi S. New medical licensing examination using computer-based case simulations and standardized patients. Acad Med. 2002;77:87–90.
114.Tsai TC, Harasym PH, Nijssen-Jordan C, Jennett P, Powell G. The quality of a simulation examination using a high-fidelity child manikin. Med Educ. 2003;37(1 suppl):S72–S78.
115.Friedlich M, Wood T, Regehr G, Hurst C, Shamji F. Structured assessment of minor surgical skills (SAMSS) for clinical clerks. Acad Med. 2002;77(10 suppl):S39–S41.
116.Margolis MJ, Clauser BE, Swanson DB, Boulet JR. Analysis of the relationship between score components on a standardized patient clinical skills examination. Acad Med. 2003;78(10 suppl):S68–S71.
117.Wilkinson TJ, Frampton CM, Thompson-Fawcett M, Egan T. Objectivity in objective structured clinical examinations: checklists are no substitute for examiner commitment. Acad Med. 2003;78:219–23.
118.McIlroy JH, Hodges B, McNaughton N, Regehr G. The effect of candidates’ perceptions of the evaluation method on reliability of checklist and global rating scores in an objective structured clinical examination. Acad Med. 2002;77:725–8.
119.Solomon DJ, Szauter K, Rosebraugh CJ, Callaway MR. Global ratings of student performance in a standardized patient examination: is the whole more than the sum of the parts? Adv Health Sci Educ Theory Pract. 2000;5:131–40.
120.De Champlain AF, Margolis MJ, Macmillan MK, Klass DJ. Predicting mastery level on a large-scale standardized patient test: a comparison of case and instrument score-based models using discriminant function analysis. Adv Health Sci Educ Theory Pract. 2001;6:151–8.
121.Colliver JA, Swartz MH, Robbs RS. The effect of examinee and patient ethnicity in clinical-skills assessment with standardized patients. Adv Health Sci Educ Theory Pract. 2001;6:5–13.
122.Humphris GM, Kaney S. Examiner fatigue in communication skills objective structured clinical examinations. Med Educ. 2001;35:444–9.
123.Wilkinson TJ, Fontaine S. Patients’ global ratings of student competence. Unreliable contamination or gold standard?. Med Educ. 2002;36:1117–21.
124.Chambers KA, Boulet JR, Furman GE. Are interpersonal skills ratings influenced by gender in a clinical skills assessment using standardized patients? Adv Health Sci Educ Theory Pract. 2001;6:231–41.
125.McKinley DW, Boulet JR. Detecting score drift in a high-stakes performance-based assessment. Adv Health Sci Educ Theory Pract. 2004;9:29–38.
126.Boulet JR, McKinley DW, Whelan GP, Hambleton RK. The effect of task exposure on repeat candidate scores in a high-stakes standardized patient assessment. Teach Learn Med. 2003;15:227–32.
127.Kaufman DM, Mann KV, Muijtjens AM, van der Vleuten CP. A comparison of standard-setting procedures for an OSCE in undergraduate medical education. Acad Med. 2000;75:267–71.
128.Kramer A, Muijtjens A, Jansen K, Dusman H, Tan L, van der Vleuten C. Comparison of a rational and an empirical standard setting procedure for an OSCE. Objective structured clinical examinations. Med Educ. 2003;37:132-9 [published erratum appears in Med Educ. 2003;37:574].
129.Wilkinson TJ, Newble DI, Frampton CM. Standard setting in an objective structured clinical examination: use of global ratings of borderline performance to determine the passing score. Med Educ. 2001;35:1043–9.
130.Mavis BE, Ogle KS, Lovell KL, Madden LM. Medical students as standardized patients to assess interviewing skills for pain evaluation. Med Educ. 2002;36:135–40.
131.Sibbald D, Regehr G. Impact on the psychometric properties of a pharmacy OSCE: using 1st-year students as standardized patients. Teach Learn Med. 2003;15:180–5.
132.Humphris GM, Kaney S. The objective structured video exam for assessment of communication skills. Med Educ. 2000;34:939–45.
133.Petrusa ER, Hales JW, Wake L, Harward DH, Hoban D, Willis S. Prediction accuracy and financial savings of four screening tests for a sequential test of clinical performance. Teach Learn Med. 2000;12:4–13.
134.Fliegel JE, Frohna JG, Mangrulkar RS. A computer-based OSCE station to measure competence in evidence-based medicine skills in medical students. Acad Med. 2002;77:1157–8.
135.Duerson MC, Romrell LJ, Stevens CB. Impacting faculty teaching and student performance: nine years’ experience with the Objective Structured Clinical Examination. Teach Learn Med. 2000;12:176–82.
136.Hodges B. OSCE! Variations on a theme by Harden. Med Educ. 2003;37:1134–40.
137.Boenink AD, Oderwald AK, De Jonge P, Van Tilburg W, Smal JA. Assessing student reflection in medical practice. The development of an observer-rated instrument: reliability, validity and initial experiences. Med Educ. 2004;38:368–77.
138.Charlin B, Roy L, Brailovsky C, Goulet F, van der Vleuten C. The Script Concordance test: a tool to assess the reflective clinician. Teach Learn Med. 2000;12:189–95.
139.Millos RT, Gordon DL, Issenberg SB, et al. Development of a reliable multimedia, computer-based measure of clinical skills in bedside neurology. Acad Med. 2003;78(10 suppl):S52–S54.
140.Lester HE, Pattison HM. Development and validation of the attitudes towards the homeless questionnaire. Med Educ. 2000;34:266–8.
© 2004 Association of American Medical Colleges