Misch, Donald A. MD
Professionalism,1 integrity, humanism, compassion, empathy, respect, altruism—these terms can be found throughout the medical literature, often accompanied by elegies for their disappearance in the modern physician. While the biomedical and technical aspects—the science—of medical practice continue to progress at an astonishing rate, it is the human part of medical practice—in particular physicians' humanism and professionalism—that seems endangered in proportion to the explosion of biomedical knowledge and rapid structural changes in the American health care system. If it is true, as succinctly stated by Francis Peabody,2 that “the secret of the care of the patient is in caring for the patient,” then the diminution of such caring cannot be ignored.
The public has regularly decried the apparent replacement of physicians' humanism with technologic sophistication, but patients and their advocates are not alone in their concerns. Physicians and medical educators themselves3,4,5,6,7,8,9,10,11,12,13,14,15 have expressed similar misgivings about the modern physician, just as various medical accrediting agencies such as the Liaison Committee for Medical Education (LCME) and the Accreditation Council for Graduate Medical Education (ACGME) (the latter through its Outcomes Project) have underscored their expectation that medical students, residents, and fellows be trained and formally assessed for professionalism and humanistic knowledge, attitudes, and skills. The call has been taken up as well by various medical specialty boards, with the American Board of Internal Medicine (ABIM)16, 17 being in the vanguard of this effort. Likewise, state licensing agencies have begun to demand that trainees' professional and humanistic competency be certified by appropriate assessment measures; the failure to do so may result in denial of medical licensure or board certification. Thus, a broad coalition, composed of individuals and entities from inside and outside the medical profession, has demanded the return of the professional, humanistic, caring physician.
DIFFICULTIES IN ASSESSING PHYSICIANS' PROFESSIONALISM AND HUMANISM
Given the apparent consensus on the importance of physicians' professional and humanistic skills, it would seem that medical educators should have no difficulty in implementing the necessary curricular alterations to foster the development of these aspects of the art of medicine. Yet, a whole host of factors has made such implementation considerably more problematic than expected; of those factors, one of the most important is the difficulty in assessing the desired professional and humanistic qualities. In the rest of this article, I discuss issues of such assessment and propose a new approach to it.
Indeed, the assessment of physicians' humanism and professionalism is problematic. For example, there are inherent difficulties associated with defining and operationalizing humanism and its components such as compassion, empathy, caring, integrity, and respect. Exactly what is compassion? Can a physician be compassionate even though his or her observable behavior does not appear to be so? How can one define the concept and process of empathy? Is it something that occurs in the mind of the physician, some sort of interpersonal transaction, or a combination of both? A physician misrepresents a patient's diagnosis so as to ensure health coverage that the physician sincerely believes is the patient's right; is this an example of physician integrity or just the opposite?
Another question: Should physicians' professionalism and humanism be assessed solely on the basis of behavior, or must a physician's knowledge and attitudes in these domains be measured so as to provide a full and valid evaluation? In grappling with this issue, the ABIM17 does not deny the importance of cognitive and attitudinal aspects of humanism; rather, it underscores the reality that whatever a physician's humanistic cognitions, attitudes, or intentions, it is in his or her interpersonal interactions with the patient that “humanism” primarily surfaces. Thus, the ABIM and other commentators assert that direct observation of physician behaviors is probably the most reliable, valid, and useful means of assessing physicians' professionalism and humanism.
Even if behavioral criteria were formulated on the basis of some consensus as to the definition of humanistic and professional qualities, any evaluation of the presence or absence of these qualities in a given situation will be highly context-dependent.17 The very same behavior that is judged to be humanistic in one culture, setting, or particular physician—patient encounter may be construed as entirely opposite in effect in another circumstance. Furthermore, who is to be the judge: the physician, the patient, a medically sophisticated third-party observer such as a nurse? How would a physician's self-assessment of his or her professional and humanistic skills correlate with those obtained from other observers?
For all of the above reasons, and others as well, the assessment of the art of medicine—professionalism and humanism as particular examples—is problematic.
CURRENT ASSESSMENT METHODS
Recognizing the many vexing questions relating to the assessment of the art of medicine, the ABIM has acknowledged that there is no “gold standard” for the evaluation of physicians' humanism. Nonetheless, the ABIM has asserted its belief that “a candidate's attainment and demonstration of these qualities can be assessed”17,p. 2 and that “abundant opportunities exist during residency training for assessing and reinforcing these qualities.”18,p. 721 In spite of the myriad definitional difficulties and operational obstacles, various commentators19,20,21 and the ABIM believe that “the most realistic approach to assessing humanism is by direct observation.”17,p. 5
Medical students' and residents' professionalism and humanism have been behaviorally assessed by a wide variety of observational techniques, evaluation instruments, and evaluators. According to Rezler,20 however, the most common method of “noncognitive” competency assessment of residents is through the completion of rating forms by faculty physicians. Typically, such assessment tools consist of semi-quantitative, clinical rating scales that contain a number of often poorly defined and poorly anchored behavioral objectives or expectations, on each of which the trainee is either given a numerical score (e.g., from 1 to 5) or marked as having performed or not performed the expected skill or intervention (e.g., a checklist by which a trainee is determined to have implemented each of the requisite steps in “breaking bad news”22, 23). Although many such instruments encourage elaborative comments from the raters, such commentary is often cursory at best, if not missing altogether.
Richards et al.20 are skeptical of the use of clinical rating scales in the evaluation of noncognitive aspects of medical care.
These clinical rating scales are … plagued by rater unreliability, generosity error—that is, most subjects are scored at the high or outstanding end of the rating scale. There is ambiguity of items and often a halo effect is noted—that is, a tendency for a rater to rate a student similarly on all of the characteristics rated. These deficiencies of rating scales strongly imply that they are unsatisfactory for evaluating noncognitive characteristics.”20
In spite of Richards' concerns, it would be incorrect to assert that semiquantitative rating scales are of no benefit in the assessment of physicians' professionalism and humanism. It would be equally incorrect, however, to overestimate the validity, reliability, and utility of current assessment methods. Existing assessment techniques should not be discarded, but the semiquantitative assessment of physicians' professionalism and humanism, which are elements of the “art” of medicine, can and should be complemented with qualitative assessment. Arnold et al.19 point out that the scientific method is not the only approach to the acquisition and interpretation of “data.” Indeed, employment of the scientific method does not preclude the use of complementary assessment modalities that may serve to provide a fuller, more accurate picture of particular characteristics of physicians. Qualitative assessment in medicine is currently considered by many to be “soft,” unreliable, and of little practical value. It is, by definition, subjective; but this does not mean that it is therefore invalid, inaccurate, or unreliable.19,24,25
A NEW APPROACH
If, indeed, professionalism and humanism are components of the art of medicine and it is professionalism and humanism that we wish to measure, then it follows that assessment in this domain may be more akin to artistic evaluation than scientific evaluation. What are the implications of assessing humanism in physicians in a way analogous to that in which “the critics” assess paintings, literature, music, theater, and other creative endeavors?
For the sake of argument, let us consider the works of the great composer Mozart. Precisely what is it that makes Mozart's musical compositions extraordinary? Clearly, it is not simply the use of particular notes themselves, for everyone, professional and amateur alike, uses the same, restricted number of musical notes. What distinguishes Mozart's work from the ordinary is the way in which the notes are put together—Mozart's ability to integrate musical notes in creative, synthetic ways that evoke powerful feelings, images, and thoughts.
In art as in science, the whole is much greater than the sum of its parts. Both music and literature may be broken down into their component parts for analysis—notes, chords, and rhythm for music; words, sentences, and paragraphs for literature—but such an analysis provides little useful information when evaluating art. Is it helpful to know that Mozart wrote more sonatas in B minor than in other keys, or that he often used 6/8 time? Similarly, how does the information that Leo Tolstoy typically employed 26 words per sentence or that his chapters averaged 19 pages in length help us capture any of the essence of his art? Without looking at the overall effect, how would one rate, on a semiquantitative scale from 1 to 5, William Faulkner's use of grammar? In the evaluation of the artistic domain, few would apply such sterile quantitative or semiquantitative measures.
I propose that the same is true of the art of medicine. Teaching and evaluating the use of particular steps in the medical interview may be a useful educational device, but the interview steps themselves are the building blocks, the “notes”; they are not the music. It is not only what the physician says that makes him or her humanistic, it is also the precise wording, tone, pacing, and inflection of the voice, the timing of the message, and the body language. The humanistic physician makes nuanced use of touch, a look, a pause—can these be quantified in an evaluatively useful way? All of these elements are key parts of the humanistic physician, and none can be assessed numerically or quantitatively. As a consequence, any method of evaluation that fails to take into account these important elements and their integration into a coherent whole is, at least partially, missing the point.
The “art critic” analogy is apt in other ways as well. For instance, there is no single right answer in art; many different techniques and approaches have yielded great works. Is not the same true of the professional or humanistic physician? Must all certifiably humanistic physicians interview patients in precisely the same way, or is there more than one effective way to obtain the necessary information? Indeed, it might be argued that a rigid, mechanical style, irrespective of the fact that it may contain the “right” steps, cannot be highly humanistic. Part of the art of humanistic medicine is a physician's ability to be flexible, creative, and constantly sensitive to patient feedback. We do not argue that Rembrandt was a great painter but Picasso was not; they are very different in style but both have elements of greatness. Why should we require the humanistic physician to follow a rigidly prescribed regimen so long as his or her chosen approach works with a given patient? Even with clinical treatment, it would be inappropriate and sometimes dangerous to follow a rigidly prescribed regimen, since therapies that work for most patients do not necessarily work for all. It is the “artistic” clinician who can sense the individual uniqueness of a patient and adjust treatment accordingly.
What is it that distinguishes the professional critic—artistic or otherwise—from the amateur enthusiast? The critic, or “connoisseur” in Eisner's26 less pejorative terminology, is an individual who brings specific expertise to the evaluation of a given entity. The connoisseur possesses refined perceptual capabilities—a “palette,” to use the lexicon of wine tasting—that enables him or her to appreciate nuance, subtleties, and complexities that the amateur misses. These refined perceptual capabilities are the result of specific training, relevant experience, and what might be called “expert memory.” They serve as “critical guideposts” or “advance organizers” that direct the connoisseur's attention to key or subtle elements typically overlooked by the novice. To the layman, a chest X-ray is nothing more than a confusing amalgamation of lines and shadows; to the radiologist—an X-ray connoisseur—it is a window into the presence and cause of human disease. Through the recognition and appreciation of the subtleties, complexities, and underlying meaning of a chest X-ray's lines and shadows, a radiologist is able to make inferences with respect to physical illness.
The latter analogy underscores an additional distinguishing characteristic of the connoisseur as compared with the amateur observer. Connoisseurs possess a vocabulary of terms that allows them to accurately record and convey the observed subtleties and complexities. Thus, art critics have defined key elements of artistic composition (e.g., line, shape, form, value, color, texture, space, figure and ground, balance, unity and variety, rhythm and repetition, dominance and subordinance), and a broad vocabulary has been developed to describe the many subcategories, variations, or possibilities within each element. Consider, for example, the relatively basic deconstruction of the element of color into hue, value, intensity, monochromatic, polychromatic, analogous, complementary, primary, and secondary. By training and experience, art critics have acquired an “enlightened eye”26 that allows them to go beyond “That's a pretty picture” to a sophisticated and nuanced appreciation and description of the work under consideration. In a similar vein, the wine connoisseur recognizes, and understands the differences between, many types of wine. Furthermore, the wine connoisseur, like the art critic, has a conceptual schema and a complementary vocabulary to describe and evaluate a given wine; for such an expert, each wine can be judged with respect to clarity, color, aroma, bouquet, body, tannin, acid, sugar, balance, finish, and appeal.
Finally, according to Eisner27, the connoisseur, in comparison with the amateur observer, is capable of providing “criticism.” In Eisner's view, criticism is a process that includes description, interpretation, and evaluation/judgment. It should not be regarded as a negative appraisal; it is instead “an educational process intended to enable individuals to recognize qualities and characteristics that might otherwise have been unnoticed and unappreciated.”28,p. 129 It is important to bear in mind that the connoisseur, in providing “criticism,” is not the bearer of the “right answer” but, rather, is offering an educated opinion, perhaps one of many.
Thus, I propose that the medical profession needs professionalism/humanism “critics” or “connoisseurs” (hereafter abbreviated as “humanism connoisseurs”) just as the arts have critics of music, literature, paintings, and theater.
What attributes would such an individual possess? He or she would be well versed in the cognitive, attitudinal, and interpersonal elements that constitute the various aspects of medical professionalism (e.g., responsibility, integrity, altruism) and humanism (e.g., empathy, compassion, respect). He or she would have the training and experience (the “expert memory,” “advance organizers,” or “critical guideposts”) to delve beneath the surface of particular behaviors or physician—patient interactions to note subtleties, nuances, and complexities that distinguish true professionalism and humanism from behaviors that merely have the superficial appearance of such. The humanism connoisseur would be able to look beyond mechanical recitation of requisite steps in a process (e.g., “breaking bad news”) to assess whether a given physician was able to demonstrate professionalism and humanism in an integrated authentic manner. Part and parcel of such assessment would be the ability to evaluate behaviors in a wide variety of contexts, eschewing a “one size fits all” notion of what it means to be compassionate, empathic or respectful. Similarly, the humanism connoisseur would be highly knowledgeable as to the various ways in which patients may respond to physicians' behaviors, recognizing that a patient's immediate verbal and nonverbal responses to a physician's intervention may or may not accurately reflect the true significance and effect of that intervention.
The development of a broad descriptive vocabulary based on relevant categories, components, and processes related to professional and humanistic qualities is a key enabling mechanism permitting the humanism connoisseur to accurately, concisely, and understandably describe and summarize his or her observations of a particular physician; without such a vocabulary, even accurate observation cannot be translated into a useful formative or summative evaluative statement. In analogy to the art critic or wine connoisseur, the humanism connoisseur must be able to deconstruct broad notions of compassion, respect, integrity, or altruism into generally-agreed-upon constituent elements, accompanied by a descriptive nomenclature. As but one example, the seemingly intuitive concept of empathy includes components (cognitive, emotional, behavioral), processes (perception, data collection and processing), and process locations (internal or intrapsychic as well as interpersonal). The humanism connoisseur would further recognize sometimes subtle but important distinctions, including, for example, the difference between having empathy and showing empathy, or the differentiation between empathy and related, overlapping concepts such as sympathy, compassion, identification, caring, and humanism. Just as the physician's broad knowledge, sophisticated taxonomy, and rich descriptive terminology of bacteria allow specific identification of a particular infectious illness and targeted therapy, so, too, will the humanism connoisseur be capable of accurately identifying—again as an example—specific empathic defects and implementing individually tailored corrective interventions.
The humanism connoisseur will possess yet another skill: the ability, once having deconstructed professionalism and humanism into their key elements and building blocks, to resynthesize these components and processes into an integrated whole. For instance, the connoisseur must not only recognize the various aspects of “breaking bad news” to a patient, but also be able to critically evaluate how well a physician puts these pieces together to create an authentic response that harmoniously integrates the relevant cognitive, emotional, and behavioral tasks. Does the physician's response flow, in a particular context; does it hold together in a cohesive, believable way? Or, in contrast, does the physician understand the steps in the process of communicating bad news to a patient but lack the capacity to smoothly and appropriately integrate them so as to produce the desired effect? By way of musical analogy, it is one thing to understand notes, key, tempo, and rhythm; it is another to truly make music.
Finally, the humanism connoisseur will probably continue to use rating scales of some kind. However, such assessment instruments will need to reflect the sophisticated conceptual understanding and rich descriptive vocabulary previously described. The rating scales will serve primarily as cues for the connoisseur to encourage elaborative comments that concisely and accurately portray an individual's strengths and relative deficits in specific aspects of physicians' professionalism or humanism.
OBSTACLES TO IMPLEMENTATION
Of the various obstacles to the implementation of more valid assessment of the elements of the art of medicine discussed above, three are worthy of discussion here. The first obstacle is the relative reluctance of physicians to complement quantitative evaluation with qualitative evaluation.
The quantitative tools of the scientific paradigm—the medical model—have served modern American medicine well. Through the use of such methods new discoveries and advances in the understanding and treatment of disease have progressed at a tremendous rate. Moreover, physicians, from their premedical years and continuing throughout their medical school and graduate training, are immersed in the scientific method; the medical model is a method in which physicians are highly trained and one with which they feel comfortable. In addition, the scientific method meshes nicely with the obsessive-compulsive, perfectionistic personality traits commonly found in physicians.29 Indeed, the model is designed to reduce ambiguity and, in the best of circumstances, provide a “single, right answer,” exactly the response sought by those with perfectionistic character traits and, conversely, anathema to most connoisseurs of the arts. Qualitative assessment methods, in contrast, are foreign to many physicians, in terms of both approach and hands-on experience. Training medical humanism connoisseurs and gaining acceptance among other faculty and students of their expertise and value will be a difficult, although not impossible, task.
A second obstacle to valid assessment of medical professionalism and humanism is related to, but extends beyond, the above-mentioned concerns. Many medical faculty believe, or at least hope, that through the use of semiquantitative rating scales, even for the measurement of aspects of the art—rather than the science—of medicine, they obtain relative protection from subsequent administrative and legal challenges to grades they have given their trainees. The underlying assumption is that qualitative evaluation is both “subjective” and idiosyncratic, while quantitative, “objective” evaluation methods can be more easily justified as appropriate and accurate. There is, in the words of Anwar et al.,30 “a great seduction to a system and process of evaluation that appears objective.” [Italics are mine.]
There are at least two problems with this argument. The first, of course, is that “objective” evaluation is often less than objective. Whether it is the presence of a palpable liver edge, a I/VI diastolic murmur, or an aspect of medical professionalism, many findings in medicine and medical education involve greater or smaller degrees of subjectivity. The second is that the use of quantitative or semiquantitative evaluation instruments lends an aura of precision to assessment that, at least in the case of the art of medicine, is misleading. A whole host of factors, as described above, impairs the precision of such assessment methods. Most important, however, the search for precision in trainee evaluation often obscures the more fundamental question of validity. Assessment measures that provide data to the third decimal place mean nothing if the evaluation method is fundamentally invalid in its application to a particular problem. Or, to put it another way, it is more important to have an imprecise answer to an important question than a precise answer to an unimportant question. The presence or absence of professional and humanistic qualities in medical trainees is an important question, and it warrants authentic assessment resulting in valid, even if imprecise, answers.
More generally, the concern about legal challenges mentioned earlier underscores the degree to which medical educators, particularly during the preclinical years of training, have lost control of their own curricula. Although progress has been made recently, assessment of medical trainees is still highly focused on cognitive testing, often of the multiple-choice-question (MCQ) format. There is little doubt that such examinations are time-efficient; tests can be administered simultaneously to many students, and grading is rapidly achieved by mechanical means. But these are not the only reasons that multiple-choice questions dominate trainee assessment, especially during the first years of medical school. Given the increasing culture of contention regarding grading and grades, MCQs offer less opportunity for protest than do other formats. Many a medical educator trembles at the thought of administering an essay exam, especially when contemplating an entire class of students contesting their numerical scores on each essay. The result, however, is that trainees, rather than medical faculty, are increasingly in control of the decision as to what are “legitimate” assessment methods in medical training. In effect, by means of the threat of student protest, faculty are proscribed from using many qualitative or subjective assessment formats, even when such formats may be the only means by which to adequately test key knowledge, attitudes, or skills.
The third, and final, barrier is pragmatic rather than attitudinal. Valid qualitative assessment of medical professionalism and humanism requires substantial commitment of resources. Just as art or wine connoisseurs need specific training and repeated experience to develop their respective assessment skills, so, too, will medical humanism connoisseurs require up-front investments of time, money, and training to become capable at their task. It is no more reasonable to suppose that the average physician is particularly skilled in evaluating medical humanism than it is to expect the average man on the street to be an astute art critic. Similarly, valid and accurate assessment of a trainee's mastery of the art of medicine will be time-intensive. Indeed, because of increasing pressure on medical faculty to generate clinical revenue and third-party payers' continuing reduction of the time patients may spend in the hospital or with physicians generally, it is becoming uncommon for a medical trainee to be directly observed by faculty during any substantial portion of a clinical interaction with a patient. This latter logistic barrier is an impediment to assessment of professionalism and humanism regardless of the method used. Ideally, a cadre of trained medical humanism connoisseurs would be specifically charged with, and given release time for, such intensive observation.
AN EDUCATIONAL IMPERATIVE: VALID ASSESSMENT OF PHYSICIANS' PROFESSIONALISM AND HUMANISM
In medical training, what is assessed (e.g., knowledge, attitudes, skills) and how something is assessed (e.g., written multiple-choice questions, clinical evaluation, grades versus pass-fail) are crucial cues by which trainees determine the “real, experienced, lived, informal, or hidden curriculum”12, 13, 31, 32 in comparison with the “formal curriculum” outlined in written goals and objectives as well as course and clerkship syllabi. Thus, the assessment methods employed by medical faculty provide an important, if indirect, message as to what medical faculty truly value in their trainees; in this respect, as in so many others, actions speak louder than words.
The implications of these hidden curricular realities are profound for the art, as compared with the science, of medicine. Professionalism and humanism are difficult to assess quantitatively, often leading to the employment of inadequate or invalid semiquantitative instruments for such evaluation or, even worse, pass-fail grading. The ultimate result of these assessment deficiencies is reinforcement of the hidden curriculum and the relative marginalization of the art of medicine. Professionalism and humanism are relegated to auxiliary matters, adjuncts to the truly important biomedical questions. In contrast, valid qualitative evaluation puts to advantageous use the axiom that assessment drives learning in medical education. Qualitative methods offer a potential vehicle to facilitate physician trainees' knowledge and skills in these important domains, and in the process, provide a means of undermining the dichotomy between objective and subjective, between the scientific method/medical model and other ways of knowing. The use of qualitative assessment methods underscores the truth that the professional practice of medicine is an interdisciplinary enterprise that integrates disease and illness, mind and body, thoughts and feelings, objective and subjective. It affirms the ideals to which most physicians subscribe and aspire.
Thus, professionalism and humanism are imperatives of the medical profession; they are the “heart” of the physician that accompanies his or her scientific mind.15 Moreover, they are crucial components of the profession's contract with the society it serves.1, 33 As such, the teaching and assessment of medical professionalism and humanism warrant the same careful implementation as any other part of the medical curriculum.
Qualitative evaluation of the art of medicine will be difficult, but easy, semiquantitative assessment approaches are often inaccurate or irrelevant. Proper qualitative assessment methods and tools, in the capable hands of trained evaluators, are the key to authentic, valid, and useful evaluation of physicians' professionalism and humanism. The importance of these dimensions of the physician dictates that medical educators muster the will and develop the methods to measure them accurately and usefully.20 Qualitative evaluation of the art of medicine: we can do it, and we must.
1. Swick HM. Toward a normative definition of medical professionalism. Acad Med. 2000;75:612–6.
2. Peabody FW. The care of the patient. JAMA. 1927;88:877–82.
3. Physicians for the twenty-first century. Report of the Project Panel on the General Professional Education of the Physician and College Preparation for Medicine. J Med Educ. 1984;59(11 Pt 2):1–208.
4. Miller SZ, Schmidt HJ. The habit of humanism: a framework for making humanistic care a reflexive clinical skill. Acad Med. 1999;74:800–3.
5. Markakis KM, Beckman HB, Suchman AL, Frankel RM. The path to professionalism: cultivating humanistic values and attitudes in residency training. Acad Med. 2000;75:141–50.
6. Educating medical students: assessing change in medical education—the road to implementation (ACME-TRI report). Acad Med. 1993;68(6 suppl).
7. Learning objectives for medical student education—guidelines for medical schools: report I of the Medical School Objectives Project. Acad Med. 1999;74:13–8.
8. Barry D, Cyran E, Anderson RJ. Common issues in medical professionalism: room to grow. Am J Med. 2000;108:136–42.
9. Swick HM, Szenas P, Danoff D, Whitcomb ME. Teaching professionalism in undergraduate medical education. JAMA. 1999;282:830–2.
10. Reynolds PP. Reaffirming professionalism through the education community. Ann Intern Med. 1994;120:609–14.
11. Wallace AG. Educating tomorrow's doctors: the thing that really matters is that we care. Acad Med. 1997;72:253–8.
12. Hundert EM, Hafferty F, Christakis D. Characteristics of the informal curriculum and trainees' ethical choices. Acad Med. 1996;71:624–42.
13. Coulehan J, Williams PC. Vanquishing virtue: the impact of medical education. Acad Med. 2001;76:598–605.
14. Wear D. Professional development of medical students: problems and promises. Acad Med. 1997;72:1056–62.
15. Wear D, Castellani B. The development of professionalism: curriculum matters. Acad Med. 2000;75:602–11.
16. Clinical competence in internal medicine. Ann Intern Med. 1979;90:402–11.
17. A Guide to Awareness and Evaluation of Humanistic Qualities in the Internist. Philadelphia, PA: American Board of Internal Medicine, 1985.
18. Evaluation of humanistic qualities in the internist. Ann Intern Med. 1983;99:720–4.
19. Arnold RM, Povar GJ, Howell JD. The humanities, humanistic behavior, and the humane physician: a cautionary note. Ann Intern Med. 1987;106:313–8.
20. Richards RW, Wolff HJ. Measuring the unmeasurable. J Am Osteopath Assoc. 1982;82:124–8.
21. Cohen R. Assessing professional behaviour and medical error. Med Teach. 2001;23:145–51.
22. Buckman R, Kason Y. How to Break Bad News: A Guide for Health Care Professionals. Baltimore, MD: Johns Hopkins University Press, 1992.
23. Ptacek JT, Eberhardt TL. Breaking bad news. A review of the literature. JAMA. 1996;276:496–502.
24. Steckler A, Eng E, Goodman RM. Integrating qualitative and quantitative evaluation methods. Hygie. 1991;10(2):16–20.
25. Steckler A, McLeroy KR, Goodman RM, Bird ST, McCormick L. Toward integrating qualitative and quantitative methods: an introduction. Health Educ Q. 1992;19(1):1–8.
26. Eisner EW. The Enlightened Eye: Qualitative Inquiry and the Enhancement of Educational Practice. Upper Saddle River, NJ: Prentice-Hall, 1991.
27. Eisner EW. The Educational Imagination: On the Design and Evaluation of School Programs. 3rd ed. New York: Macmillan, 1994.
28. Worthen BR, Sanders JR, Fitzpatrick JL, Worthen BR. Program Evaluation: Alternative Approaches and Practical Guidelines. 2nd ed. New York: Longman, 1997.
29. Gabbard GO, Menninger RW. The psychology of the physician. In: Gabbard GO, Menninger RW (eds). Medical Marriages. Washington, DC: American Psychiatric Press, 1988:23–38.
30. Anwar RA, Bosk C, Greenburg AG. Resident evaluation: it is, can it, should it be objective? J Surg Res. 1981;30(1):27–41.
31. Hafferty FW. Beyond curriculum reform: confronting medicine's hidden curriculum. Acad Med. 1998;73:403–7.
32. Stern DT. In search of the informal curriculum: when and where professional values are taught. Acad Med. 1998;73(10 suppl):S28–S30.
33. Wynia MK, Latham SR, Kao AC, Berg JW, Emanuel LL. Medical professionalism in society. N Engl J Med. 1999;341:1612–6.