Parker, Malcolm MBBS, M Litt, M Health & Med Law; Luke, Haida PhD; Zhang, Jianzhen BMed, MPH(TH), PhD; Wilkinson, D MBChB, MSc, MD, PhD, DSc; Peterson, Raymond M App Sc, PhD; Ozolins, Ieva MBBS, PhD
Many medical educators have addressed the challenge of teaching and assessing professionalism at medical school in response to numerous recent concerns about clinical and behavioral failures, as well as inadequacies in the profession’s self-regulatory responses.1–4 The public perceives the profession as too slowly exchanging its traditional culture of professional solidarity for one of greater public accountability,5,6 and medical educators are keen to accelerate change. The public and the medical profession alike accept the importance of doctors demonstrating professional behavior,7 and a flurry of definitions,8,9 proclamations,10 and official statements11 now exist to help guide developments for both the profession and medical education. A further impetus has been recent research demonstrating a relationship between unprofessional student behavior and subsequent involvement of doctors with their registration authorities.12,13 We accept the broad definition of professionalism, encompassing requirements for clinical competence and patient safety as well as behavioral standards, which these statements usually employ. The Pyramid of Professionalism reflects this broad understanding, although our focus here is on the management of unprofessional attitudes and behavior.
Deans and curriculum designers have expended much effort to integrate this hitherto ignored and conceptually different and difficult area into crowded medical curricula, in the face of continuing resistant cultural forces.14–16 Professors often introduce professionalism, particularly its attitudinal and behavioral aspects, as a component of bioethics and medical ethics curricula, but this has frequently resulted in inadequate assessment of these areas.
In this paper, we describe a now well-established program that integrates a medical ethics, law, and professionalism curriculum with a personal and professional development (PPD) process which primarily supports referred students but also includes an assessment function for unprofessional attitudes and behaviors. We report data on 507 students managed by the PPD process between 2000 and 2006, and we demonstrate that even a large medical school can meet the considerable challenges in assessing the noncognitive domain. This is a descriptive study because no systematic data concerning the management of poor performance in the noncognitive domains are available from the previous undergraduate course. Nevertheless, we describe a model with coherent theoretical underpinnings, which facilitate assessment in this fraught area of professionalism, as an integrated aspect of the academic program.
Context, Setting, and Participants
Medical education in Australia
Until little more than a decade ago, all Australian medical programs admitted the majority of students directly from secondary school, rather than from college programs, with only small quotas of older and international students. Many of these traditional five- or six-year programs have undergone considerable modification in recent years. The University of Queensland joined the University of Sydney and Flinders University in the mid-1990s to transform their programs into four-year, graduate-entry ones, requiring a university bachelor’s degree. In these transformed programs, problem-based learning (PBL) formats dominate the preclinical years, and expanded entry criteria include interviews. Since then, other universities have moved to graduate entry or hybrid entry (graduate and school leaver streams), and a number of new medical schools have followed the graduate-entry model. These programs have brought Australian medical education a little closer to the U.S. model, though a number of distinctions remain. The graduate programs accept students with any university degree, unlike the general custom in the United States, where medical schools prefer a liberal arts and science premedical preparation; although Australia is now beginning to see the development of these more focused programs. The first two years of medical school in Australia provide instruction in basic sciences and pathology, although early clinical contact is also a feature. The third and fourth years are clinical clerkships or rotations, and university graduation at the end of the fourth year is followed by an internship year which leads to unconditional registration with the state medical board. Although this is the minimal legal requirement, most graduates work as junior doctors in a hospital setting for two or three years before moving into specialist training streams.
Formal curriculum in ethics, law, and professionalism
From the commencement of the University of Queensland graduate-entry medical program in 1997, the school of medicine (SOM) administrators developed the domain of ethics and professional practice (EPP), and instructors teach and assess it as a component equal in status to that of the scientific, clinical, and population health domains. The formal curriculum incorporates a range of ethical, legal, and professional topics, most of which are currently covered in years one and two; the bolded items in List 1 indicate those with particular relevance to professionalism. Instructors assess student mastery of this formal curriculum through written tests and objective structured clinical examinations (OSCEs), but the SOM administrators recognized that although performance in these assessments demonstrates understanding of professional issues, this does not validly reflect actual attitudes and behaviors. Consequently, they developed a PPD support and assessment process linked to, but distinct from, the formal curriculum, starting in 1999.
Developing the PPD process
The ultimate, explicitly stated aims of the PPD process are to contribute to patient welfare and prevent patient harm.17 By helping students recognize error and personal and professional shortcomings; by supporting student health and welfare; by providing feedback, insight, and remediation regarding unprofessional behavior; and by assessing the noncognitive domain of student development, the process lays foundations to help fulfill these aims. It also reflects developments in professional regulation.
The SOM administration developed “needs assistance” categories (List 2) and support and assessment processes (described below) via a Delphi process involving SOM staff and committees, community members, and representatives of the profession (the Delphi process is a structured or semistructured process for exploring ideas and information from a group of experts or interested people, to develop an outcome for the group or the sponsoring organization18). The categories include many areas of concern raised in formal teaching sessions, as outlined in List 1. The descriptors pertaining to each needs assistance category comprise specific ideals to which students should aspire, such that a concern arises when any aspect of the student’s performance is perceived as inadequate and further exploration is desirable. The SOM piloted the process in 1999 and has implemented it fully each year beginning in 2000.
At the project’s inception, research conducted by an SOM PhD student and reported in November 2000 demonstrated that there was strong support (83% of instructors and 85% of students) for formal assessment of professionalism but considerable initial caution over practical ethical issues such as transparency, fairness, and implementation.19 Teachers expressed, through qualitative, free-text responses, concerns such as lack of objectivity, personal biases, and unfair exclusion, but they strongly supported intervention in cases of mental health problems and lack of self-care. Teachers and students expressed, again through qualitative responses, additional concerns, including reliability of evaluation, differential weighting of the categories, standards for academic failure on the basis of unacceptable behavior, perceptions of the process as punitive, incompatibility between teaching and assessment roles, an emphasis on negative attitudes and behaviors, and appropriateness of criteria according to year level. Thirty-nine percent of instructors disagreed or strongly disagreed that time spent by teachers with students for evaluation purposes was adequate, and 77% disagreed or were unsure about having had adequate briefing in using the process. Thirty-two percent of instructors were concerned that possible litigation and legal appeals would influence their participation.
The SOM conducted a review in 2001 and 2002, involving semistructured surveys and interviews of instructors and students in years one and two. As a result of the review, SOM administrators realized that, despite their good intentions in terms of developing a supportive process, significant proportions of students and staff perceived it as somewhat punitive. Consideration of the results of this review and the PhD student’s research led to a greater focus on student support, the development of a comprehensive instructor guide, improved explanations during orientation week, and greater attention to following up with at-risk students. These changes were effected through the PPD committee interviews, SOM committees, and the teaching program. The SOM introduced further refinements. For example, the chair of the Personal and Professional Development Committee (PPDC) now regularly briefs instructors in their first or second year at the SOM at the commencement of teaching blocks. The chair explains the PPD principles, processes, and templates; gives advice on providing feedback and support to students; provides information on university, SOM, and community support services; and, finally, offers examples of student concerns and exemplary responses by instructors during the PPD process.
A “Commitment to Professionalism” document, which students sign at the commencement of their first year, reinforces the principles and requirements of the process at an early stage and provides evidence that students understand and accept the SOM expectations of attendance and behavior which have assessment implications.20 The process is formative during medical students’ first year, but a policy change allowed year-one notifications to contribute to summative documentation from year two onwards. This means that a single needs assistance report or even multiple reports cannot be used as the basis of a failure on professional behavior grounds in year one, but the grounds for failing a student in years two, three, and four, in cases of repeated reports and PPD interviews, can include reports from year one as well as those from subsequent years.
Implementing the PPD process
The PPD process (Figure 1) encourages PBL instructors and clinical teachers to formally notify the chair of the PPDC of any concern from the listed categories (List 2) about a current student, which the instructor or teacher has not been able to resolve in situ. At briefings, the PPDC chair encourages instructors to use the instructor guide for student advice and to discuss students about whom they are concerned with the PPDC chair, if they feel the need. The chair strongly encourages instructors and teachers to refer students who they feel continue to need assistance despite their initial counsel, so that they may quickly resolve their issues, but the guide and the chair also promulgate the clear message that any single referral will not, of itself, constitute an impediment to the academic progress of the student. There is no stipulated threshold of seriousness up to which instructors must manage issues themselves before discussing the concern or referring the student for assistance.
We have found that attendance (and, to a lesser extent, participation) concerns are often the presenting symptom of a range of underlying student welfare or other kinds of issues, including illnesses (e.g., depression); part-time work commitments that, at times, interfere with commitment to the program; family and personal dilemmas; and so on. The significant extent to which the PPD processes are represented to students as supportive augments the idea that responding to concerns about attendance and participation does not amount to assessing the students as morally blameworthy but, rather their behaviors as concerns to be clarified. This approach is consistent with that of medical regulators in Australia (the state medical boards), where complaints about doctors are investigated and clarified before attaching any blame and applying sanctions. In some cases, attendance does turn out to constitute a significant lapse of professionalism—for example, when a student lacks sufficient maturity and insight into the need to accrue sufficient clinical experience to establish an adequate basis for internship training. Persistent absenteeism, in spite of support, advice, and, in some cases, reprimand, becomes a clear case of unprofessional conduct. Many early cases of student absenteeism, of course, are innocent lapses or reflect the casual culture of university life.
The PPDC sends a letter to students who are notified for the first time as having poor attendance, which invites them to communicate any difficulties and encourages their reflection. All subsequent attendance notifications and most needs assistance reports result in an invitation to attend an interview with the PPDC. The chair of the PPDC makes the decision concerning the need to attend an interview. Usually, only those students whose cases have already been managed by another school or university process are not required to attend an interview. The PPDC consists of the chair (the chair of the EPP domain of the program—currently a male), an academic psychiatrist (currently a female), and a medical students’ society representative (male or female). At the interview, the student responds to the instructor’s or teacher’s report, and the student and committee members clarify the issues which arise and negotiate a response/remediation plan and follow-up arrangements, if required. The chair of the PPDC sends a transcript of the interview to the student as well as a letter to the notifying instructor or teacher, indicating that the interview has occurred. In some cases, the PPDC invites students to subsequent meetings for further support and/or to monitor progress.
In cases of repeated needs assistance notifications, a single serious notification concerning behavior (such as a recent case involving attempted sexual molestation of a child by a medical student during a voluntary family attachment, with the student admitting to the charge in court), or a combination of unjustified absences and one or more notifications the PPDC may, by consensus, recommend that the head of school convene a subcommittee of the board of examiners. The head of school has discretion in deciding on convening a subcommittee. The head usually convenes a subcommittee on the PPDC’s recommendation, but, on two occasions, he has interviewed and further counseled students, with follow-up monitoring, in preference to moving to the assessment function of the process. The subcommittee first examines the relevant documentation and interviews the student and then makes a provisional judgment as to whether the student has satisfied the program requirements of professional conduct. The full board of examiners reviews the provisional decision and ultimately determines whether the student passes or fails the year on the basis of unsatisfactory professional attitudes or behavior (Figure 1). There is no fixed number or defined seriousness of notifications which constitute a failure. This flexibility reflects the enormous variation in the nature and seriousness of the issues raised, the need for careful negotiation in each interview, and the requirement for fair but appropriate judgment in relation to accumulated notifications. The PPDC has made recommendations to the head of school only when, on the basis of accumulated evidence, it is clear that the student has failed to respond to repeated advice and has established a significant prima facie pattern of unprofessional attitudes/behaviors. In contrast to most U.S. programs, Australian medical schools do not use deans’ letters to indicate student problems to residency programs, although in some Australian states medical students register with the medical registration boards, and, in these circumstances, boards become involved in the welfare and conduct problems of some students. The relationships between registration boards and medical schools regarding student conduct is in a state of flux at present, and it is likely that registration of students will expand as national rather than state-based registration of practitioners develops further.
We extracted PPD data for the years 2000 to 2006, during which 2,630 students participated in the medical degree program, from a secure Microsoft Access database and secure hardcopy interview paperwork, managed by the PPD administration officer. We entered the data into one central registry, exported it into Microsoft Excel, and then imported it into SAS for data analysis. We used original paper reports to clarify data when required, including assigning needs assistance categories when instructors had not done so. We collapsed some notified categories on the basis of accompanying comments from instructors; for example, an instructor may note both responsibility/reliability and participation categories in cases where the fundamental concern is one about participation in classes. On the other hand, a number of reports indicated two or more distinct categories of concern for individual students. We cross-checked the data entered in the central registry with the original database.
The performance of students in traditional assessments of EPP is similar to the other areas of the curriculum. For example, a total of 40 students failed the year-one EPP domain component (written exams and OSCE stations) from 2001 to 2003 (5.7%). These students sat supplementary examinations. One student (of these 40) repeated a year on the basis of subsequent EPP failure during the study period.
Between 2000 and 2006, 507 students, from the total enrollment of 2,630 (19%), received 711 PPD reports, including 420 unsatisfactory attendance reports, 120 reports with both unsatisfactory attendance and needs assistance, and 171 needs assistance reports. (The total enrollment of 2,630 comprises the 10 entering cohorts from 1997 to 2006; see Table 1.) These were distributed across the four years of the program in the following proportions: first-year students received 132 (18.6%) of the PPD reports, second-year students received 292 (41.1%), third-year students received 155 (21.8%), and fourth-year students received 132 (18.6%). Figure 2 indicates the proportion of total needs assistance reports (291) from each of the categories used from 2000 to 2006. The most frequent category was responsibility/reliability (136 reports or 46.7%), which reflects the fact that the most frequent trigger for instructor concern is attendance and punctuality, but it is clear from interviews, and not surprising, that a wide variety of issues (e.g., illness, personal or financial hardship, part-time work) underlies unsatisfactory attendance. The next-most-frequent category was participation (122 reports or 41.9%), with different contributing issues (e.g., personality or culturally based shyness, lack of confidence about knowledge) again accounting for many of these notifications. Thirty-four reports (11.7%) noted concerns in more than one category. For example, concerns about both responsibility/reliability (e.g., poor attendance related to a student’s part-time work commitments) and self-appraisal (e.g., poor insight into requirements of the program and the student’s marginal or poor academic performance) can coexist. Self-appraisal is often a relevant category for students who are interviewed, supported, and advised but then subsequently notified by another instructor to be in need of assistance and, therefore, interviewed again. This is particularly the case for those students who are interviewed numerous times. A small number of students are incapable of assimilating and responding to advice concerning, for example, offensive or careless interpersonal behavior, and some lack the insight required to accept that their performance is inadequate. The PPDC interviewed 117 students once and 25 students two or more times. It is important to note that not all students interviewed on multiple occasions demonstrated unprofessional behavior. For example, a student interviewed three times on the basis of presenting problems such as responsibility/reliability (List 2) may have personal, financial, or health problems and be attending follow-up and supportive interviews. Most such students welcome the invitation to return for discussion, and their experiences have contributed to dispelling perceptions amongst the student body of the process as punitive.
The PPDC referred four students to the head of school when it considered that the supportive and advisory aspects of the process had not been effective. Two students failed on the basis of unacceptable professional behavior, with a requirement to repeat the year. These students had 6 and 10 PPD interviews, respectively. Because the EPP curriculum and PPD processes constitute a part of the mainstream program, these were academic failures on grounds of professionalism, in contrast to most universities’ traditional, nonacademic disciplinary methods of dealing with problematic student behavior. This reflects the acceptance of medical professional attitudes and behavior as an essential component of professional performance. The other two students did not fail, but the head of school interviewed them, gave them further advice, and monitored their progress the subsequent year. (Subsequent to the 2000–2006 study period, two more students have failed a year on the basis of unprofessional behavior.)
Medical educators agree that no admissions process will successfully prevent all future professionally inadequate students/doctors from entering medical programs.21 They also recognize that some high academic achievers exhibit unacceptable professional behaviors in subsequent professional practice. At one point during the study period, a student who subsequently became involved with medical board disciplinary processes achieved the highest cohort score in EPP examinations! Our results from seven years demonstrate that the attitudinal and behavioral aspects of professionalism can be assessed within the contexts of the teaching and learning of ethics, law, and professionalism through adequate support and feedback; through rigorous record-keeping; and through collegiate judgment.
Issues and responses
Problems contributing to attendance and punctuality reports include physical and mental health problems, personal and financial difficulties, and overcommitment to part-time work. Participation notifications include quiet types, but personality disorders, anxiety, and depression also interfere with participation and attendance. Some international students indicate that their home culture encourages them to passively receive knowledge from their superiors, which both poses difficulties for them in a PBL environment and affects group functioning. We tentatively interpret the significantly higher number of needs assistance reports from year two as a reflection of a number of aspects of students, teachers, and the program’s structure. During year one, students generally exercise caution while negotiating a novel domain. By the second year, they have found their feet and become familiar with the program, but a number of students remain somewhat immature and, in these contexts, more readily express their foibles and inadequate capacity for reflection. By years three and four, they are more widely dispersed throughout a vast geographical area, are seriously engaged in clinical contexts, and are somewhat more mature. Many of those who have been interviewed do not attract further notifications and, presumably, have reflected and modified their approaches. Further, teachers who instruct third- and fourth-year students are not as easily reached for professional support and development in these newer areas of welfare and professionalism, and notification of concerns for these students is likely to be not as efficient or as complete as for students in the earlier years.
Occasionally, the PPDC makes urgent referral arrangements for mental health assessment and management. More frequently, the committee advises students to see their general practitioners. When appropriate, the PPDC reinforces attendance and punctuality expectations by calling students’ attention to program rules and expectations, and the committee emphasizes the assessment implications of further notifications by calling students’ attention to the program’s requirement for demonstrating adequate professional attitudes and behaviors in order to progress (see below).
Responding to problems of self-appraisal, respect, and discrimination (see specifications in List 2) is often less clear-cut. Students may state that their apparent failings were not malicious, that others have misinterpreted them, or that their instructors could have intervened at an earlier stage to resolve the problem. When it is clear that the student lacks insight into the inappropriateness and possible harmful consequences of a particular behavior, the PPDC gives clear guidance about the standards expected by both the community and the profession. Negotiating responses in these instances must be subtle and flexible because students may be genuinely ignorant of the potential for harm of their behavior (such as flippant comments or jokes), and a number of notifications result from differences of either perception or approach between students and instructors. For example, some overseas-trained instructors express difficulty with the often boisterous camaraderie which can characterize some group learning settings. This means that the PPDC often advises students that whereas some people may understand that the students did not intend their behavior to be harmful or potentially harmful, they must reflect on the possibility that others may perceive their behavior as damaging or possibly damaging, and they (the students) must exercise greater mindfulness and consideration. The PPDC has occasionally concluded that the instructor has treated the student unfairly, and, in fact, instructors have been removed from the teaching program on the basis of independent comments from a number of students.
We do not initially distinguish between concerns for student welfare and concerns about attitudes and behavior, and U.S. readers may find this model somewhat different from their own, where the helping and sanctioning processes are more clearly separated. But some problems of student well-being have potential implications for patient welfare and safety, and behavioral failings may be based on mental health problems, personal stress, or even coercive pressures in the student’s learning environment. We believe this intermingling of factors can be quite complex in the presenting phases, so that, for some time at least, attributions of blame and consequent sanctions are inappropriate. Moreover, medical regulators now take into account the various factors (e.g., illness, substance abuse) which contribute to behavior resulting in complaints, in an effort to rehabilitate affected doctors and, when possible, keep them in the workforce. The PPD support–assessment continuum similarly provides students with advice, feedback, and support while simultaneously underscoring the fact that failure to heed advice may have assessment implications, just as the recalcitrance of practitioners, impaired through any cause but provided with avenues of rehabilitation, may ultimately result in disciplinary action. Despite these complexities, we have explicitly promulgated a clear distinction between the support and assessment processes, which has gone some way to ensuring students that an invitation to attend a PPD interview is a supportive, not a disciplinary or punitive, matter.
Integrating the PPD processes in the program: the Pyramid of Professionalism
Professional attitudes and behaviors contribute to professional competence and should be assessed. The literature reports a wide range of teaching formats,22–24 but the PPD process itself also serves an important educative function. Most students integrate the principles of professionalism (List 2) into their own, already well-developed personal value systems, and we regard students who have not been interviewed, and those interviewed but who do not reappear, as achieving satisfactory standards in professional attitudes and behaviors.
A model which focuses on unacceptable behaviors differs from familiar graded assessments which reward high achievement, because of the categorical differences between cognitive and moral/behavioral competencies. We agree with Huddle25 that although the cognitive capacities (knowledge and skills) of medical students require implantation and cultivation de novo, and accretive learning and its evaluation, the roots of professional attitudes and behavior are well and truly established at the commencement of studies and are not as amenable to cumulative, positive evaluation as are those other attributes. Actual behavior during routine activity is the object of our behavioral assessment, not special assignments or cognitive hurdles such as written or simulated examinations,26 which essentially assess students’ knowledge about desired behavior, not their behavior itself. Positive measurements of acceptable behavior generally require simulated assessment, but this does not assess actual, day-to-day, behavior. Further, the demonstration of positive levels of professional virtues such as honesty is conceptually and practically problematic. What would being ranked as more honest than someone else who, nevertheless, is also considered to be honest, amount to? Would it require being placed in a situation of demonstrating one’s honesty under some duress, for example? And would that not imply some inequity of opportunity in being able to demonstrate grades of honesty? Although there may be aspects of professionalism which lend themselves more readily to positive assessment, depending on how broadly professionalism is defined (e.g., attendance and punctuality), and although most schools are likely to develop some combination of positive and negative student assessment, we believe that in the area of attitudes and behavior, our model will become more attractive as faculty face the conceptual and practical challenges of behavior assessment more squarely. Without such an approach, the aims of evaluating actual behavior and of preventing the progress of students who, it is agreed, ought not to progress, will remain elusive.
Reliable assessment requires the collation of notifications over time27,28 from multiple independent observers, across different settings.29 Attaching significance to such patterns, assuming acceptable attitudes and behaviors at entry, accepting as satisfactory those students who are not referred to the PPDC or who do not reappear, and meticulously documenting notifications, interviews, and feedback all mirror the processes of professional self-regulation used by the medical registration authorities. Self-regulation is a fundamental and indispensable component of professionalism, contrary to some recent proposals,30 and the Pyramid of Professionalism effectively introduces students to the regulatory environment of the profession. The model for assessment of professional attitudes and behaviors also mirrors a distinction made at the level of professional practice in Queensland. The state medical board is responsible for investigating claims against practitioners, but when investigations uncover prima facie cases of unprofessional conduct, the board refers them to either a professional conduct review panel, where decisions are made by two peers and a lay person, or to the health practitioners tribunal, where the decision maker is a district court judge assisted by two professional peers and a lay person. The panels deal with relatively less serious clinical matters (e.g., matters of clinical competence not resulting in serious harm), whereas the tribunal deals with more serious breaches of conduct or clinical competence (e.g., sexual impropriety or cases resulting in serious harm). Only the tribunal can suspend registration or deregister a doctor. In a roughly similar way, the PPDC takes on the role of support, clarification, and negotiation, up to the point of referring prima facie cases of recalcitrant unprofessional behavior to the subcommittee of the board of examiners. Only this group, like the tribunal, can make summative assessment decisions and impose academic sanctions.
The student body has strongly supported the PPD support and assessment processes. The PPDC student member is usually the president of the medical student society, which encourages considerable confidence in the process. Senior students participate during orientation activities in familiarizing first-year students with the process. Student representatives on SOM committees regularly communicate student concerns and participate in deliberations and decisions concerning the PPD process. The student society is a representative body whose executive members sit on SOM committees, contribute to curriculum development, and run student support programs, study groups, mentoring schemes, advisory services, and social functions. Student members of the PPDC have frequently provided relevant information and advice to students being interviewed—advice that would otherwise not have been available and that frequently involves specific support from the society. The society has also proposed an extension of the program to include peer assessment,31 which would further increase confidence in the process.
We designed, implemented, and continue to refine a support and assessment process relating to the welfare and professional attitudes and behaviors of medical students, integrated with the formal teaching and assessment of ethics and professional practice. The model is consistent with the continuing development of professional regulation regimes. Academic failure has resulted from poor professional behaviors. We encourage other schools to consider adapting the model for their own circumstances.
The authors are grateful to the two independent reviewers and the reviewer from the journal whose comments have significantly helped to clarify and strengthen the paper.