Papadakis, Maxine A. MD; Loeser, Helen MD, MSc; Healy, Kathleen
Since 1995, the University of California, San Francisco (UCSF) School of Medicine has operated an evaluation system that monitors students' professional behaviors longitudinally in their third and fourth years of medical school.1 In this evaluation system, a form called the Physicianship Evaluation Form is submitted about a student who receives a less than satisfactory rating on professional skills at the end of any clerkship. The identified student and the school work to remediate the student's deficiencies. If deficiencies in professional skills are identified in two or more clerkships, the dean's letter for application to residency programs will document these areas of concern or deficiencies. In addition, the student will be placed on academic probation and may be subject to academic dismissal, even if passing grades have been attained in all clerkships. The goals of this system have been to identify medical students who demonstrate unprofessional behaviors in order to remediate their deficiencies and to give the school an administrative structure to deal with such behaviors. We have now expanded this professionalism-evaluation system to the first two years of medical school, and are reporting here on our experience.
REASONS FOR EXPANDING THE EVALUATION SYSTEM
As we gained experience with the physicianship-evaluation system in third- and fourth-year medical students, the clerkship directors and the administration realized that several students identified as having deficiencies in physicianship skills during their clerkships had manifested problematic behaviors earlier in medical school. In addition, we observed behaviors of concern in several first- and second-year students for whom implementing early remediation would have been optimum. These behaviors included unnecessary interruptions in class; inappropriate behaviors in small groups both with peers and with faculty; unacceptable timing of requests for special needs for taking examinations. We suspect many of these students have received similar feedback in other settings; such students are particularly resistant to incorporating such feedback into their behaviors. Therefore, we felt the need for a formal structure that brings these issues to a student's attention in a new, earlier, and heightened way.
We found that one way to identify students who were later likely to have deficiencies in physicianship skills was to look at their evaluations from a course entitled Foundations of Patient Care. This is a course on doctoring skills and the social and ethical contexts of medical care that spans the first two years. It consists of small-group settings and preceptorships in physicians' offices. Some of the course directors in Foundations of Patient Care are also clerkship directors. When we looked back at evaluations from this course, problematic areas for students who later behaved unprofessionally had already been identified. Even though the course faculty provided feedback to these students about their behaviors, we recognized a “missed opportunity” for an institutional effort to provide intensive remediation.
Previous work also supported expansion of our program to the first two years of medical school. In 1993, Phelan and colleagues reported on a groundbreaking program to evaluate students' noncognitive professional attributes at the University of New Mexico School of Medicine.2 In their system for reporting problematic professional issues, basic scientists often identified problem students, and adverse reports on students were concentrated in the first two years of medical school. Thirty percent of all reports were submitted on first-year students, and an additional 46% of all reports were submitted on second-year students.
Another factor that supported the expansion of our program to the first two years is that close and prolonged observation of students is a critical factor in the identification of professional deficiencies. Foundations of Patient Care is an example of a course that affords that opportunity. In the third and fourth years of medical school, working with teams that have the same attending physicians and residents for a month is no longer the norm. Rather, students may rotate to different sites and to different evaluators several times within the same week. Frequent, fragmented, and limited interactions between third- and fourth-year medical students and their evaluators are commonplace.3
HOW THE EVALUATION SYSTEM WAS EXPANDED
The associate dean for student affairs (MAP) is responsible for the physicianship evaluation system. In 1998, she presented the data on the use of the Physicianship Evaluation Form from third- and fourth-year students to the course directors at the meetings of the first- and second-year course committees. Student representatives from the first- and second-year classes are also members of the course committees. The associate dean for student affairs discussed the need for earlier identification of students with professionalism deficiencies so that remediation could occur or at least begin before the students started their clerkships. The first- and second-year course directors made expansion of the professionalism-evaluation process to the first two years an objective for the academic year. Over subsequent meetings, the course directors used the third- and fourth-year Physicianship Evaluation Form as a template and modified it to fit professional developmental issues in the first two years of medical school. Greater emphasis was placed on student relationships with other students, staff, and faculty within a learning environment. A code of conduct, Upholding the Medical Student Statement of Principles, was incorporated into the form.
After discussion with their faculty, some course directors wanted the physicianship-evaluation system to apply to students who came late to lectures or missed laboratory or small-group sessions. Despite the aggravation it causes lecturers, the associate dean for student affairs felt that class attendance was covered under the existing course-evaluation system and, therefore, to include it would be peripheral to the intent of the physicianship-evaluation system. However, the behavior of a disruptive student during a lecture or small-group session would be addressed by the physicianship-evaluation system.
The course directors decided to include a “preamble” to the first- and second-year Physicianship Evaluation Form. This preamble states that the student who receives the form needs help developing physicianship skills and that the faculty is concerned about the student's behavior. It also gives an example of the kind of behavior that warrants a Physicianship Evaluation Form—for example, a student repeatedly does not show up for a preceptorship and has not communicated with the preceptor; attempts to give the student feedback about the issue have been unsuccessful.
After several meetings within the academic year, the first- and second-year course committees agreed upon a physicianship-evaluation system for the first two years of medical school. The Committee on Curriculum and Educational Policy (curriculum committee) endorsed the evaluation system and it went into effect in July 1999. The university's legal counsel reviewed the evaluation system and supported it without reservation.
THE FIRST- AND SECOND-YEAR EVALUATION SYSTEM
Below we describe the Physicianship Evaluation System as used in the first and second years of our medical school.
A faculty member who has concerns about a student's professional behavior reports that behavior to the course director. The course director makes inquiries about the student's behavior. Such inquiries usually include direct communication with a preceptor, small-group leader, or lecturer. If the course director is convinced that the student has deficiencies in professional development, the course director meets with the student to provide feedback and review the contents of the Physicianship Evaluation Form. (This form is reproduced in Chart 1 in this article.) The student is asked to sign the form, acknowledging the opportunity to discuss its contents with the course director. The form also includes a section for the student to include optional comments. The student may provide information that negates the Physicianship Evaluation Form, and the course director may choose to retract the form. More often, the form is submitted to the associate dean for student affairs. Deadline for such a submission is eight weeks into the student's next course or rotation.
The associate dean for student affairs meets with the student to identify the problematic issues, to counsel, and to remediate. She facilitates referrals to appropriate professional counseling. At the quarterly academic screening (promotions) committee meetings, course directors are asked about the student's progress in professional growth. Course or clerkship directors are notified of the student's educational needs so that the most appropriate preceptor or clerkship site can be chosen to help that student. The faculty member in charge of a course can submit only one Physicianship Evaluation Form about a student. If a course is given throughout the academic year—for example, Foundations of Patient Care—a Physicianship Evaluation Form can be submitted once for each quarter that the course is offered.
In order to enhance early recognition and remediation of problematic behaviors, the academic consequences of receiving a Physicianship Evaluation Form in the first or second year of medical school differ from those of receiving such a form in the third or fourth year. If a student receives a Physicianship Evaluation Form from one clerkship in the third or fourth year, the student meets with the associate dean for student affairs for counseling and remediation. However, if Physicianship Evaluation Forms are submitted from two or more clerkships, the physicianship problem is described in the dean's letter of recommendation for residency. In addition, the student is placed on academic probation, and may be eligible for academic dismissal from medical school even if passing grades have been attained in all courses. In contrast, in the first two years, even if the student receives multiple Physicianship Evaluation Forms, no mention is made of this process in the dean's letter of recommendation for residency, and the student is not necessarily placed on academic probation. However, if a student receives two or more Physicianship Evaluation Forms (from two or more courses) during the first two years, and receives a subsequent form in the third or fourth year from a clerkship or rotation, this indicates a persistent pattern of inappropriate behavior. The academic consequences are the same as if two clerkships had submitted Physicianship Evaluation Forms in the third and fourth years.
The goal of the professionalism-evaluation process is to help students understand why the evaluation was written, give them appropriate feedback on their behaviors, and help the students develop ways to improve so they have a good start on becoming physicians. The process is intended to be educational, not punitive. However, students who have received these forms frequently disagree with that interpretation and experience the process as punitive (see “Experience with the System,” below).
EXPERIENCE WITH THE SYSTEM
In the first year of its implementation, three students received Physicianship Evaluation Forms. Student A, a first-year student, repeatedly missed assigned small-group sessions in the patient interviewing class (Psychiatry 101). Of particular concern was when the student missed the small-group session where s/he was assigned to interview the patient. The faculty judged that the student needed improvement in the areas of “reliability and responsibility.” The student received feedback from the small-group leader and the course director. S/he also met with the associate dean for student affairs. The student was referred for psychiatric counseling to explore whether depression was a factor in his/her performance.
Student B, a second-year student, received a Physicianship Evaluation Form because s/he missed several preceptorship sessions, was not sensitive to the needs of patients, and did not respond to feedback and make appropriate changes in behavior. A community-based preceptor, who is an experienced faculty member who had precepted many students, initiated these comments. The course director was convinced of the merits of the observations after discussions with the preceptor, who was considered to have excellent judgment and insight. The student felt that it was inappropriate that s/he receive a Physicianship Evaluation Form. This resulted in extensive communications over a three-month period between the student and the preceptor, course director, ombudsperson, and administration. An informal resolution session was held between the student, the ombudsperson, the course director, and the associate dean for student affairs. The student supplied documentation that demonstrated his/her appropriate attendance at the preceptorship. The course director retracted that aspect of the negative evaluation that had to do with attendance, but not the other remaining deficiencies in professional development. The student received extensive feedback during the multiple interactions, but it is uncertain whether the student was able to incorporate the feedback constructively. The preceptor was so annoyed with the frequency and intensity of the interactions that he is no longer willing to precept our students.
Student C, a second-year student, received a Physicianship Evaluation Form under the category of “relating well to faculty in a learning environment” for making an inappropriate comment about the quality of a written examination to fellow classmates while the examination was in session. At the meeting with the associate dean for student affairs, the student explained that the course director had established a casual environment with friendly interchanges between faculty members and students. The student readily acknowledged his/her behavior, but felt that submission of the form was punitive rather than educational. The student also did not know of the existence of this new professionalism-evaluation system, materials for which had been distributed at the orientation at the beginning of the academic year. S/he made constructive, insightful comments about the process. This interaction highlighted that there are many ways to help a student with professional development. In this instance, one-on-one feedback may have sufficed, which would have lessened the pain that the student felt because s/he received a form.
We strongly believe that if professionalism is important enough to evaluate, it must be taught in the curriculum. A 1998 survey found that nearly 90% of medical schools offer formal instruction in professionalism and that much of that instruction occurs during the first two years of medical school.4 We have such a curriculum, and as at most other schools, it is concentrated in the first two years. During our orientation for the incoming first-year students, professionalism is introduced, defined, and discussed in a formal session, and we ask all students to sign the UCSF School of Medicine Statement of Principles. At the White Coat Ceremony, highlighting the end of the orientation, that statement of principles is signed by the dean and returned to the students. They also receive the written policy on our professionalism-evaluation system and a copy of the Professionalism Evaluation Form in “Nuts and Bolts,” a handbook of essential information. Further emphasis on development of professional skills occurs mainly in Foundations of Patient Care.1
Although most U.S. medical schools have curricula on professionalism, several essential attributes of medical professionalism are generally covered inadequately, such as the attribute “respond to societal needs and reflect a social contract with the communities served.”4 Also, professional development is a combination of the formal, hidden, and informal curricula.5,6,7,8 An example of the hidden curriculum is found in a study by Stern. Over six months, he audiotaped eight inpatient medicine ward teams and showed that the value of interprofessional respect, which includes respect by generalists for subspecialists and conversely, is actually taught as disrespect.9
The UCSF School of Medicine is in the process of a major curricular reform, and our new curriculum has explicit learning objectives on professionalism. The format in the new core curriculum of the first two years of the school of medicine shifts substantially to small-group and collaborative learning, so there is increased opportunity for observation of students' behaviors and performances relating to professionalism. The portion of the professionalism curriculum in Foundations of Patient Care is sustained and enhanced. Additional focus on professional skill development and professionalism is part of a longitudinal “intersession” course that occurs between core clerkship blocks. The thematic focus of this intersession series includes ethics and personal reflection on professional development. Students also have longitudinal clinical placements throughout their core clerkships year, in which professionalism is formally assessed. This addresses in part the concern that observation and assessment in the clinical setting have become fragmented.
Advisory colleges, composed of clusters of students and faculty members, have recently been established to enhance professional and personal growth. Designated mentors are responsible for discrete groups of students, and will attend to individual students' learning and advising needs. It is anticipated that the advisory colleges' faculty will develop specific curricula to meet these objectives.
At the UCSF School of Medicine, the admission process is independent of the associate deans for curricular and student affairs. Conversation is under way with the admission committee on the enhanced commitment of the school of medicine to teach and value professionalism standards.
In courses where there are rich opportunities to assess attributes of professionalism, we undertook explicit faculty development, particularly for small-group facilitators. Faculty are instructed about the professionalism-evaluation system and assured that their input is heard and acted upon. Emphasis is on the individualized opportunity to identify, as early as is feasible, students who are in need, and to remediate effectively. This addresses some longstanding concerns that such observations are inadequately addressed by the administration.
STUDENTS' RECOURSE AND LEGAL RAMIFICATIONS
Students and faculty must perceive the professionalism-evaluation system as fair and accurate. A “safety net” process, which had evolved after we instituted the system for third-and fourth-year students, was expanded to the first two years. An important protection for students against inappropriate submission of Physicianship Evaluation Forms is that only course directors can submit these forms. Issues that are picked up by preceptors or section leaders are referred to the course director, who must investigate the issues for their validity. Course directors are generally more knowledgeable about this evaluation system through discussions at quarterly promotions committee meetings and interactions with the associated dean for student affairs. Also, course directors may be more skilled at giving effective feedback to students or faculty members. A concrete deadline for submission of a Physicianship Evaluation Form is in place; students are not held in indefinite “jeopardy” by a course. School of Medicine faculty ombudspersons can facilitate informal resolutions of disagreements between students and course directors over the submission of a form. Last, students can formally grieve the submission of a Physicianship Evaluation Form using the same process that would be used to grieve a course grade.
There is ample legal precedent to support our professionalism-evaluation system. Student dismissal from medical school can be either academic or disciplinary. Academic dismissals involve the professional evaluation of a student's academic and/or clinical performance. Disciplinary dismissals involve fact-finding regarding violations of institutional rules or policies. Courts treat academic dismissals and disciplinary dismissals differently in significant ways. The courts accord substantial deference to the faculty's professional academic judgment in an academic dismissal case, and due-process issues are minimized.10 Supreme Court Justice Powell wrote that “university faculties must have the widest range of discretion in making judgment as to the academic performance of students and their entitlement to promotion or graduation.”10,11 Professionalism is a core characteristic of the profession of medicine and is a fundamental component of clinical competency.8,12,13 Therefore, in most instances where unprofessional behaviors warrant dismissal from medical school, the dismissals can be considered “academic” rather than “disciplinary.” Some faculty members and administrators may have concerns about the legal consequences of writing negative subjective evaluations on students that could lead to dismissal from medical school. Irby urges that faculty should be reassured that they have “nothing to fear from the courts and that they should uphold high academic standards.”10 This physicianship-evaluation system provides a systematic mechanism for documentation and longitudinal assessment of professional development deficiencies.
Some have questioned the legality or wisdom of prospectively informing course or clerkship directors about a student who has some previous difficulties. The courts have upheld that faculty members can evaluate students in difficulty in greater depth than other students and can prospectively alert other faculty to a problem student so that the faculty member can be more helpful and can do a more thorough job of evaluating performance.10 Yet, we do not disclose to new faculty or residents who will interact with and evaluate a student that that student is having difficulties, for fear of “labeling” the student. Our policy is to inform only course or clerkship directors of a prospective student with difficulties so that the best available educational environment can be crafted for that student, such as placement with an experienced preceptor or at a rotation site with a highly structured curriculum. The course or clerkship director then makes generalized inquiries into the student's performance at appropriate intervals.
LESSONS LEARNED AND CONTINUING ISSUES
We have learned that the faculty has embraced this evaluation system, and that within a year of its use, it became part of the culture of the UCSF School of Medicine. What remains difficult is the intensity of the student response that can occasionally be engendered with the submission of the Physicianship Evaluation Form. It has been important to include those faculty members who signal concerns in the planning for remediation, as students commonly return to question faculty about the merits of their reported assessment. Student acceptance has been generally favorable. Even most students who receive a Physicianship Evaluation Form agree with the need for such an evaluation system, yet may disagree that their actions warranted a submission.
We have also learned about the need for a realistic deadline for submission of the Physicianship Evaluation Form in this climate of mobility and complexity in clinical placements. In order to provide timely feedback to students, the initial deadline for submission of a Physicianship Evaluation Form in the third and fourth years was two weeks after the end of a clerkship. However, there were several instances where a student should have received a Physicianship Evaluation Form but evaluation submission and direct communication with preceptors could not be accomplished within the two-week deadline. This deadline was particularly difficult for those clerkships that had students in multiple sites or with multiple evaluators. The policy has now been changed so that the submission deadline for a Physicianship Evaluation Form throughout the four years is eight weeks after the start of the student's subsequent rotation or course.
As with Student C, there are sometimes hard lessons a student needs to learn, but they may not be at the level of justifying a formal Physicianship Evaluation Form. However, such students do need feedback on their behaviors.
A continuing issue is how to capture unprofessional student behaviors that occur outside the confines of a course or rotation. “Over the top” reactions of a student about grades, or about the associate dean's choice of the adjective used to describe the student in the dean's letter of recommendation for residency programs, are examples of such behaviors. Other examples include adverse interactions in the registrar's or financial aid office, or with core administrative staff. We also consider inadequate immunization status, despite repeated reminders from the school, as unprofessional behavior. It may be that we will expand our professionalism evaluation system to transcend its current boundaries.
Student mistreatment by residents is an unfortunate reality.14,15,16 Our students' responses over the years to the Association of American Medical Colleges Graduation Questionnaire document that some residents display unprofessional behaviors toward students. To address student mistreatment by residents, the associate dean for student affairs is working with the associate dean for graduate medical education and the residency directors. Our proposal is to expand the professionalism-evaluation so that students can submit a version of the Physicianship Evaluation Form when residents mistreat them. We are hopeful that the existence of such a system will act as a deterrent to unprofessional behaviors by residents and will help to identify residents in need of remediation. The challenge is to create a system that the students will perceive as safe for them. Concerns include to whom the forms should be submitted. Many students wish to stay at the UCSF School of Medicine for residencies. Residency directors chair residency selection committees. A student may be reluctant to submit the form to these faculty for fear of being labeled a “troublemaker.”
Our future plans are to track the performances of students who have received Physicianship Evaluation Forms throughout medical school and residency. We wish to determine whether problems with professional development in medical school are predictive of problems in residency training.
Traditionally, medical education has placed its highest values on the attainment of scientific knowledge and clinical problem solving. We believe that the evaluation process described in this article raises awareness among students and faculty members about the core value of professionalism. It also demonstrates the importance that the institution places on professionalism development. Most of our curriculum on professionalism occurs during the first two years of medical school. Expansion of our professionalism-evaluation system to encompass the first two years reinforces the importance of the concurrent curriculum on professionalism.
The school has created an administrative structure whose goal is remediation of students' professionalism deficiencies. However, if remediation is unsuccessful, academic dismissal can occur even if the students have obtained adequate scientific knowledge and problem-solving skills, as demonstrated by receiving passing grades in all courses.