Lingemann, Kerstin; Campbell, Teresa; Lingemann, Christian; Hölzer, Henrike PhD; Breckwoldt, Jan MD, MME
Simulated patients (SPs), also widely known as standardized patients, play an important role in preparing medical students for clinical encounters.1,2 Accordingly, numerous studies have evaluated aspects of teaching with SPs. Topics such as the quality of SPs' instructiveness and feedback,3–5 quality of SP performance,6 and use of SPs in assessment7 have been addressed. However, to the best of our knowledge, there are no systematic analyses of the teaching encounter from the perspective of the SP. In this study, we wanted to analyze the SP's perception of this encounter to gain information on how to improve teaching quality. Results could be incorporated in faculty development programs to promote the most effective teaching while posing as little distress on the SP as necessary.
We focused on SP teaching sessions that were conducted from April to November 2008 as a compulsory component of the standard curriculum at Benjamin Franklin Medical Centre of Charité–University Medicine Berlin. In Germany, students begin medical school after high school. The first two years of undergraduate training include predominantly basic science learning, and clinical subjects are introduced at the beginning of the third year. We analyzed 50-minute clinical teaching sessions delivered during an emergency medicine course at the end of third-year undergraduate training. Each session was delivered to four to six students by clinical teachers from anesthesiology or cardiology, with teaching experience ranging from 1 to 15 years.
Teaching sessions focused on one of two SP roles: (1) a 50-year-old man with acute coronary syndrome (ACS), or (2) a 30-year-old woman with acute dyspnea and a history of allergic asthma. SPs were trained by the Simulated Patients Program of Charité–University Medicine Berlin according to international standards.8 Classrooms were equipped with standardized material (ECG monitor, prepared pathologic 12-lead ECG tracings, pulse oximeter, oscillometric blood pressure monitor, equipment for physical examination, oxygen masks and supply, and mock medications). The clinical teachers received standardized verbal and written instructions beforehand, containing the SP's role description and the procedures to which SPs had agreed or disagreed (e.g., taking blood). During the lessons, the students' task was to take the history, perform physical examination, and initiate treatment.
With the permission of all persons involved, the lessons were videotaped. All students and teachers had the option of declining to be recorded at any time during the session. They were given the option to view the video after the lesson. The study was approved by the ethical committee of Charité–University Medicine Berlin.
In attaining SP feedback about the teaching sessions, we wanted to minimize cognitive patterns which the SPs might have already adopted (i.e., specific feedback rules). We decided to use Think Aloud (TA) methodology, by which thoughts and feelings during a given task may be made accessible for analysis.9–11 During the TA session, the participant is instructed to speak aloud what comes to his or her mind with respect to a specific question, without filtering the content. Verbalizations are recorded, transcribed, and analyzed by qualitative methods. In the case of teaching, however, TA is impossible to perform without disturbing the teaching process, so sessions were video recorded, and retrospective TA (rTA)9–11 on the recording was performed later.
To obtain an adequate amount and richer quality of information from the SP participants, we wanted the teaching task to be on a higher level of difficulty. We chose to analyze emergency medicine sessions because in this setting students simultaneously take the SP's history, perform a physical examination, and decide on initial treatment.12
To attain SP feedback, we used rTA on the video recording of the teaching session. We selected the following specific standardized sequences within the recordings for analysis: (1) introduction of the session by the teacher (one to two minutes), (2) scenes from history taking (three minutes), (3) examination (three minutes), (4) treatment (three minutes), and (5) feedback by the teacher and the SP (one to two minutes), if available. In all cases, we selected the first part of the sequence.
At the end of the term in August through December 2008, SPs were invited back to conduct rTAs without specifically having been informed about this project beforehand. All SPs were able to accept the invitation. The mean time elapsing between teaching encounter and rTA was three months.
To start the rTA session, all SPs received a short, standardized introduction to the method. Then they practiced thinking aloud on a video from a standard teaching session for a maximum of 12 minutes until they felt comfortable with the method. Finally, they performed rTA on the selected sequences from their own teaching sessions, in total 11 to 13 minutes.
They were instructed to speak out loud thoughts that came to their mind without structuring or filtering in response to the question “What made you feel comfortable during the teaching session and what made you feel uncomfortable?” If the flow of talking discontinued, the SP was encouraged to proceed with verbalizations. During all rTA sessions, the same person (K.L.) was present to give instructions. No other person was in the room so as not to disturb the flow of thoughts. SPs who had had teaching sessions with more than one teacher watched each video on a separate occasion to minimize direct comparisons between the teachers.
The SPs' comments were audiotaped and transcribed. All transcripts were blinded for analysis and then processed using qualitative content analysis (QCA).13 During QCA, four members of the research group (K.L., C.L., T.C., J.B.) analyzed 10% of the transcripts to identify categories and subcategories related to the question “When did the SP feel comfortable (or uncomfortable) during the course of the teaching session?” If the group initially disagreed on a category, consent was found by discussion. Subsequently, we classified all comments from the rTA sessions according to these categories and whether each comment was positive, negative, or neutral.
We also extracted all comments that implied a significant influence of the teacher, whether the comments were directly or indirectly related to the teacher. For adequate sample size, we predetermined saturation of information to be reached at 15 rTAs, but we performed more rTAs to be able to account for the two different SP roles and to gain enough information on the teacher.
We conducted rTAs on 28 teaching sessions (14 for each SP role). In total, 23 teachers and 11 different SPs participated. In one case, a teacher and an SP were matched twice. We excluded the second of these protocols because the data might have been influenced by the first encounter. Using QCA, we extracted 283 relevant statements from the remaining 27 transcripts. Fourteen statements classified as neutral were not included in the final analysis. Of the remaining 269 comments, 165 were negative, and 104 were positive.
Categories of comments
Comments were sorted into the six main categories identified by the research group: communication of students (118), actions taken by students (56), clinical teacher (33), atmosphere (24), behaviors of medical students (19), setting (16), and others (3). For an overview of all categories and subcategories, see Table 1. We provide a brief summary of the most frequent comments in Table 2.
Communication of students.
Most comments (n = 118) referred to communication. They were distributed to six subcategories as follows: attentiveness, history taking, physical contact, student as contact person, language, and body language. SPs expressed their comfort if the student introduced himself or herself, was attentive and caring, and used appropriate physical contact.
SPs also appreciated if the student provided clear personal contact and talked in a calm and clear manner. In total, we identified 38 positive comments in this category. SPs noted that “[t]he way he's [student] holding my hand is calming me down,” or “[t]he body language is great … shows concentration and alertness.”
We identified 80 negative comments in this category. SPs felt uncomfortable when they were excluded from communication, or when they perceived that students ignored their specific needs. For example, one participant observed, “No one is paying any attention to me; they are just talking among themselves,” and another noted, “I was confused because the person I had been talking to the entire time suddenly left to get the blood pressure cuff.”
SPs also felt uncomfortable when the students all talked at the same time, asked too many questions at once, or communicated in an inappropriate tone or fashion. One participant noted feeling uncomfortable when “[t]hey're asking me so many questions at the same time that I actually can't answer.” Another participant observed, “She [the student] is talking very loudly; it feels like being screamed at.”
Finally, SPs expressed their discomfort with too much physical contact—noting, for example, “She [the student] keeps patting my arm, which I really don't want at that moment.” On the other hand, SPs were uncomfortable if students' body language conveyed disinterest or lack of confidence. One participant observed, “If you look at their body language … it's difficult if someone stands in front of you with one hand on his back and the other playing with the necklace all the time.”
Actions taken by the student doctor.
This category included 56 comments. SPs made positive comments when students explained the actions they were taking and when students were able to give information on the medical condition. For example, “It's very good how he [the student] always announces what the effects [of the medication] could be.” Also, SPs felt comfortable if the consultation was well structured by the students and if they felt that students reacted to their symptoms within an adequate time: “Very good … they're taking the most important measures first.”
SPs expressed discomfort if the students' actions were unstructured or if the SP felt the student forgot aspects of the treatment. One SP noted, “This doesn't seem very structured; I don't really know who the doctor is and who the nurse is.” Another observed, “This is going too slowly. I really need help and they're not doing anything!”
Furthermore, SPs were uncomfortable if students did not know how to use the medical equipment properly: “The one who is treating me does not know how the equipment works; that's very irritating.”
Thirty-three comments directly referred to the performance of the clinical teacher. SPs felt comfortable if the clinical teacher was calm and attentive and guided the students if necessary: “The teacher seems very competent. If the students don't know what to do next she helps them without humiliating them.” They appreciated if the role-play was well structured by the teacher (i.e., provided introduction, feedback). In contrast, SPs felt uncomfortable if the teacher interrupted the role-play without a proper time-out, if he or she did not respect the SP's limits, and if he or she was unaware of feedback rules. For example, one SP noted, “Even though I told the teacher that I couldn't play my part this often [three times in 50 minutes,] he just continued.”
In the 24 comments in this category, SPs expressed their comfort with a calm atmosphere and unambiguous student behavior.
Behaviors of medical students.
Nineteen comments referred directly to the medical students. SPs felt comfortable if the students were well prepared for the lesson and open for feedback. Conversely, SPs disliked when the students did not stay in the role-play: “Don't laugh, come on! If I had been a real patient that would have really annoyed me.”
We identified 16 comments in this category. SPs valued time to discuss the schedule of the course with the teacher before the class began. They disliked if they could not properly prepare themselves for the role-play, if the teacher did not give clear instructions before the role-play began, or if there was no clear ending of the role-play. Characteristic comments in this category were “It was very important to me to agree on the setting before the class begins” and “What was difficult here was that I didn't get a break after the role-play to prepare my feedback.”
Comments reflecting a potential influence of the teacher
One objective of this study was to describe quality of teaching from the SP perspective. Therefore, we identified all comments in which behaviors of the clinical teacher influenced the comfort of the SP. Apart from the 33 comments directly related to the teacher, we found another 111 indirect comments which were indirectly influenced by teachers' behavior. These indirect comments were spread over all the categories, focusing on teachers' support of the students during role-play, giving a clear structure to the role-play, and communicating with the SP about the teaching session in advance. In summary, 54% of all comments (144 of 269) were directly or indirectly related to the teacher. An overview of these is shown in Table 3.
Ratio of positive to negative comments
Overall, the SPs made more negative than positive comments (ratio, 1.6:1). This was especially the case for the categories “communication” (2.1:1) and “setting” (7.0:1). The other categories showed no significant predominance (1.3:1). With respect to teacher-related comments, SPs made significantly more negative comments than positive ones (1.6:1).
Differences between SP roles
When we compared rTA comments between the two SP roles (dyspnea and ACS), we did not find substantial differences (see Table 4). Statistical methods could not be applied because the number of specific comments was not high enough in most categories. Nevertheless, the distribution to categories and between negative and positive comments was very similar between SP roles. In only one category—comments directly related to the teacher—ACS SPs gave more positive comments than did dyspnea SPs. When we considered comments indirectly related to the teacher in addition to the directly related comments, there was no difference between the SP groups' positive and negative comments.
In this study, we found that SPs focus their attention on communication with the students and on the actions of the students. Sixty-five percent (174/269) of the comments were directly related to communication and students' actions. SPs expressed comfort with competent and well-prepared students, who were able to structure the consultation and to offer explanations to the SPs. Because SPs are trained to give this kind of feedback, this finding in itself is not surprising. However, these comments clearly state the necessity that students should be prepared before the teaching session to keep appropriate verbal and physical contact and to provide a transparent structure for the SP. This might also be important in communication with real patients.
We found SPs' comments on students' behavior and attitudes (n = 19) to be in agreement with the comments related to students' communication. SPs valued a calm atmosphere and appropriate physical contact, indicating that teachers should encourage their students to develop trust and empathy with the SP.
A remarkably high proportion of comments (54%; 144/269) were directly or indirectly related to the teacher, implying that he or she has substantial influence in the SP encounter. SPs highlighted the importance of a clear structure to the role-play (58 comments), which can only be provided by the teacher. SPs also expressed the desire to have the setting and schedule of the session clarified in advance. Again, this function to moderate the student–SP encounter lies within the responsibility of the teacher. Accordingly, cooperation between SP and teacher should be addressed during teacher training.
Agreement of negative and positive comments
When looking for contradicting positive and negative comments, we found no important disagreement. To the contrary, many positive comments were confirmed by corresponding negative comments. As an example, the positive comment “The way the student–doctor holds my hand is calming me down” may be matched with the negative comment “I was confused because the person I had been talking to … suddenly left to get the blood pressure cuff.” Similar observations can be made for some comments in Table 2.
Ratio of positive and negative comments
SPs made significantly more negative comments than positive ones in the category “communication” (2.1:1). A possible explanation might be that the third-year student participants were at the very beginning of their clinical education and still had much room for improvement. The second category with significantly more negative comments was the teaching setting. This might indicate that some of the teachers could benefit from specific training, for instance with respect to coordination and planning of the session together with the SP.
All SP participants successfully performed rTA. The analysis of the rTA protocols produced very detailed information on how SPs feel during the course of a lesson and what factors might influence these feelings. The data were less filtered than in other types of interviews, which might be due to rTA's distinct relation to real-time situations shown on the video. While performing rTA, SPs moved out of their didactic role of giving feedback, therefore providing more information about their personal feelings.
Regarding the two different SP roles, we found no relevant differences in the distribution of categories or negative versus positive rTA comments. This finding may support the robustness of our data.
We conducted the rTAs with 23 different clinical teachers and 11 different SPs to accommodate the existing course structure. Our aim was to gain as much rich qualitative information as possible, so we only excluded one rTA in which teacher and SP were matched a second time. Four teachers participated twice, but with different student groups, giving rise to various new teaching aspects during the rTAs. However, in these cases, the SPs' comments about the four teachers' sessions did not differ in general information.
Application to teaching with real patients
Clearly, transferring insights about SPs to real patients is difficult. It is very likely that SPs focus their attention on other aspects of the clinical encounter than do real patients with a manifest illness. SPs, for instance, will be much more sensitive to students' mistakes. Nonetheless, some of our results might also be applicable to teaching situations involving real patients, including providing a clear structure of the session, coming to an agreement with the patient on the course of the session, and promoting communication between students and patients.
This was a single-center study, so its generalizability to other institutions remains to be demonstrated. Also, the study was conducted in the context of emergency medicine teaching; other specialties may yield different findings. SPs' rTA feedback might be less pronounced in less complex teaching settings, or it might be more distinct in teaching encounters with more sensitive communication issues.
In this study, we identified factors that influence the comfort of SPs during teaching sessions in emergency medicine. SPs highly valued (1) a clear structure of the teaching setting, (2) an established agreement between SPs and teachers about the course of the session beforehand, and (3) students' preparedness for the teaching session. If clinical teachers wish to improve teaching from the perspective of SPs, they should prepare their students and SPs before the session and provide a clear teaching structure. These findings might serve as an aid for clinical teachers when working with SPs and might also be used in teacher training. Whether findings could be transferred to other medical specialties, or to teaching with real patients, remains to be investigated.
1. Cleland JA, Abe K, Rethans JJ. The use of simulated patients in medical education: AMEE guide no. 42. Med Teach. 2009;31:477–486.
2. Bokken L, Rethans JJ, Scherpbier AJ, van der Vleuten CP. Strengths and weaknesses of simulated and real patients in the teaching of skills to medical students: A review. Simul Healthc. 2008;3:161–169.
3. Nestel D, Kneebone R. Authentic patient perspectives in simulations for procedural and surgical skills. Acad Med. 2010;85:889–893.
4. Bokken L, Rethans JJ, Jöbsis Q, Duvivier R, Scherpbier AJ, van der Vleuten CP. Instructiveness of real patients and simulated patients in undergraduate medical education: A randomized experiment. Acad Med. 2010;85:148–154.
5. Bokken L, Linssen T, Scherpbier AJ, van der Vleuten CP, Rethans JJ. Feedback by simulated patients in undergraduate medical education: A systematic review of the literature. Med Educ. 2009;43:202–210.
6. Wind LA, van Dalen J, Muijtjens AMM, Rethans JJ. Assessing simulated patients in an educational setting: The MaSP (Maastricht Assessment of Simulated Patients). Med Educ. 2004;38:39–44.
7. Adamo G. Simulated and standardized patients in OSCEs: Achievements and challenges 1992–2003. Med Teach. 2003;25:262–270.
8. Wallace P. Coaching Standardized Patients: For Use in the Assessment of Clinical Competence. New York, NY: Springer Publishing; 2006.
9. Ericsson KA, Simon HA. Protocol Analysis: Verbal Reports as Data. Rev ed. Cambridge, Mass: Bradford Books/MIT Press; 1993.
10. Fonteyn ME, Kuipers B, Grobe SJ. A description of think aloud method and protocol analysis. Qual Health Res. 1993;3:430–441.
11. Young KA. Direct from the source: The value of ‘think-aloud’ data in understanding learning. J Educ Enquiry. 2005;6:27–32.
12. Aldeen AZ, Gisondi MA. Bedside teaching in the emergency department. Acad Emerg Med. 2006;13:860–866.
13. Mayring P. Qualitative Content Analysis [in German]. Weinheim, Germany: Beltz; 2008.
This study was approved by the ethical committee of Charité–University Medicine Berlin.