Secondary Logo

Journal Logo

Research Reports

Resuscitating the Socratic Method: Student and Faculty Perspectives on Posing Probing Questions During Clinical Teaching

Abou-Hanna, Jacob J. MD; Owens, Sonal T. MD; Kinnucan, Jami A. MD; Mian, Shahzad I. MD; Kolars, Joseph C. MD

Author Information
doi: 10.1097/ACM.0000000000003580

Abstract

Questioning has played an important role in the education process dating back to ancient times. The Socratic method, a form of systematic questioning from a teacher, draws on the learner’s knowledge to guide understanding, affirmation of baseline knowledge, or deeper questioning.1 When done correctly, this form of questioning should engage the learner in critical thinking and promote self-directed learning. When done poorly or misinterpreted by the learner, questioning can be viewed negatively, with concerns that this practice has been a means of minority marginalization and is more favored by male faculty.2,3 In 1989, Brancati described the term “pimping” in medicine as the faculty physician (the pimper) posing a series of difficult questions to a resident or medical student (the pimpee).4,5 Though the colloquial definition of pimp—a criminal who lives off the earnings of prostitutes6—was never mentioned, this satirical piece by Brancati defined a transformed, ill-intentioned version of a time-tested teaching method designed to embarrass the learner and reinforce medical hierarchy rather than promote learning.7

With increasing attention on mistreatment and lack of civility during medical student education,8–11 the blurred lines between the well-intentioned Socratic method and poorly received “pimping” may have faculty hesitating to ask questions.12 One medical student noted a resident’s hesitancy to use questions to teach and disclosed that faculty had forbidden the practice.13 Accusations of maltreatment may leave educators feeling particularly vulnerable when institutions are more likely to side with learners, which leaves faculty feeling as if they are walking on eggshells when it comes to teaching medical students.14 Anecdotally, we have heard passing comments from faculty at our own institution about a hesitancy to pose questions to clerkship students for fear of misperceived intent.

The study we describe here was constructed for 2 purposes. The first was to understand the student perspective on the role of questions from faculty during a core clerkship. The second was to understand the faculty perspective, including their perceptions of student preferences (i.e., What do faculty think that students think?). We hypothesized that there is a gap in perception between faculty and students that can be remedied by explicitly stating the purpose of posing questions.

Method

Setting

To provide a focused environment, students from the University of Michigan Medical School were instructed to reflect on their inpatient service assignments during their internal medicine clerkship rotation. These required rotations took place at either a university or Veterans Affairs (VA) hospital. The internal medicine rotation was chosen because it is the longest rotation during medical students’ core clerkship year.

Participants

Eligible participants were medical students from the 2020 graduating class (n = 165) who had completed their internal medicine clerkship during their second year of medical school. At the time of the survey, students had completed their clerkship year 7 months prior and taken the United States Medical Licensing Examination (USMLE) Step 1 exam 5 months before completing the survey. Faculty participants (n = 144) were internal medicine faculty who directly supervised at least one of the eligible student participants on an inpatient service. Students were exposed to an individual faculty member for 1 to 3 weeks.

Surveys

Surveys for this study were constructed using an iterative process. We drafted questions informed by anecdotal comments from faculty and students and tested them with 10 students and 8 faculty for clarity, relevance, and suggested revisions until inputs did not provide any new dimensions or questions. One 19-item survey for students was created (see Supplemental Digital Appendix 1 at http://links.lww.com/ACADMED/A991) with a similar 22-item survey for faculty (see Supplemental Digital Appendix 2 at http://links.lww.com/ACADMED/A991). In addition to demographic questions, survey items required participants to rate how much they agreed or disagreed with a statement using a Likert scale of 1–5. We avoided the term “pimp” in the survey. Instead, we introduced the term “probing questions,” which we defined with the following statement: “This survey aims to understand your perspectives on (being asked/asking) questions that ‘probe’ medical knowledge and concepts, from now on referred to as ‘PROBING QUESTIONS.’” The surveys included examples of what did and did not constitute a probing question.

The surveys were distributed and collected anonymously in April 2019 via the electronic Qualtrics XM platform (Qualtrics, Provo, Utah). Students and faculty were given 1 week to complete the survey, and we sent a total of 3 email reminders to each cohort. Students were given a $5 gift card for completing the survey. Internal funding was provided by the University of Michigan Medical School Dean’s Office.

Data analysis

Likert scale responses were converted to a numerical value (i.e., strongly disagree = 1, strongly agree = 5). We performed 2-sample t tests using Microsoft Excel Data Analysis ToolPak (Microsoft Corporation, Redmond, Washington) to compare survey response means between the 2 groups. Linear regression was performed to assess for correlation between survey items, using R for Windows 3.6.0 (The R Foundation, University of Auckland, Auckland, New Zealand). While the survey responses were anonymous, we were able to view all responses from a single participant, which allowed for a more in-depth analysis—including regression analysis between different survey items.

Institutional review board approval

This study was deemed exempt from review by the University of Michigan Medical School and Michigan Medicine institutional review board.

Results

A total of 140/165 students (85%) and 112/144 faculty (78%) completed the surveys. Demographic information for faculty participants is listed in Table 1. Demographic information for the medical student class of 2020 was obtained from the medical school administration office; it included 73 (44%) males and 92 (56%) females with an average age of 27 (range 23–41).

Table 1
Table 1:
Demographic Characteristics of Faculty Respondents to a Survey Regarding Probing Questions During Clinical Teaching, University of Michigan Medical School, April 2019

Table 2 displays mean responses (strongly disagree = 1, strongly agree = 5) to survey items and comparisons of means between groups. Of 112 faculty participants, 109 (97%) agreed that probing questions are valuable for student education, but only 73 of 111 faculty (65%) perceived that students agreed that probing questions are valuable (item 1, P < .001). However, the value students placed on questions was significantly higher than faculty perceived it to be (125/140 [89%] students agreed with the statement vs 73/111 [65%] faculty, item 1, P < .001), with only 8/140 (6%) students disagreeing that probing questions are valuable. Similarly, faculty underestimated students’ preference for more questions (98/140 [70%] students agreed vs 45/112 [40%] faculty, item 2, P < .001) and for being asked too many questions versus no questions (115/140 [82%] students agreed vs 67/112 [60%] faculty, item 3, P < .001).

Table 2
Table 2:
Means and Standard Deviations of Student and Faculty Responses to Select Survey Items Regarding Probing Questions During Clinical Teaching, University of Michigan Medical School, April 2019

Of 140 student participants, 55 (39%) agreed that they feel humiliated when they answer a question incorrectly, and the mean response to how faculty perceived students’ feelings was similar (item 4, P = .18). In contrast, faculty overestimated both students’ preference for questions to stop if they answer a question incorrectly (20/140 [14%] students agreed vs 25/110 [22%] faculty, item 5, P = .02) and the likelihood of students providing a negative evaluation of faculty who ask probing questions (1/140 [1%] student agreed vs 21/111 [19%] faculty, item 8, P < .001). Only 7/140 (5%) students agreed that faculty ask probing questions to humiliate them (item 6).

Of 140 student participants, 57 (41%) agreed that their formal evaluation from faculty was primarily based on their ability to correctly answer probing questions, but only 21/111 (19%) faculty agreed with this statement (item 9, P < .001). Lastly, 86/112 (76%) faculty agreed that it is difficult to assess students’ competency without asking probing questions (item 10).

On the student survey, a linear regression with responses to items 1 (Probing questions are a valuable component of student education) and 7 (Attendings ask students questions to assess for teaching opportunities) showed a strong association (R = 0.65, 95% confidence interval 0.55–0.74, P < .001).

Survey response means were compared between different demographic groups. For the faculty survey, this included gender (male [n = 70] vs female [n = 37]), age (≤ 50 years old [n = 70] vs > 50 years old [n = 39]), experience as a faculty physician (≤ 20 years [n = 76] vs > 20 years [n = 35]), faculty rank (instructor/lecturer/assistant professor [n = 59] vs associate professor/professor [n = 51), and faculty who only attend on VA general medicine service (n = 19) vs faculty who only attend on university general medicine service (n = 17). Faculty older than 50 and with a rank of at least associate professor more strongly agreed that their evaluation of a student is primarily based on the student correctly answering probing questions as compared with faculty younger than 50 (response mean 2.74 vs 2.17, P = .01) and with a rank of assistant professor, lecturer, or instructor (response mean 2.60 vs 2.17, P = .04), respectively. No other comparisons—including gender—achieved statistical significance.

Student survey response means were compared by clerkship grade (nonhonors [n = 97] vs honors [n = 41]) and USMLE Step 1 exam score (≤ 250 [n = 74] vs > 250 [n = 41]). Students who received honors more strongly disagreed that faculty ask questions to humiliate students (response mean 1.24 vs 1.58, P = .01), more strongly agreed that questions were to assess for teaching opportunities (response mean 4.56 vs 4.26, P = .02), and more strongly preferred faculty who ask increasingly more challenging questions (response mean 4.37 vs 3.86, P < .01). Of note, honors is awarded to the top 20%–30% of a rotation cohort. Students who scored at least 250 on the USMLE Step 1 exam more strongly agreed that probing questions are valuable (response mean 4.65 vs 4.39, P = .048), more strongly disagreed that they feel humiliated when they answer a question incorrectly (response mean 2.54 vs 3.01, P = .04), and more strongly disagreed that faculty ask questions to humiliate students (response mean 1.24 vs 1.54, P = .03). No other comparisons achieved statistical significance.

Discussion

The practice of questioning as a method of clinical teaching has been criticized, especially so since the effects of humiliation have become more recognized.10 Of note, the 2018 Association of American Medical Colleges Medical School Graduation Questionnaire15 revealed that 40% of students reported at least one incident of humiliation. The value of “questioning” and its relationship to humiliation have been unclear. Wear and colleagues7 interviewed 11 fourth-year medical students and showed that all students had positive feelings about “pimping” as a learning tool. Zou and colleagues16 conducted a survey-based study of 74 radiology students that showed 81% preferred learning radiology via interactive dialogue with the teacher and 73% thought “pimping” was an effective learning method. Both Wear and colleagues and Zou and colleagues highlight a major limitation in related studies3 regarding the use of the term “pimping.” Though it is often assumed to be synonymous with the Socratic method among medical trainees,17 “pimping,” by definition, has a negative connotation that can likely prejudice survey participants. Our study aimed to address this limitation by soliciting unbiased perspectives (i.e., not using the word pimping) and sampling a larger and more representative student group about their experiences with the internal medicine clerkship—a rotation common to all medical students. Additionally, we explored faculty impressions of students which have not been addressed in the literature.

In this report, we documented a number of key findings. First, almost all students and faculty value probing questions for clinical learning. In fact, most students preferred more questions than they are currently being asked and would rather be asked too many questions than no questions. Second, faculty underestimated students’ valuation of probing questions and their desire to be asked questions. This disconnect is further illustrated by the fact that students disagreed that receiving questions from faculty would lead them to submit an unfavorable evaluation, in contrast to significantly more concern reported by faculty. Third, faculty agreed that it is difficult to evaluate a student’s competency without asking probing questions. Altogether, these items suggest that faculty should not hesitate to ask students probing questions; not doing so may please the minority at the expense of teaching the majority and properly evaluating students’ competency. Probing questions are a real-time option for faculty to stimulate learners during discussions of their own patients, offering medical students a valuable opportunity to think on their feet—a skill which is at the core of clinical practice. However, not all students may understand or agree with this practice.

In this cohort, 8 (6%) students felt that probing questions are not valuable. Although they were a small subset of all students who participated, faculty may be hesitant to ask questions if they suspect they will be poorly received by even one student. We speculate that the value students put on probing questions is related to the perceived purpose of the questioning. In other words, if students suspect that faculty are asking questions with bad intent, they may interpret the whole experience as negative. This is further suggested by the moderate to strong correlation between items 1 (Probing questions are a valuable component of student education) and 7 (Attendings ask students questions to assess for teaching opportunities) in Table 2. Additionally, 5 of the 8 students (63%) who do not find probing questions valuable (item 1) are the same 5 students (of 6, 83%) who do not think faculty ask questions to assess for teaching opportunities (item 7). Lastly, students more strongly agree than faculty that their evaluation from faculty is primarily based on correctly answering probing questions (item 9, P < .001). This may suggest that students think probing questions are more for assessment than learning. We speculate that explicitly stating the purpose of probing questions may result in better reception by students.

Lastly, our data show that a significant portion of students (55/140 [39%]) feel humiliated when they answer a question incorrectly (item 4). Faculty seem to be aware of this phenomenon, given that there is no significant difference between student and faculty responses to this item. What remains unknown is how much of the humiliation students attribute to faculty. This is not within the scope of this study, but we can make some hypotheses based on a few other survey items. Despite 39% of students feeling humiliated when they answer a question incorrectly, only 20/140 (14%) students want questions to stop if they answer a question incorrectly (item 5) and only 7/140 (5%) students agree that faculty ask them questions to humiliate them (item 6). These findings inspire 2 key questions worthy of future exploration: (1) Can the discomfort of “not knowing” be explicitly reframed so as to mitigate feelings of humiliation? and (2) Can faculty role model or discuss their own knowledge gaps so as to normalize “not knowing” while constructively emphasizing what needs to be learned?

There are several study limitations worth noting. First, the generalizability of the data is limited since this is a single institution study. Similarly, the surveys focused only on internal medicine inpatient rotations. Although this allows for a low inference among student participants when considering the specific learning environment, it does not capture student and faculty perspectives across all departments. In fact, many student participants suggested they received even fewer questions on other rotations. Second, participating students took this survey 7 months after completing their final clerkship rotation and 5 months after completing the USMLE Step 1 exam. It is possible that student perspectives may have been different after completing these medical school milestones as compared with students still on their clerkships. The literature would strongly benefit from expansion of this survey to other institutions, disciplines, and phases of medical training, including preclinical, clinical, and postgraduate levels.

Additionally, by surveying only one medical school class and one faculty department, we were limited on the demographic information that we could solicit, notably gender, race, and ethnicity, while maintaining participant anonymity. Only the gender information of faculty participants was collected, and it showed no difference in survey responses between males and females, suggesting broad faculty preference for this teaching method regardless of gender. Importantly, these aspects require further exploration given that previous studies have demonstrated gender, race, and ethnic biases in faculty at academic institutions.18–20

Despite these limitations, we surveyed a larger cohort of students than previously reported, and our study was the first to compare the perspectives of students with those of their faculty. The large majority of students prefer probing questions, which argues against faculty retreating from the use of the Socratic method. However, our review of the literature did not reveal any studies that explored approaches to make the purpose of the questioning process more apparent and to minimize feelings of humiliation when correct answers are not provided. These approaches, as well as their outcomes, are worthy of future exploration.

Acknowledgments:

The authors wish to thank Dr. Cyril Grum, Dr. Daniel Cronin, and Mr. Erkin Otles for their input on crafting the survey and recruiting participants.

References

1. Oh RC, Reamy BV. The Socratic method and pimping: Optimizing the use of stress and fear in instruction. Virtual Mentor. 2014;16:182–186.
2. Chen DR, Priest KC. Pimping: A tradition of gendered disempowerment. BMC Med Educ. 2019;19:345.
3. McEvoy JW, Shatzer JH, Desai SV, Wright SM. Questioning style and pimping in clinical education: A quantitative score derived from a survey of internal medicine teaching faculty. Teach Learn Med. 2019;31:53–64.
4. Brancati FL. The art of pimping. JAMA. 1989;262:89–90.
5. Detsky AS. The art of pimping. JAMA. 2009;301:1379–1381.
6. Definition of PIMP. https://www.merriam-webster.com/dictionary/pimp. Accessed May 20, 2020.
7. Wear D, Kokinova M, Keck-McNulty C, Aultman J. Pimping: Perspectives of 4th year medical students. Teach Learn Med. 2005;17:184–191.
8. Fried JM, Vermillion M, Parker NH, Uijtdehaage S. Eradicating medical student mistreatment: A longitudinal study of one institution’s efforts. Acad Med. 2012;87:1191–1198.
9. Heru A, Gagne G, Strong D. Medical student mistreatment results in symptoms of posttraumatic stress. Acad Psychiatry. 2009;33:302–306.
10. Mavis B, Sousa A, Lipscomb W, Rappley MD. Learning about medical student mistreatment from responses to the medical school graduation questionnaire. Acad Med. 2014;89:705–711.
11. Scott KM, Caldwell PH, Barnes EH, Barrett J. “Teaching by humiliation” and mistreatment of medical students in clinical rotations: A pilot study. Med J Aust. 2015;203:185e.1–185e.6.
12. McCarthy CP, McEvoy JW. Pimping in medical education: Lacking evidence and under threat. JAMA. 2015;314:2347–2348.
13. van Schaik KD. A piece of my mind. Pimping Socrates. JAMA. 2014;311:1401–1402.
14. Gold MA, Rosenthal SL, Wainberg ML. Walking on eggshells with trainees in the clinical learning environment—Avoiding the eggshells is not the answer. JAMA Pediatr. 2019;173:907–908.
15. Association of American Medical Colleges. Medical school graduation questionnaire: 2019. All schools summary report. https://www.aamc.org/system/files/2019-08/2019-gq-all-schools-summary-report.pdf. Accessed June 16, 2020.
16. Zou L, King A, Soman S, et al. Medical students’ preferences in radiology education: A comparison between the Socratic and didactic methods utilizing PowerPoint features in radiology education. Acad Radiol. 2011;18:253–256.
17. Stoddard HA, O’Dell DV. Would Socrates have actually used the “Socratic method” for clinical teaching? J Gen Intern Med. 2016;31:1092–1096.
18. Morgan HK, Purkiss JA, Porter AC, et al. Student evaluation of faculty physicians: Gender differences in teaching evaluations. J Womens Health (Larchmt). 2016;25:453–456.
19. MacNell L, Driscoll A, Hunt AN. What’s in a name: Exposing gender bias in student ratings of teaching. Innov High Educ. 2015;40:291–303.
20. Osseo-Asare A, Balasuriya L, Huot SJ, et al. Minority resident physicians’ views on the role of race/ethnicity in their training experiences in the workplace. JAMA Netw Open. 2018;1:e182723.

Supplemental Digital Content

Copyright © 2020 by the Association of American Medical Colleges