Secondary Logo

Journal Logo

Using a Simulation of a Frustrated Faculty Member During Department Chair Searches: A Proof of Concept Project

Shapiro, Daniel E. PhD; Abbott, Lisa M. MBA; Wolpaw, Daniel R. MD; Green, Michael J. MD; Levi, Benjamin H. MD, PhD

doi: 10.1097/ACM.0000000000001788
Innovation Reports
Free

Problem Vitae reviews, interviews, presentations, and reference checks are typical components of searches used to screen and select new department chairs/heads, but these strategies may fail to identify leaders who can communicate effectively with faculty in common, tense situations.

Approach Between May 2015 and November 2016, the authors piloted simulation scenarios in four department chair searches at Penn State College of Medicine/Penn State Health to assess candidates’ skill at handling common, challenging situations with faculty members. In the scenarios, a frustrated faculty member complains that he/she has too little time for academic pursuits. Candidates were provided the scenario approximately two weeks in advance. They were asked to explain their goals prior to the 10-minute simulation, do the simulation, and then debrief with the search committees, who observed the interactions.

Outcomes Approximately two-thirds (20/29; 69.0%) of candidates were judged to have successfully passed the simulation and were ultimately advanced. In most cases, the simulations revealed wide variation in candidates’ style, substance, and even underlying values that were not otherwise identified through the other parts of the recruitment and screening process. In some cases, candidates who performed well during group and individual interviews did poorly during simulations.

Next Steps The authors will build a larger pool of simulation scenario cases, create a rubric, and formally measure interrater reliability. They will study whether the strategy successfully identifies chairs who will be skilled at navigating common faculty challenges, and if this skill results in greater faculty satisfaction, engagement, and retention.

D.E. Shapiro is vice dean for faculty and administrative affairs, Penn State College of Medicine, Hershey, Pennsylvania; ORCID: http://orcid.org/0000-0002-7166-7705.

L.M. Abbott is senior vice president for human resources, Lifespan, Providence, Rhode Island.

D.R. Wolpaw is professor, Departments of Medicine and Humanities, Penn State College of Medicine, Hershey, Pennsylvania; ORCID: http://orcid.org/0000-0002-7567-2034.

M.J. Green is professor, Departments of Medicine and Humanities, Penn State College of Medicine, Hershey, Pennsylvania.

B.H. Levi is professor, Departments of Pediatrics and Humanities, Penn State College of Medicine, Hershey, Pennsylvania; ORCID: http://orcid.org/0000-0003-2380-7109.

Funding/Support: The Arnold P. Gold Foundation supported this work.

Other disclosures: None reported.

Ethical approval: This project was deemed to not be human subjects research by the Penn State College of Medicine Institutional Review Board.

Previous presentations: An earlier version of this manuscript was presented as a poster at a Graduate Faculty Affairs Conference sponsored by the Association of American Medical Colleges in Vancouver, British Columbia, on July 15, 2016.

Correspondence should be addressed to Daniel E. Shapiro, Penn State College of Medicine, 500 University Dr., Hershey, PA 17033; telephone: (717) 531-8779; e-mail: Shapiro@psu.edu; Twitter: @drdanshaps.

Back to Top | Article Outline

Problem

Department chairs are critically important to the governance of medical schools and teaching hospitals. As of November 2016, there were 145 MD-granting schools in the United States, with approximately 3,500 department chairs managing roughly 160,000 full-time faculty members, or about 1 chair for every 45 faculty members.1 In addition to faculty, chairs may also manage advanced practice clinicians, co-manage nurses, and support staff and trainees. In short, the responsibilities and human relations complexity of these positions are enormous.

In 2004, Grigsby and colleagues2 noted that emotional competence and communication skills are critical qualities of future-oriented department chairs. Indeed, new chairs are often surprised at the intense interpersonal issues they must manage. A former division chief, vice chair, or research lab director accustomed to managing 15 or fewer people may suddenly find him- or herself the final arbiter for 50, 100, or even 500 professionals. At any given time, a portion of these professionals will be disgruntled, a handful may be contemplating a change in work portfolios that could significantly disrupt key department functions, and a few may be experiencing serious personal or professional crises.

Structured queries of administrative stakeholders including deans and hospital chief executive officers have revealed that, from their perspective, physician–leaders frequently lack the fundamental administrative and people skills required for their positions.3 Interviews with sitting department chairs have confirmed that many feel unprepared for the interpersonal challenges they encounter and that they are surprised by the time and energy these situations consume.4

A department chair’s communication ability in tense situations is a core component of their leadership; if they speak clearly and with emotional intelligence their department may weather even the harshest storms and ultimately flourish. But if they misread key situations and misspeak, they may hemorrhage key talent, alienate critical allies, and doom themselves to a short tenure. Interpersonally unskilled chairs also burden other administrators who must often step in to repair singed relationships. It is not surprising that these issues emerge; while vitae reviews, interviews, presentations, and reference checks are typical components of searches used to screen and select new department chairs/heads, these strategies may fail to identify leaders who can communicate effectively with faculty in common, tense situations.

Recognizing the interpersonal nature of leadership, in 2013, the Association of Academic Health Centers held a leadership-focused workgroup that identified 14 key characteristics of successful academic medical leaders (see List 1). Of the skills listed, the first, “[self-awareness] with a high level of emotional intelligence,” is especially critical in these situations.

To address and improve department chairs’ fundamental management skills, some institutions have developed courses, resources for coaching, and new metrics.5,6 These are laudable efforts, but for many chairs, such training and oversight may fail to compensate for fundamental deficits.

Other institutions have focused on improving the chair recruitment process. One widely used strategy is behavioral event interviewing, which focuses on candidates’ past experiences, assuming that future performance is best predicted by past performance. An example of a typical behavioral event interviewing question is “Describe a conflict with someone who worked for you.” The challenge of using behavioral event interviewing is that responses may reflect interviewing ability rather than interpersonal ability. If the candidate responds with an anecdote that ends happily, the search committee cannot assess the seriousness of the situation, what the candidate said and did, nor how he/she was perceived by the employee or other witnesses. In response to this challenge, we speculated that a standardized role-playing simulation scenario during department chair interviews might be a more effective method.

Notably, in other industries, the use of simulations and role-playing scenarios is now common. As an example, for decades, industrial organizational psychologists have called for the use of realistic job sample tests in hiring,7 and roughly half of federal departments use job simulations to screen candidates, with a significant portion using role-playing scenarios.8 Role-playing scenarios, a form of a job sample test, are also common in private industries. For example, Edward Jones, an investment firm, now routinely requires candidates for investment positions to interact with a simulated client upset with poor investment results.9

In medicine, brief patient-focused, role-playing scenarios have been used to select residents as part of multiple mini-interviews,10 but to our knowledge, role-playing scenarios have not been used in the selection of department chairs.

Back to Top | Article Outline

Approach

Between May 2015 and November 2016, we piloted simulation role-playing scenarios in four department chair searches (neurology, family medicine, pediatrics, and pathology) at Penn State College of Medicine/Penn State Health to assess candidates’ skill at handling common, challenging situations with faculty members. Our goal was to demonstrate “proof of concept,” showing (1) candidates would engage in the simulated scenarios and (2) search committees and the dean would find them valuable.

Back to Top | Article Outline

Initial response

The use of a simulation scenario in the interview process was met with some initial resistance. One search committee member worried the scenario would measure acting ability more than true competence, and a search firm principal warned we might alienate potential candidates. Other search committee members, search chairs, and search firm principals were curious and, in a few cases, enthusiastic about the idea. The use of a simulation scenario in the interview process had the full support of our dean, who is also the chief executive officer of our health system.

Back to Top | Article Outline

The simulation scenario and actors

We created the following scenario, which was used in all four searches: A faculty member who has worked at the institution for 18 months in outpatient settings is frustrated with how little time he/she has for academic pursuits. He/she is a skilled teacher with highly satisfied patients but is less clinically productive than his/her colleagues. (The only difference among the four chair searches was that in the pathology scenario the faculty member was a hematopathologist assigned to interpret samples.)

D.E.S. acted the role of the frustrated faculty member in the scenarios, with one exception when D.E.S. was unavailable and B.H.L. substituted, and followed the rules outlined in the actor instructions. (D.E.S. had college acting training but, perhaps more important, has spent considerable time with frustrated faculty.)

The full scenario (provided to candidates) and the actor instructions (not provided to candidates) are given in Appendix 1.

Back to Top | Article Outline

Goals of the simulation scenario

A vice dean and sitting chair (D.E.S.) familiar with issues brought to chairs initially designed the simulation scenario, and three other sitting clinical department chairs (anesthesia, surgery, and family and community medicine) confirmed its realism. The scenario was designed to assess how well department chair candidates interacted with faculty in a common, challenging situation. It was not designed to be a general problem-solving test but, rather, a measure of critical communication skills when missions conflicted. The emotional intensity of the scenario was designed to be moderate, and not a “once-in-a-career” conflict.

Back to Top | Article Outline

Timing and scoring of the simulation scenario

The simulation scenarios occurred during airport interviews, in front of the search committees (one candidate per interview), between May 2015 and November 2016. They were scheduled to occur roughly 20 to 30 minutes into the 90-minute interviews and to last no more than 15 minutes, including the candidate’s debriefing with the committee and any follow-up questions asked by committee members.

We had some argument about when to release the scenario to candidates and ultimately decided to provide it approximately two weeks in advance. We decided to send it to candidates in advance as we were less interested in their spontaneous problem-solving skills since the majority of difficult conversations department chairs have happen with some notice. Given that we are willing to hire administrators who recognize when they have deficits, seek help, and are coachable, we ultimately accepted the possibility that with advance notice of the scenario, candidates could receive coaching. We also decided to accept the possibility that we might “scare off” viable candidates who were experiencing easier recruitments at other institutions.

Prior to the 10-minute simulation scenario, candidates were asked to explain their goals for the interaction, and immediately after the scenario, candidates debriefed with the committees, sharing their impressions of the interaction, including what went well, what they would have hoped to have gone differently, and how they would follow up. In a few cases, candidates also answered questions from committee members. Immediately after candidates left, committees voted on whether the candidate passed or failed the simulation scenario (see below).

We did not develop a scoring rubric for the simulation scenarios as we were unsure which behaviors were most important. Instead, we relied on search committee members, most of whom were faculty, to generate global decisions about the quality of the interactions. An administrative assistant took detailed notes during the simulation scenarios, including notes on content of the interactions, subsequent committee deliberation, and votes both about whether candidates passed or failed the simulation and, later, about whether candidates should be advanced in the search process.

Back to Top | Article Outline

Outcomes

We used the simulation scenario in four department chair searches, involving a total of 29 candidates, between May 2015 and November 2016. The simulations revealed wide variation in most candidates’ style, substance, and even underlying values that were not otherwise identified through interviewing or other screenings. Virtually all candidates (28/29; 96.5%) started the interactions well, introducing themselves and welcoming the faculty member. Almost all candidates (27/29; 93.1%) also ended the interactions well, suggesting follow-up within a specific time frame.

However, in 9/29 (31.0%) cases, candidates’ responses during the simulation scenario revealed philosophical beliefs or communication behaviors that diverged from our institution’s core cultural values, and thus disqualified (in all but one case; see below) otherwise viable candidacies. Examples of troubling behaviors exhibited during the simulation scenarios are as follows. One candidate responded to the faculty complaint by calmly, if not in a somewhat patronizing way, suggesting the faculty member had no business pursuing scholarship. Another leapt immediately and harshly to the faculty member’s lower productivity, failing to validate or consider the faculty member’s primary concern (access to protected time). A third assumed the role of therapist, inquiring about the faculty member’s marriage and other personal issues that were both intrusive and irrelevant. Finally, two candidates held court, lecturing and providing virtually no opportunity for the faculty member to express concerns. Notably, search committee members were nearly uniform in identifying these and similar behaviors as deeply problematic, though there was debate regarding one candidate who exhibited some of these behaviors (that candidate was deemed to have failed by majority vote but was still later advanced by the committee).

In contrast, the search committees judged approximately two-third (20/29; 69.0%) of candidates to have successfully passed the simulation scenario and ultimately advanced these candidates. There was also wide variability in how these candidates approached the scenario, suggesting the committees felt there are many ways to handle these situations well.

The successful candidates did display some consistent behaviors. They listened carefully and asked many questions. They validated the concerns of the faculty member and normalized them, saying things like, “Yes, I’ve faced that problem too, it is frustrating.” When they asked about productivity issues, they had a curious, nonharsh tone and suggested other information they might gather, such as inviting a division chief to participate. In addition, they showed an appreciation for the value of hierarchy.

During the debriefings, which occurred immediately after the simulation scenarios, some candidates noted they could minimize the chances the scenario would occur by facilitating routine contact between mentors and division chiefs. Others discussed how they would work with the faculty member over time and wisely noted the importance of distinguishing between what needs to happen today as opposed to what needs to happen over the coming months. One talented candidate lamented a subtle joke he had made during the exercise, demonstrating introspection and the propensity for self-correction.

The simulation scenario we used was research focused; it is reasonable to consider whether candidates with stronger research backgrounds were preferentially advantaged. We were interested in improving the research stature of these departments, but this was secondary to seeking candidates who can communicate effectively in tense situations. Notably, candidates with stronger research backgrounds tended to do either very well in the simulation scenario or very badly. Specifically, a number of candidates who had the most impressive research backgrounds talked too much, lecturing the faculty member during the interaction and failing to seek additional information.

Additionally, on the basis of the search committees’ pass or fail votes, candidates from specialties that are sometimes stereotyped as being stronger interpersonal communicators (family medicine) appeared to be no more successful than candidates from specialties sometimes stereotyped as being less successful with interpersonal communication (pathology).

Two department chairs have been hired as a result of these searches. In follow-up e-mails sent before they came to campus, both indicated they felt the use of the simulation scenario revealed that the institution was serious about its values and was going the extra step to ensure that the right person was hired. One indicated some surprise that in other chair searches his aptitude for the real job had not been more directly assessed. Finally, both also indicated that being provided with the scenario in advance reduced concerns they would have had otherwise.

Back to Top | Article Outline

Next Steps

The use of standardized simulation role-playing scenarios has become a routine part of our hiring process for department chairs and is under consideration for division chief hiring. Our dean and chief executive officer plans to continue the practice because he believes it conveys that we expect our chairs to embody our values and have the skills to apply them. Sitting chairs have also expressed interest in expanding the use of simulations for division chief recruitments.

List 2 contains our advice to other institutions hoping to construct their own simulation scenarios for department chair, institute director, or division chief searches.

Our future work will explore whether simulation scenarios can successfully predict which department chairs will be skilled at navigating common faculty challenges and, further, whether this skill results in greater faculty satisfaction, engagement, and retention. We plan to build a larger pool of simulation scenario cases, create a rubric, and formally measure interrater reliability for search committee members observing the interactions. Our future work might also explore whether getting the scenarios in advance results in “coached” rather than “true” performances.

Back to Top | Article Outline

Acknowledgments:

Lynn Matincheck and Margaret Hopkins assisted with the preparation of this manuscript.

Back to Top | Article Outline

References

1. Association of American Medical Colleges. Faculty roster. https://www.aamc.org/data/facultyroster/. Accessed March 31, 2017.
2. Grigsby RK, Hefner DS, Souba WW, Kirch DGThe future-oriented department chair. Acad Med. 2004;79:571–577.
3. Souba W, Notestine M, Way D, Lucey C, Yu L, Sedmak DDo deans and teaching hospital CEOs agree on what it takes to be a successful clinical department chair? Acad Med. 2011;86:974–981.
4. Lieff S, Banack JG, Baker L, et al.Understanding the needs of department chairs in academic medicine. Acad Med. 2013;88:960–966.
5. Revere L, Robinson A, Schroth L, Mikhail OPreparing academic medical department physicians to successfully lead. Leadersh Health Serv (Bradf Engl). 2015;28:317–331.
6. Wilmott RW, Abramson JS, Davies HD, Felice ME, Stanton B, Jacobs RFEvaluation of a mentoring program for new pediatric department chairs. J Pediatr. 2015;167:787–788.
7. Asher JJ, Sciarrino JARealistic work sample tests: A review. Pers Psych. 1974;27:519–533.
8. U.S. Merit Systems Protection Board. Job simulations: Trying out for a federal job. http://www.mspb.gov/mspbsearch/viewdocs.aspx?docnumber=452039&version=453207&application=ACROBAT. Published September 2009. Accessed March 31, 2017.
9. Skinner LForget personality tests, Edward Jones uses job simulation to assess potential hires. InvestmentNews. March 13, 2015. http://www.investmentnews.com/article/20150313/FREE/150319947/forgret-personality-tests-edward-jones-uses-job-simulation-to-assess. Accessed March 31, 2017.
10. Ogunyemi D, Alexander C, Tangchitnob E, Kim DSMini surgical simulation, role play, and group and behavioral interviews in resident selection. J Grad Med Educ. 2016;8:410–416.
Back to Top | Article Outline

Appendix 1 Department Chair–Frustrated Faculty Member Interaction Simulation Scenario and Actor Instructions, Used in Searches for Four Department Chairs, Penn State College of Medicine/Penn State Health, May 2015–November 2016

A. Simulation scenario as provided to chair candidates in advance

Our culture is important to us, and we find chairs play a critical role in encouraging the type of culture we seek. Understanding that chairs spend considerable time and effort interacting with faculty, we have designed this simulation to assess your capacity to engage in what we consider one critically important facet of being a chair—helping faculty navigate common challenges. We understand most leaders do not respond perfectly in every tense situation. There may be awkward moments and this is common. The person playing the frustrated faculty member will not be overwhelmingly challenging; rather, our goal is to simulate a common situation, and not a “once-in-a-career” conflict.

Scenario:

A faculty member (hired 18 months ago) has asked to see you. He/she is upset because his/her clinical load continues to increase as his/her time to pursue academic interests diminishes.

You have spoken to clinic managers who note his/her clinical productivity is low for his/her group—relative value units generation is at 40% for the last 18 months, compared to a mean of 55% for your other faculty. Additionally, in the past year, he/she has frequently canceled his/her clinics at the last minute. His/her teacher ratings are good, he/she is well liked and respected by both residents and medical students, and his/her patient satisfaction scores are above average for our clinics. Based upon your knowledge of the annual reviews with this faculty member, he/she has not published since joining the faculty.

Now you are going to meet with the faculty member for approximately 10 minutes.

You are asked to:

  1. Before the interaction, share your goals for this meeting with the search committee.
  2. Interact with the faculty member. Expect the interaction to be stopped after approximately 10 minutes.
  3. Following the encounter, you will share your impressions of the interaction with the committee including what went well and what you would have hoped to have gone differently. You will share your ideas for how you would want to follow up.
Back to Top | Article Outline

B. Actor instructions (not shared with candidates)

Be respectful but direct. Be modestly to moderately upset. Never raise your voice.

After brief introductions indicate you are coming to the chair because you are “deeply frustrated.” Nonverbally be calm, look at the floor or your own hands when you are talking and at the chair candidate when he/she speaks.

When asked for more information, indicate you understood when you were hired 18 months ago that you were going to be a clinician who worked eight clinical sessions, with Monday afternoon and Friday morning available for academic time, and that you expected to have these two nonconsecutive sessions for academic work.

Emphasize to the chair that this academic time feels that it is never available because you are “stalked” by clinical demands including phone calls, orders, etc. You have two manuscripts you want to work on but have not made any progress beyond the literature searches. (Specific topics vary by specialty but are generally focused on safety and quality and the data is already collected.) You are worried about promotion. If asked, indicate you do have mentorship, the mentorship is “fine,” and you have met three or four times.

If you are asked, life outside of work is fine, you have settled into the community, and you generally like your colleagues but are puzzled at how they appear to function clinically more efficiently. Be proud that you are so meticulous.

If the candidate asks inappropriate or probing personal questions, redirect them to your professional concerns.

If asked if you have already discussed this frustration with your division chief, clinic director, or other supervisor, indicate you did but felt patronized by the response. Quote the supervisor as saying, “‘That happens to everyone.’” Then say, “I felt like they were patting me on the head.”

If complimented about your teaching, be humble and pleasantly surprised.

If asked about canceling clinics, indicate one of your children has had some minor medical issues (asthma) but that these are safely controlled now.

Agree politely, but not enthusiastically, to all suggestions the candidate makes unless those suggestions are vague, in which case ask for clarification.

Stop the interaction if it goes longer than 10 minutes.

Copyright © 2017 by the Association of American Medical Colleges