The Accreditation Council for Graduate Medical Education (ACGME) requires medical residency programs to implement curricula that enhance resident teaching skills.1 Many residency programs seek to expand their resident-as-teacher (RAT) training opportunities to meet not only ACGME requirements but also resident demand. Much research has shown that residents recognize their vital role as teachers, wish to teach, and respond favorably to formal teacher training.2 In a 2013 needs assessment at our institution (Children’s National Health System; see below), 23 of 25 pediatric residents (92%) responding to a survey indicated that they desired to improve their teaching skills, and 11 (44%) reported that the presence of a RAT program affected their ranking of pediatric residency programs.
Residency programs are increasingly embracing innovative teaching techniques that engage learners at high levels of cognition; however, creating effective RAT programs and evaluating them has proved challenging. In a 2010 national survey, 98 of the 113 responding pediatric residency program directors (87%) reported having a formal RAT curriculum, and 71 respondents (63%) reported that their programs evaluate the RAT curricula.1 Among these 71, only 19 (27%) considered their program to be very or extremely effective.1 According to the report, the reasons for effectiveness varied among respondents, and most directors evaluated their RAT curricula using measures of satisfaction or knowledge rather than changes in teaching behavior or learning outcomes.1 Adding to the challenge, residency training programs often have few resources with which to implement a robust curriculum and can have difficulty fitting such programming into the 80-hour resident workweek. The objective of this Innovation Report is to describe the development, features, and initial evaluation of a novel, one-day, flipped-classroom-based RAT curriculum at Children’s National Health System in Washington, DC.
In 2013, to address the ACGME requirement to promote trainee teaching skills, we created and implemented a novel RAT curriculum constructed around the flipped classroom, a pedagogical method in which learners prepare for scheduled teaching sessions by familiarizing themselves with the relevant material in advance of a classroom session.3 This approach offers several advantages. First, it activates learners by having them prepare prior to each teaching session. This model also “levels the playing field” such that all learners come to each session with the same basic information. Third, it promotes higher-level, active learning because students spend classroom time applying the knowledge and theory that they have previously read about rather than passively receiving it. Fourth, it holds learners accountable to one another, as their individual advance preparation and class participation are vital to the success of the group. Last, it promotes efficiency; learners prepare before the class session, freeing class time to apply, discuss, and evaluate new information, rather than simply listening to the instructor transmit background material.
The flipped classroom approach is grounded in social constructivist learning theory, which emphasizes the construction of new knowledge through social interaction. We planned for our learners to increase their knowledge and skills through small-group learning, peer teaching, problem solving, observing one another, and receiving feedback.4
Recognizing the potential of the flipped classroom model for effectiveness and efficiency, we incorporated it into a novel, intensive one-day RAT curriculum and used standardized learners (SLs) for evaluation. A literature search did not reveal any similar programs.1,2
In constructing our RAT curriculum, we followed the six steps of the Kern model.5 We (1) identified the problem, (2) conducted and reviewed a needs assessment, (3) defined goals and objectives, and (4) established educational strategies (as described above)—all prior to (5) implementing and (6) evaluating the curriculum. Our flipped classroom RAT curriculum was incorporated into the educational curriculum for second-year pediatric residents (PL-2s) at the Children’s National Health System. We designed the curriculum to comprise a series of four workshops to be given back-to-back in one day, targeting four core teaching elements: (1) adult learning principles, (2) giving feedback, (3) teaching a skill, and (4) orienting a learner (see Chart 1). We conducted five iterations of the same daylong workshop over the course of one academic year (2013–2014), each including seven or eight residents (whom faculty assigned to a session according to the resident’s schedule availability).
Children’s National Health System is a freestanding children’s hospital providing premier pediatric care in Washington, DC. It is affiliated with the George Washington University School of Medicine and Health Sciences. The pediatric residency program trains approximately 120 trainees in five main program tracks: categorical pediatrics, community health, primary care, child neurology, and medical genetics.
Prior to each daylong RAT session, PL-2 participants read two brief articles from the British Medical Journal on adult learning principles to prepare them for the first workshop. For each of the subsequent three workshops, residents read additional topic-specific articles during 30 minutes of independent prereading time built into the day’s schedule, as outlined in Chart 1.
At the beginning of each RAT day, to prepare the learners to effectively participate in the day’s activities, the facilitators oriented the PL-2s to the concepts and benefits of the flipped classroom approach. The format of the daylong RAT workshop followed a modified team-based learning approach—specifically, we used the individual and group readiness assurance test processes.6 At the start of each of the four workshops, the participants individually completed a 10-question quiz based on the readings. Next, in small groups of three or four (assigned by faculty facilitators at random), participants discussed the questions on the quiz and, via consensus building, settled on a best answer for each. Then, the facilitators prompted discussion among the whole group to clarify the best answer to each question and engaged the learners in a 10-minute interactive dialogue, highlighting the theory and evidence from the literature and resolving lingering questions. These activities spanned approximately 30 minutes, allowing residents to spend the rest of the hour engaging in activities to apply their knowledge. Underlying the design of our program was the idea that discussion and interaction among group participants not only allows for a deeper understanding and synthesis of key facts but also facilitates the transfer of knowledge into practice.4
The application and higher-order thinking began with interpreting a teaching segment from a commercial film (Dead Poets Society) for the first workshop and advanced to practicing with SLs in a simulated scenario for the remaining three workshops. Each interaction with the SLs involved a relevant situation that allowed the PL-2 learners to practice, problem solve, and subsequently solidify their acquired knowledge. At the end of each interaction, the PL-2 learners received immediate feedback from the SLs, their peers, and the faculty facilitators. The SLs, predominantly fourth-year medical students from the George Washington University School of Medicine and Health Sciences, were participants in the school’s “Teaching and Learning Knowledge and Skills” elective who had specially trained for their participation in the RAT program.7 Two SLs participated in each daylong RAT session, and another set of SLs participated in the objective structured teaching examinations (OSTEs).
To ensure continuity of the program across academic years and potential faculty turnover, we used a train-the-trainer approach to teach faculty members how to facilitate the RAT curriculum. During the first session (2013), two senior faculty members from the GWU Clinical Learning and Simulation Skills Center (B.B. and L.W.G.)—both with experience in applying flipped classroom theory—led the RAT workshops while chief residents (current and rising chief residents, B.D.C., H.K.S., K.R.) and other faculty members (P.B., J.R.K.) observed. Then, the chief residents and faculty who observed the first session facilitated the workshops for the remaining four sessions, while the senior faculty observed and offered feedback. At the end of the inaugural year, six chief residents and faculty members were trained to individually conduct the workshops, securing the presence of the program for subsequent classes of residents.
We evaluated the effect of participating in this flipped classroom RAT curriculum using three methods. First, a group of SLs assessed residents through pre and post OSTEs on their proficiency in each of the three core teaching skills. Second, participants completed two questionnaires (available through the authors). The first, modified from Greenberg et al,8 comprised 27 total items scored on a five-point Likert scale, and assessed participants’ attitudes toward teaching using a retrospective pre–post technique.9 The second, modified from the Baker10 questionnaire, consisted of 14 questions and assessed participants’ self-efficacy on each of the targeted skills before and after the workshops, again using a retrospective pre–post technique. Both questionnaires have been shown to have face validity. Other researchers have used these tools in similar studies.8,10 We analyzed mean changes in pre- and postworkshop performance on the OSTEs and attitudinal and self-efficacy assessments using paired t tests. This evaluation was marked as exempt by the Children’s National Health System institutional review board.
Of the 40 PL-2s at Children’s National Health System in the 2014–2015 school year, 39 (98%) participated in the curriculum during the inaugural year, and 29 (73%) participated during the study period when we were able to use OSTEs to evaluate the effectiveness of the curriculum.
Resident mean performance values in pre- and postworkshop OSTEs showed statistically significant short-term improvement on all three core skills: giving feedback, teaching a skill, and orienting a learner. Mean performance improved the most for teaching a skill, followed by orienting a learner, and lastly giving feedback. The standardized effect sizes (measured as Cohen d) were large, ranging from 0.81 to 1.10 (see Table 1).
Resident ratings on the Greenberg instrument indicated more positive attitudes surrounding teaching after the training (P < .001). Residents also retrospectively rated themselves as more effective teachers after the training on the Baker measures (P < .001). Both standardized effect sizes were large in magnitude (see Table 2).
Our pilot experience suggests that a flipped classroom RAT model represents an effective and efficient curriculum for training residents to become better teachers and thus may be useful to other residency programs. This intensive, learner-centered approach, implemented as a series of one-hour workshops delivered over the course of a single day, allowed full engagement of all participants. Further, this approach enabled sessions to be concise—a critical consideration in the development of any new curriculum competing for time amid an 80-hour workweek.
In future iterations of the RAT program, we intend to adjust the size of the small groups. During the study period, each workshop had seven to eight total participants; therefore, small-group activities included three or four participants each. Team-based learning guidelines6 call for groups of five to seven participants to optimize shared resources for problem solving. Anecdotally, this difference did not dampen resident enthusiasm for each workshop; nonetheless, we still plan to increase the size of the small groups.
Several limitations of our pilot study warrant further research. First, we lacked a direct comparison group of residents receiving a more traditional RAT intervention against which to compare and contrast outcomes. Second, the postintervention assessment demonstrated very positive behavior change in residents, which is to be expected after intensive and focused instruction. However, our program evaluation lacks long-term follow-up with resident participants to determine whether short-term improvements in teaching competence are sustained and whether these improvements translate into better teaching with real learners. Third, the evaluation questionnaires relied on learners’ perceptions, not on objective evidence. Lastly, the retrospective pre–post technique may be limited by the accuracy of learners’ recall.
We are committed to the continuation of this curriculum, and as we move forward, we anticipate conducting long-term follow-up with participants as part of our program evaluation. We are strategizing how best to evaluate outcomes higher in the Kirkpatrick model.2 Specifically, we hope to address whether this intervention results in long-term behavior change and improvements in resident teaching during clinical work on the hospital wards and in clinics. Furthermore, we hope to evaluate whether the RAT intervention translates into improved medical student and/or peer teaching and learning.
Acknowledgments: The authors acknowledge the important contributions of Sally Dudley, standardized patient educator at the George Washington University Clinical Learning and Simulation Skills Center, and Aisha Davis, MD, associate program director of the Pediatric Residency Training Program and assistant professor of pediatrics at Children’s National Health System.
1. Fromme HB, Whicker SA, Paik S, et al. Pediatric resident-as-teacher curricula: A national survey of existing programs and future needs. J Grad Med Educ. 2011;3:168175.
2. Hill AG, Yu TC, Barrow M, Hattie J. A systematic review of resident-as-teacher programmes. Med Educ. 2009;43:11291140.
3. McLaughlin JE, Roth MT, Glatt DM, et al. The flipped classroom: A course redesign to foster learning and engagement in a health professions school. Acad Med. 2014;89:236243.
4. Whitman N. A review of constructivism: Understanding and using a relatively new theory. Fam Med. 1993;25:517521.
5. Kern DE, Thomas PA, Howard DM, Bass EB. Curriculum Development for Medical Education: A Six-Step Approach. 1998.Baltimore, MD: Johns Hopkins University Press.
6. Haidet P, Levine RE, Parmelee DX, et al. Perspective: Guidelines for reporting team-based learning activities in the medical and health sciences education literature. Acad Med. 2012;87:292299.
7. Blatt B, Greenberg L. A multi-level assessment of a program to teach medical students to teach. Adv Health Sci Educ Theory Pract. 2007;12:718.
8. Greenberg LW, Goldberg RM, Jewett LS. Teaching in the clinical setting: Factors influencing residents’ perceptions, confidence and behaviour. Med Educ. 1984;18:360365.
9. Skeff KM, Stratos GA, Bergen MR. Evaluation of a medical faculty development program: A comparison of traditional pre/post and retrospective pre/post self-assessment ratings. Eval Health Prof. 1992;15:350366.
10. Gaba ND, Blatt B, Macri CJ, Greenberg L. Improving teaching skills in obstetrics and gynecology residents: Evaluation of a residents-as-teachers program. Am J Obstet Gynecol. 2007;196:87.e187.e7.