Model 1: Learning through immersion in a system, delivering an intervention
Data for Model 1 came primarily from six Phase 1 sites and one Phase 2 site. In Model 1, the goals and purposes of the experiences allowed students to actively participate in delivery of patient care to address a specific gap or need in the system. Students participated in activities such as health coaching, motivational interviewing, medication reconciliation, or mobilization of hospitalized patients. These experiences included a systems component in that students contributed to an SI intervention, even if not explicitly defined as such. Supervisors facilitated students’ involvement in the workflow, legitimizing students’ presence and participation, providing students context for the intervention, and teaching relevant clinical skills and knowledge.
Regarding learning and systems outcomes, students typically enjoyed the opportunity to interact with patients and felt that they made meaningful contributions to the workplace, especially when performing tasks that others lacked time to perform. In interviews, supervisors acknowledged that by filling a gap in the system, students helped provide a valued service to patients and/or providers and staff. Supervisors recognized that the value of the experience could diminish over time if students performed the same tasks repeatedly over multiple sessions with little opportunity to increase complexity or develop advanced skills. Model 1 included rich opportunities for systems-oriented learning such as understanding how a specific gap evolves in the system, why the system fails to serve some populations of patients, and roles of various health professionals. However, this learning often was informal or unintentional. Additionally, many learning opportunities focused on clinical knowledge and skills.
Model 2: Learning through immersion in an SI initiative
Data for Model 2 came primarily from two Phase 1 sites and two Phase 2 sites. In Model 2, goals and purposes centered on engaging students in an SI effort already under way at the site, teaching some of the skills associated with SI, and introducing students to a culture that embraces SBP. These experiences addressed SBP explicitly, unlike the more indirect approach in Model 1. In this model, students’ involvement in both clinical and systems activities depended on the improvement effort in which they were involved, the type of clinical work at their site, and students’ interest in combining clinical and systems activities. Common types of systems activities included observation of a specific process in a microsystem (e.g., discharge process, rounds); data collection through observation, interviews, surveys, and chart reviews; data analysis; and application of SI tools (e.g., creation of process maps and fishbone diagrams, attending meetings with SI project team members).
Site supervisors and/or other on-site faculty and staff members were usually well versed in SBP. These individuals taught SBP concepts and skills; facilitated students’ integration into existing SI initiatives; and provided context, clinical knowledge, and skills directly relevant to the SI initiative. Students were highly engaged in SI activities, particularly if they understood the overarching goals for the initiative, could offer insights, and could incorporate some of their own ideas into it.
Model 2 offered many systems and clinical learning outcomes. Systems learning outcomes were more prominent in Model 2 than in Model 1 and included how to analyze a system, specific techniques used in SI, patient/provider/staff experiences in a system, and roles and responsibilities of different participants in the system. Clinical learning outcomes included specific medical language and knowledge and skills relevant to the SI initiative as well as other opportunities provided by supervisors and other faculty and staff at the site. Students’ impact on the system in Model 2 tended to be embedded in a larger effort. Students’ project summaries and interview data—in which students and supervisors described perceived system-level impact—suggest that students contributed important information or data to support problem identification or evaluation of an intervention, performed critical tasks that others did not have time to do, offered insights or ideas for improvement, and ultimately helped ensure the progress or completion of valued SI efforts. However, when students failed to recognize their role in a larger ongoing initiative, they wondered whether they had met expectations and had a positive impact.
Model 3: Learning by leading/performing an SI initiative
Data for Model 3 came primarily from two Phase 1 sites and three Phase 2 sites. In Model 3, goals and purposes focused on students’ involvement in their sites and partnerships with key stakeholders to design, implement, and evaluate a small-scale, focused improvement project or test of change. In contrast to Models 1 and 2, students in Model 3 had a significant role in a whole improvement cycle, and they felt ownership and investment in the project. Their projects were either embedded within a larger initiative or launched through students’ own initiatives. Although full implementation and evaluation were the goals, students could not necessarily achieve these in two to three months. While students spent most of their time engaged in specific SBP activities, they participated in clinical activities directly related to design, implementation, and evaluation of their projects. Some supervisors incorporated additional clinical experiences at students’ request. The supervisor’s role in Model 3 was largely to form a partnership with the students to guide them and provide the resources needed for their projects. As such, supervisors tended to be less directive in Model 3 than in Models 1 and 2.
On the basis of our observations of students’ works-in-progress sessions and information provided in interviews and written reflections, students in this model learned specific skills related to SBP, SI in general, as well as various components of a plan–do–study–act cycle and how specific processes work in a clinical setting. Students also gained a realistic understanding of SI, including the frustration of setbacks and the excitement of small successes. In contrast to Model 1, in Model 3 the clinical learning outcomes tended to occur informally and implicitly, whereas systems-oriented learning was explicit. Students’ project summaries, along with interview data from faculty and students, showed that impact on the system varied by project. Some projects made important contributions that would be sustained, while in other projects the intended intervention was not implemented or evaluated before the student’s rotation ended. In this sense, students gained a realistic and authentic experience with SI that fostered both positive and negative impressions.
Additional case: Learning through immersion in a system
We identified one additional experience that was part of the SOWLE pilots in Phase 2 but did not fulfill the stated goals of SOWLEs. We include this case even though it deviates from the others because it may represent challenges germane to future SOWLE sites. In this example, the experiences of one of the students focused primarily on participation in routine patient care activities rather than on SI activities. By assisting with these patient care activities, the student implicitly learned about roles and relationships among health professionals; workflow within and across clinical services; rules, norms, and safety procedures; and communication processes. The student unintentionally incorporated a systems angle into the experience by creating a guidebook that documented many of the systems features he had to learn to function in the workplace.
In this example, the site supervisor and other faculty and staff who worked with the student had limited training and experience with SI, and the microsystem lacked a culture and commitment to SBP. The student was particularly interested in the clinical work performed at the site and eager to participate in as many clinical activities as possible. Correspondingly, SBP was not a learning priority.
SBP and SI are increasingly recognized as essential parts of high-quality, high-value health care delivery. Many faculty and staff members are still learning how to apply the concepts while simultaneously engaging residents, and now students, in SBP activities. To assist and support faculty in this new educational endeavor, our study’s findings allowed us to identify three models of SOWLEs for early learners and to describe promising practices associated with each model. All three models can provide systems-oriented learning experiences for students, but several factors must be addressed to optimize the workplace experience. Below, we discuss four of these factors, which are related to curriculum design and supervisor roles/pedagogical strategies in the AEEs presented in List 1. We also discuss their implications for sites and institutions that are seeking to design and implement quality SOWLEs in any of the three models.
Selecting authentic activities for students and legitimizing their participation in these activities
A key component of successful AEEs is selecting authentic activities for students and legitimizing their participation in these activities.13,14 Ideally, students’ SOWLEs occur in sites with an established culture of SBP; regular improvement activities; and supervisors, faculty, and staff who are well versed in and committed to systems thinking and improvement. Unlike clinically oriented AEEs, where learning generally aligns with the core activities of the workplace, systems-oriented activities may not be well integrated into routine work. Many sites selected for student placements may be in the process of building a culture of improvement. Acknowledging this and identifying students’ contributions to the process can be a valuable learning opportunity.
In light of these circumstances, our findings suggest that any of the three models could be used for systems-rich learning experiences even in less-than-ideal conditions. The key is for one or more faculty members to carefully consider the opportunities for systems learning in their environment, select the optimal model, and identify specific gaps that will need to be filled. For example, Model 1 (in which students deliver an intervention to improve quality of care or patient experience) may work well in settings where the activities themselves have high authenticity and value and a faculty member can guide students to see the connections to systems learning and improvement. Model 2 (in which students participate in an existing SI intervention) may be well suited to sites where there is support for SI, but capacity to do the work is limited and thus the student can produce valued information to support sites’ efforts (e.g., needs assessments, process maps). Model 3 (in which students conduct a small test of change) can take place at sites where minimal systems work is in place but a faculty expert and champion can provide significant mentoring to the students and facilitate relationship building and partnerships between students and key stakeholders. Faculty efforts to model systems thinking and commitment to improvement help to reassure students that they are doing critical work and not simply performing tasks that no one else has time to do or that are irrelevant to core work of the site (e.g., patient care).
Promoting learner agency
A second key component of successful AEEs is promoting learner agency.13Agency describes learners’ perceptions of their ability to select, direct, and/or influence activities and circumstances in their environment.22 In the context of SOWLEs, we found that all three models promoted agency that pertains to students’ sense of their own capacity to change or improve a system through participation in an intervention or project. Important concepts for promoting students’ agency include preparing them to engage in the workplace with site-specific training and expectations; encouraging them to share their insights, questions, and suggestions; and providing them access to resources and support. In all three models, students can have opportunities to share insights through including them in project meetings and giving progress reports to key stakeholders. Faculty can also support students’ agency by allowing students latitude and independence in selecting and shaping a project, particularly in Model 3.
Providing active support and guidance
The third key element to highlight from the literature on AEEs is providing active support and guidance during the workplace experience.12,13 The ability to support students’ learning of SBP and SI requires that they have access to individuals in the workplace who are knowledgeable in SI, skilled at teaching SI, and committed to guiding students’ learning. These individuals must clearly understand the goals and expectations of the medical school, the types of activities appropriate for early learners, and the level of guidance and support those learners will need. For example, site leaders in our study had to determine the appropriate scope of work for early learners, the amount of background and contextual information needed (given students’ limited clinical experience), and how to provide structure for students’ daily activities and assignments while also allowing students enough flexibility and independence to incorporate their own ideas and address their own learning goals. Site leaders also had to identify ways to engage other faculty and staff members in guiding and supporting students because they often had limited amounts of time to spend with students.
Engaging students in critical reflection and debriefing
A fourth key component of learning involves engaging students in critical reflection and debriefing of their workplace experiences.13 In our study, this occurred in several ways. Site supervisors had daily or weekly check-ins with students. Often, these were informal conversations rather than structured reflections on systems issues. In the Phase 2 pilots, students had weekly works-in-progress sessions that provided rich opportunities away from the workplace to share and debrief experiences with peers and nonsite faculty, learn how to think about their experiences in a systems framework, and receive feedback and encouragement. Students found that these sessions were particularly valuable and that they enhanced the learning experience regardless of the model.
Limitations and opportunities for further investigation
The three models we identified are based on a limited number of experiences in a single institution. Many of these were pilots, and thus carefully selected to provide robust educational experiences for students. The students who selected into these pilots may have been more motivated and interested in SI than is the typical medical or pharmacy student. Expanding SOWLEs beyond a limited number of pilots and students may result in more variability and potentially more models. Phase 2 sites all had an immersion structure, with students devoting 30 to 40 hours per week to their sites with minimal competing curricular activities. However, these experiences may be good representations of the higher end of the spectrum of what students can learn and contribute.
There are several opportunities for further investigation based on the models identified in our study. Questions about appropriate preparation for participation in each model are important. Students participating in the Phase 2 pilots all received a core curriculum as well as additional training at their sites and the opportunity for debriefing away from the sites. Future work could explore the balance between a general preparatory curriculum for SOWLEs and a customized, site- or model-specific curriculum and between on-site and off-site debriefing. Future work might also examine faculty and staff development needs for each model as well as possibilities for sites to transition from one model to another as faculty and staff gain SI knowledge and skills and as resources, culture regarding SI, and the status of SI initiatives change.
SOWLEs, as a type of AEE, offer a promising approach for early learners to begin developing competence in SBP. Every workplace differs in culture, activities, workflow, people, relationships, and performance standards and therefore affords different learning opportunities. This variation is a reality of workplace learning experiences that requires both flexibility in the structure of the experience as well as diligent attention to the goals, objectives, and guiding principles of the SOWLEs.
Our study provides a framework and strategies to guide future design and development of SOWLEs. We suggest that some models may be better suited to certain site characteristics than others. We recommend that, at minimum, all sites have an openness to working with early learners and one or two experienced champions of SBP to legitimize students’ roles and activities, serve as role models, and provide guidance and support.
The three models we identified offer sites options for structuring SOWLEs depending on experience and support for SBP at the site, the status of SI initiatives, and the comfort of supervisors and workplace faculty and staff with SI and working with students. By attending to key factors such as framing activities to highlight systems issues, integrating and legitimizing students’ participation, promoting learner agency, and engaging students in constructive reflection and debriefing, faculty site leaders can enhance both the value of the experience for student learning and also its impact on the workplace.
Acknowledgments: The authors wish to thank the University of California, San Francisco (UCSF) Curricular Ambassadors program for supporting and participating in this project and members of the UCSF ESCape community for helpful feedback on this manuscript.
1. Lucey CR. Medical education: Part of the problem and part of the solution. JAMA Intern Med. 2013;173:16391643.
3. Cooke M, Ironside PM, Ogrinc GS. Mainstreaming quality and safety: A reformulation of quality and safety education for health professions students. BMJ Qual Saf. 2011;20(suppl 1):i79i82.
4. Johnson JK, Miller SH, Horowitz SD. Henriksen K, Battles JB, Keyes MA, Grady ML. Systems-based practice: Improving the safety and quality of patient care by recognizing and improving the systems in which we work. In: Advances in Patient Safety: New Directions and Alternative Approaches (Vol. 2: Culture and Redesign). 2008.Rockville, Md: U.S. Agency for Healthcare Research and Quality.
5. Colbert CY, Ogden PE, Ownby AR, Bowe C. Systems-based practice in graduate medical education: Systems thinking as the missing foundational construct. Teach Learn Med. 2011;23:179185.
6. Jones AC, Shipman SA, Ogrinc G. Key characteristics of successful quality improvement curricula in physician education: A realist review. BMJ Qual Saf. 2015;24:7788.
7. Wong BM, Etchells EE, Kuper A, Levinson W, Shojania KG. Teaching quality improvement and patient safety to trainees: A systematic review. Acad Med. 2010;85:14251439.
8. Varkey P. Educating to improve patient care: Integrating quality improvement into a medical school curriculum. Am J Med Qual. 2007;22:112116.
9. O’Connell MT, Rivo ML, Mechaber AJ, Weiss BA. A curriculum in systems-based care: Experiential learning changes in student knowledge and attitudes. Fam Med. 2004;36(suppl):S99S104.
10. Littlewood S, Ypinazar V, Margolis SA, Scherpbier A, Spencer J, Dornan T. Early practical experience and the social responsiveness of clinical education: Systematic review. BMJ. 2005;331:387391.
11. Yardley S, Littlewood S, Margolis SA, et al. What has changed in the evidence for early experience? Update of a BEME systematic review. Med Teach. 2010;32:740746.
12. Yardley S, Brosnan C, Richardson J, Hays R. Authentic early experience in medical education: A socio-cultural analysis identifying important variables in learning interactions within workplaces. Adv Health Sci Educ Theory Pract. 2013;18:873891.
14. Yardley S. Sense made common: How to add value to early experience. Clin Teach. 2014;11:59.
15. Sheu L, O’Brien B, O’Sullivan PS, Kwong A, Lai CJ. Systems-based practice learning opportunities in student-run clinics: A qualitative analysis of student experiences. Acad Med. 2013;88:831836.
16. Chen CA, Park RJ, Hegde JV, et al. How we used a patient visit tracker tool to advance experiential learning in systems-based practice and quality improvement in a medical student clinic. Med Teach. 2014;17:3640.
17. Ogrinc G, Nierenberg DW, Batalden PB. Building experiential learning about quality improvement into a medical school curriculum: The Dartmouth experience. Health Aff (Millwood). 2011;30:716722.
18. Gould BE, Grey MR, Huntington CG, et al. Improving patient care outcomes by teaching quality improvement to medical students in community-based practices. Acad Med. 2002;77:10111018.
19. Creswell JW. Qualitative Inquiry and Research Design: Choosing Among Five Approaches. 2013.3rd ed. Los Angeles, Calif: Sage Publications.
20. Stake R. The Art of Case Study Research. 1995.Thousand Oaks, Calif: Sage Publications.
21. Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15:12771288.
22. Bandura A. Human agency in social cognitive theory. Am Psychol. 1989;44:11751184.
Supplemental Digital Content
Copyright © 2017 by the Association of American Medical Colleges