Sheu, Leslie MD; O’Brien, Bridget PhD; O’Sullivan, Patricia S. EdD; Kwong, Austin; Lai, Cindy J. MD
To prepare medical students to interact with a complex and resource-limited health care system, national organizations suggest including instruction on systems-based practice (SBP) in undergraduate medical education.1–6 Defined by the Accreditation Council for Graduate Medical Education as the ability to “demonstrate an awareness of and responsiveness to the larger context and system of health care, as well as the ability to call effectively on other resources in the system to provideoptimal health care,” SBP encompasses interprofessional teamwork to enhance safety and quality of care, coordination of care, resource acquisition and allocation, and identification of system errors and systems-based solutions.7
Although medical school clerkship curricula include SBP content (e.g., quality improvement projects and online patient cases),8–10 37% of graduating medical students responding to the 2011 Association of American Medical Colleges Medical School Graduation Questionnaire reported that the health systems instruction they received was inadequate.11 Although there is no consensus on the optimal method or timing of SBP instruction,12 advocates recommend introducing it early in the curriculum to provide students with a foundation for subsequent experiences.6
Preclerkship SBP course work is primarily didactic and classroom based.13–15 However, situated learning theory, a theory rooted in participation, suggests that students also need to learn through active and legitimate involvement in their professional community.16 Consistent with this theory, student-run clinics (SRCs), which are widespread and continue to grow in number,17,18 may be ideal settings in which to promote early understanding of SBP as medical students assume central roles in patient care and systems organization.
In a recent review, Meah and colleagues19 described how SRCs might offer SBP learning opportunities for medical students. They recommended that future research examine ways in which students participate in SBP activities and whether students can achieve competency in SBP through participation in SRCs. In this study, we explored these topics by interviewing medical students and faculty about students’ involvement with SRCs. We also compared students’ perceptions of what they learned about SBP in the SRCs and their formal preclerkship curricular experiences.
Participants and setting
The University of California, San Francisco (UCSF), School of Medicine has four independently run SRCs that provide care for underserved populations (for an overview of the clinics, see Table 1). Each year, approximately 70% (n = 105) of the school’s first-year medical students serve as volunteers who participate directly in patient care (with some students volunteering at more than one SRC), and 25% (n = 40) of first-year students become coordinators, assuming a leadership role with responsibilities for organizing and running the clinics. Coordinators continue in their positions during their second year of medical school, and many students continue to volunteer in their second year. At three of the four SRCs, interprofessional students from pharmacy, nursing, and/or dental schools also provide patient care and act as coordinators.
As our study was exploratory in nature, we conducted in-depth, semistructured interviews with medical studentswho were or had been involved with the SRCs and with SRC faculty advisors. Using the interview guide we developed, one of us (L.S.) conducted pilot interviews with four former SRC coordinators.On the basis of these interviews, wemodified the guide to improve the flow of the questions, but we did not change the content. We asked all student intervieweesto provide basic demographic information, describe the different roles of students participating in the SRCs, discuss the opportunities they had to learn about SBP through their SRC participation and formal curricular experiences, and reflect on their overall SRC experience. We used the same guide for the faculty interviews, but we asked faculty advisors to describe students’ SBP learning opportunities. (Our interview guide is available as Supplemental Digital Appendix 1 at http://links.lww.com/ACADMED/A129.)
Between November 2011 and February 2012, we recruited and interviewed former SRC volunteers or coordinators (medical students in their third or later year), current SRC volunteers or coordinators (second-year medical students), and SRC faculty advisors who had been in their roles for several years at the time of our study. Including these three groups gave us a range of perspectives and allowed for triangulation of themes to increase validity of our results.20
After the pilot interviews, we recruited additional participants using a combination of purposive and snowball sampling techniques.21 One of us (L.S.) contacted 21 current and former coordinators using information available online and interviewed the first 2 to 3 responders from each SRC for a total of 14 (2 current and 1–2 former coordinators per SRC, including the 4 pilot interviewees). We targeted this number of coordinators, anticipating saturation of ideas, and found this to be the case. L.S. asked the participating coordinators to recommend volunteers to interview and invited 14 past and current volunteers to participate. She interviewed the first current and past volunteer who responded per SRC. She stopped conducting volunteer interviews following these 8, after hearing similar ideas with no new responses.
After completing the student interviews, L.S. interviewed one of the primary faculty advisors for each SRC. These faculty were familiar with and responsible for overseeing the clinics’ activities.
All potential interviewees were contacted by e-mail. The recruitment e-mails assured them that their responses would be confidential. Interviews were conducted in person or over the telephone, depending on interviewee preference and availability. All interviews were audio-recorded and deidentified by L.S. They were transcribed by L.S. and A.K. This study was approved by the UCSF Committee on Human Research.
We analyzed the interview transcripts using thematic analysis.22 Prior to analysis, we agreed to broadly define SBP as “how patient care relates to the health care system as a whole and how to use the system to improve the quality and safety of patient care.”12 We also agreed to frame our coding to explore the four key elements of SBP outlined by Meah and colleagues19: “interdisciplinary and team-based care,” “resource acquisition and allocation,” “advocacy and navigation of the health care system to overcome barriers,” and “quality care.” Three of us (L.S., B.C.O., C.L.) read the same two coordinator transcripts independently to develop initial coding categories. At this stage, we modified Meah and colleagues’ four domains and added two other domains. In addition, we created codes for capturing students’ comparisons of SBP learning opportunities in SRCs to content in the formal curriculum, as well as comparisons of volunteer and coordinator experiences.
Once we finalized our coding categories, three of us (L.S., B.C.O., C.L.) independently coded one transcript and reviewed discrepancies to clarify the definition of each code. Two of us (L.S., C. L.) each coded the remainder independently and discussed any disagreements until we reached consensus. We then coded the faculty interviews and used those data to check for similarities and differences between faculty and student perspectives.
We used NVivo version 8 (QSR International, Melbourne, Australia) to organize data for thematic analysis. We reviewed all the content coded in each category to identify themes and representative examples.
We interviewed 26 people (63% of the 41 individuals invited; 8 volunteers, 14 coordinators, 4 faculty advisors, evenly distributed across the four SRCs). Ten (45%) of the 22 students were female, and the average age of students was 26 years (SD = 2). Twelve students (55%) were in the second year of medical school (8 coordinators, 4 volunteers), and 10 (45%) were in the third or later year (6 coordinators, 4 volunteers). The volunteers reported that they had volunteered at an SRC one to eight times in their two preclerkship years; the majority of UCSF students who participate in SRCs volunteer twice.23 Interviews ranged from 30 to 96 minutes.
SBP learning opportunities in SRCs
Our analysis identified six major domains related to SBP learning opportunities in SRCs: interprofessional roles and collaboration; clinic organization; patient factors affecting access to care; awareness of the larger health care system and continuity of care; resource acquisition and allocation; and systems improvement. These domains appeared in both student and faculty interviews.
Interprofessional roles and collaboration
Most students valued working with peers from other health care professions as a way of learning about professional roles and fostering collegiality. They described SRC experiences as teaching interprofessionalism more effectively than formal classroom-based instruction on the topic. One volunteer explained:
The pharmacy student came into the patient room with me and interviewed the patient about medications, and that was great, because they have much more impact and specific questions to ask. I can remember still, they were checking if the patient understood what every medicine was for, trying to figure out if there was any problem with compliance and how to change that, asking specific questions about side effects…. I might’ve missed something or may not know what one of the medications was for but they would easily know. (Volunteer, SRC D)
SRC coordinators were familiar with all student roles within the SRC and with the educational goals of other health professions students. The roles and goals of students in other health professions were not as clear to SRC volunteers, particularly those working in SRCs where students were purposely clustered according to their professional schools (e.g., medical students and dental students working at separate stations). Coordinators reported that collaborating “behind the scenes” on SRCs’ interprofessional student governing boards enhanced their understanding of other professions’ goals and improved their comfort working with interprofessional peers.
Students understood the components of the SRC system, a critical aspect of understanding how this “microsystem” fits into the larger health care system. They were able to describe the roles of coordinators (overseeing clinic management, finances, publicity, volunteers, didactic electives) and volunteers (performing histories and physicals, presenting to preceptors, documenting patient encounters) as well as clinic flow, volunteer training, and preparation. One coordinator summarized the coordinator role as follows:
We make sure the clinic itself runs smoothly, … orient volunteers, make sure everything is set up properly, triage patients and oversee general issues about the clinic flow. But then I think a lot has also been trying to manage the organization as a whole, we do a lot of outreach and grants. (Coordinator, SRC B)
Patient factors affecting access to care
SRC volunteers and coordinators were attuned to the challenges of providing health care to underserved populations. They reported addressing limited English proficiency with systems solutions such as using on-site volunteer interpreters (college students from various campuses who were proficient in a second language and interested in pursuing health care professions). Additionally, they described systems factors leading to patient noncompliance and loss to follow-up, highlighting the importance of patient education, geographically convenient pharmacies and referrals, and cost minimization. Many students discussed how firsthand exposure to patients with barriers to access helped them better understand health care disparities.
I think what is most valuable is just the opportunity to learn from the patient, the patients are the best teachers … we get to see firsthand what it feels like not to have access to care, or what it’s like to be somebody who just immigrated to the country and doesn’t understand English or somebody who just doesn’t have insurance and doesn’t have anywhere to go…. I really learn a lot but earlier [thanthe third year], it kind of shapes how our careers unfold in the future. (Volunteer, SRC B)
Awareness of the larger health care system and continuity of care
Both coordinators and volunteers felt that they understood the organization of the larger health care system better after participating in an SRC. In particular, they expressed greater awareness of insurance options and community resources through having provided direct patient care in underserved settings.
I just went to [SRC A] a couple weeks ago, and one of the first-year medical students spent like half an hour on her phone trying to locate a resource for a patient that we didn’t have, and was urgent. So it’s like, you really get to see, she took it upon herself to really explore the system in that sense, and that was really cool…. In third-year clerkships, we don’t get any of that. You just order whatever, and it happens. It’s not like that in real life. (Coordinator, SRC A)
Both coordinators and volunteers identified challenges to ensuring successful continuity of care (i.e., follow-up and referrals). These included patient factors such as cost, transportation, lack of time, and misunderstanding, as well as systems factors such as difficulty contacting patients who do not have stable housing and lack of continuity of SRC staffing. Students provided examples of how they saw SRCs try to mitigate these barriers by partnering with local health centers for referrals, being consistent in the days they were open, providing patients with reminder cards and phone calls, emphasizing patient education, and implementing patient navigator programs. Coordinators described efforts to connect patients to primary care clinics and discussed specific community partnerships to facilitate this, but nearly half of the volunteers were not aware of their SRC’s referral or follow-up mechanisms.
Resource acquisition and allocation
Each SRC’s finances are managed by a student finance committee and/or student coordinator. Although coordinators could identify specific funding agencies and general areas to which money was allocated (e.g., supplies, publicity, electives), volunteers could not. Neither volunteers nor coordinators could articulate the actual costs of sustaining their clinic operations.
Both volunteers and coordinators commented on the impact of funding and resource limitations on SRC operations, including insufficient equipment and specialty services and inability to write prescriptions, provide medications, or draw labs. Several coordinators reported performing needs assessments to determine whether (and which) additional services could be provided.
If there is a need then what we do is evaluate what our capacity is and see whether or not we can do it…. Originally with our funding, we couldn’t have a lot of our services running and couldn’t get all the supplies and equipment we needed so we’ve had to revisit this and you know, look for more funding. (Coordinator, SRC D)
Students described opportunities to discuss suboptimal systems and potential solutions as built-in components of each SRC, although they did not label these activities as “systems improvement.” Such discussions primarily occurred during postclinic feedback sessions and board meetings.
The first time I went there I had given a lot of feedback after my first session…. I gave a laundry list of suggestions, and didn’t realize I was probably hurting the feelings of the coordinators a little bit. But when I went back my second time, not only did they remember me, they incorporated all my feedback…. I felt like I was really heard. (Volunteer, SRC B)
Students’ examples of systems improvement efforts included addressing bottlenecks in clinic flow, changing volunteer roles (e.g., increasing interprofessional interactions, adding patient navigators), and implementing patient databases. Volunteers and coordinators reported maintaining the quality of patient care through conducting patient surveys and focus groups, using interpreters and patient navigators, checking patient identifiers such as name and birth date at multiple points during the visit (e.g., vaccinations, blood draws) to avoid medical errors, and enforcing policies on the number of phlebotomy attempts made by first-year medical students before preceptors step in. Coordinators also stressed the importance of working with community partners to analyze the clinic services and align them with the community needs.
Relationship between level of SRC involvement and SBP learning opportunities
All students agreed that their awareness of SBP increased as their degree of SRC involvement increased, either by volunteering more frequently or serving as a coordinator. Most volunteers reported feeling that they did not have much awareness of the clinic system itself or its relationship to the larger health care system during their first one to two sessions at an SRC.
With your first clinic session, maybe you’re getting used to the flow, so you’re not learning about interprofessional communication, or optimizing patient follow-up. I feel like the more you get to know a system, the better you get at identifying challenges for it, because you see a problem that comes up again and again and seeing all the past solutions that were tried and seeing if any new ones are better. (Volunteer, SRC D)
Many students—and all faculty advisors—thought that coordinators had more exposure to and awareness of SBP by virtue of being at the clinics often and having expanded organizational roles. Coordinators indicated that their behind-the-scenes roles allowed them to understand the larger system of care—both the organizational system within the SRC itself and the SRC in the context of the public health structure.
I think we as coordinators always want to make our next clinic better, so that is what we usually reflect on…. I think [volunteers] will probably reflect on their interaction with patients, how they can better answer this particular question that the patient asked, or things like that. I think it’s more individualized. I think [volunteers] reflect more on doctor–patient interactions whereas we reflect more on system-based issues. (Coordinator, SRC B)
Comparison of SBP learning opportunities in SRCs and the formal preclerkship curriculum
Students indicated that the formal preclerkship curriculum included lectures on health policy and small-group discussions of clinical and systems issues. They highlighted the value of real patient encounters and working with underserved patients in SRCs. Most students felt that their SRC experiences provided them with greater exposure to SBP learning opportunities than did the formal curriculum, particularly in terms of interprofessionalism, patient access issues, community services, and the process of running a clinic.
The formal curriculum is a lot of book knowledge, I think they definitely cover [SBP topics]. But it’s a different kind of knowledge than actually being in the clinic, which has more hands-on, more realistic assessment of how things are run, which I think has a really important role in education. (Coordinator, SRC B)
However, despite acknowledging that SRCs enhanced their general awareness of systems issues, students described explicit teaching of SBP topics in the clinics as minimal. Volunteers who participated only once or twice reported they had learned more about direct patient interactions and clinical skills than about systems issues.
Faculty advisors indicated that SRCs provide valuable opportunities for students to be exposed to and apply SBP. The perceptions they expressed in their interviews corroborated those of the student volunteers and coordinators. One faculty advisor commented on the ways in which SRCs can activate students to learn about SBP:
If you’re going to be successful in running a free clinic, you need to know what the gaps are in the system. That’s how your clinic is going to contribute to taking care of people. So just because of the reason a student-run free clinic exists, it inherently gives you this opportunity [to learn about SBP]…. I think the way you activate health professional students to learn [about SBP] is to show them how it can help you overcome these barriers that you inevitably encounter as you are trying to provide high, good quality care for these patients. (Faculty advisor, SRC D)
Studies and surveys consistently report that medical students perceive themselves as being ill prepared to address issues of quality, costs, and limited resources in caring for patients.11,24,25 Meah and colleagues19 proposed the value of SRCs in addressing this issue and identified the need for future research to understand how students’ SRC experiences may facilitate SBP learning. Through our interviews with SRC student volunteers, student coordinators, and faculty advisors, we identified six domains in which preclerkship students learn SBP in SRCs: interprofessional roles and collaboration; clinic organization; patient factors affecting access to care; awareness of the larger health care system and continuity of care; resource acquisition and allocation; and systems improvement. Students who participated in SRCs developed an awareness and knowledge of SBP beyond that gained through their formal curricular experiences. Compared with volunteers, coordinators appeared to have a more in-depth understanding of how to work effectively within a health care system. These findings suggest that SRCs may function as important venues for early medical students to learn about access to care and health care delivery.
Although our analysis generally supported the four key elements of SBP described by Meah and colleagues,19 we identified themes in six domains. They described “advocacy and navigation of the health care system to overcome barriers,” but we identified this as two distinct domains: “patient factors affecting access to care” and “awareness of the larger health care system and continuity of care.” While volunteers and coordinators both reported learning about patient factors that affect access to care, our analysis showed that volunteers had less opportunity than coordinators to understand continuity of care and the larger health care system because volunteers’ SRC involvement was transient. We also elaborated on Meah and colleagues’ element of “quality care,” which we captured as the domains of “clinic organization” and “systems improvement.” Not surprisingly, students’ understanding of clinic organization deepened as their involvement in clinic operations increased. This elaboration also emphasizes students’ engagement in improving delivery of care.
The interviews provided insight into the elements of SRCs that enabled students to learn about SBP, another area for future research highlighted by Meah and colleagues.19 By having legitimate roles in a community of practice,16 students participating in SRCs learn about health care disparities, systems improvement, and the larger health care system. Because SRCs provide a narrow range of services to a small population of patients, students may be better able to “own” clinic operations and effect systems-based change than they can in traditional clinical or hospital settings. Furthermore, because most SRCs provide services to disadvantaged populations17 and students assume primary responsibility for patient care, students gain firsthand awareness of systems issues and barriers. SRCs also provide opportunities for students to develop skills in clinic management and leadership.19,26 From a broader educational perspective, the design of other preclerkship clinical experiences could be enhanced by incorporating the principles that help drive SBP learning in SRCs.
We also recommend assessing how SBP learning can be maximized within SRCs. Although students’ comments identified opportunities for SBP education in SRCs, there seemed to be little focus on the teaching of SBP. For instance, many students commented on improving clinic operations through postclinic feedback sessions and board meetings, but most did not describe their efforts as “quality improvement.” Explicitly labeling the SBP work that the students are doing could make them more attuned to this important competency. Preceptors and small-group postclinic feedback sessions could include a focus on SBP as it comes up naturally in patient care. In addition, SRCs have great potential to promote collaboration across health professions. More deliberate structuring of clinical experiences could enhance this benefit.
The conclusions drawn from our qualitative analysis may be limited by the voluntary nature of the sample (i.e., first responders may be the strongest supporters or most engaged learners). Additionally, former volunteers and coordinators were asked to draw on experiences that took place one or more years before the interviews. Further, the questions we asked during the interviews may have led students to reflect on SBP experiences in a way in which they may not have otherwise. However, our study has several strengths. We interviewed students and faculty from four independently run SRCs within a single medical school, allowing for triangulation of perspectives and comparisons of preclerkship students’ different SRC experiences with their experiences in the same formal curriculum. Our design also allowed us to determine an effect of degree of involvement in SRCs, which has been a limitation of previous studies and is an area of interest for research.19,26,27
Our findings provide insight into the level of SBP learning that students can achieve through preclerkship participation in SRCs, particularly for students who volunteer more frequently or take on leadership roles. We believe our findings have implications for preclerkship student involvement in SBP within and beyond SRCs. We encourage educators to view SRCs as valuable settings for SBP instruction and to apply elements of successful SBP learning in SRCs to other clinical experiences.
Acknowledgments: The authors thank the student-run clinics’ faculty advisors, student coordinators, and student volunteers for sharing their experiences.
Other disclosures: None.
Ethical approval: This study was approved by the University of California, San Francisco, Committee on Human Research.
Previous presentations: Preliminary results of this study were presented as an oral presentation at the Association of American Medical Colleges Western Group on Educational Affairs meeting in Asilomar, California, April 2012, and as a poster presentation at the Society of General Internal Medicinenational meeting in Orlando, Florida, May 2012.
1. Academic Medical Center Working Group of the Institute for Health Care Improvement. . The imperative for quality: A call for action to medical schools and teaching hospitals. Acad Med. 2003;78:1085–1089
3. Institute of Medicine. Crossing the Quality Chasm. 2001 Washington, DC National Academy Press
4. Liaison Committee on Medical Education. . Functions and Structure of a Medical School: LCME Accreditation Standards. http://www.lcme.org/standard.htm
. Accessed February 15, 2013.
5. Pascoe JM, Babbott D, Pye KL, Rabinowitz HK, Veit KJ, Wood DL. The UME-21 project: Connecting medical education and medical practice. Fam Med. 2004;36(suppl):S12–S14
6. Riegelman R. Commentary: Health systems and health policy: A curriculum for all medical students. Acad Med. 2006;81:391–392
7. Graham MJ, Naqvi Z, Encandela J, Harding KJ, Chatterji M. Systems-based practice defined: Taxonomy development and role identification for competency assessment of residents. J Grad Med Educ. 2009;1:49–60
8. Jacobsohn V, DeArman M, Moran P, et al. Changing hospital policy from the wards: An introduction to health policy education. Acad Med. 2008;83:352–356
9. Rabinowitz HK, Babbott D, Bastacky S, et al. Innovative approaches to educating medical students for practice in a changing health care environment: The National UME-21 Project. Acad Med. 2001;76:587–597
10. Rivo ML, Keller DR, Teherani A, O’Connell MT, Weiss BA, Rubenstein SA. Practicing effectively in today’s health system: Teaching systems-based care. Fam Med. 2004;36(suppl):S63–S67
12. Johnson JK, Miller SH, Horowitz SDHenriksen K, Battles JB, Keyes MA, Grady ML. Systems-based practice: Improving the safety and quality of patient care by recognizing and improving the systems in which we work. In: Advances in Patient Safety: New Directions and Alternative Approaches. Vol. 2: Culture and Redesign. 2008 Rockville, Md Agency for Healthcare Research and Quality http://www.ahrq.gov/downloads/pub/advances2/vol2/Advances-Johnson_90.pdf
. Accessed February 15, 2013.
13. O’Connell MT, Rivo ML, Mechaber AJ, Weiss BA. A curriculum in systems-based care: Experiential learning changes in student knowledge and attitudes. Fam Med. 2004;36(suppl):S99–104
14. Boltri JM, Vogel RL, Tindall WN, Tedders SH. Improving medical students’ knowledge of systems-based practice. Manag Care Q. 2002;10:29–32
15. Mou D, Sarma A, Sethi R, Merryman R. The state of health policy education in U.S. medical schools. N Engl J Med. 2011;364:e19
16. Lave J, Wenger E Situated Learning: Legitimate Peripheral Participation. 1991 New York, NY Cambridge University Press
17. Simpson SA, Long JA. Medical student-run health clinics: Important contributors to patient care and medical education. J Gen Intern Med. 2007;22:352–356
18. Ferrara A, Moscato B, Griggs R, Thomas R, Smith S. Research session. Paper presented at: Society of Student-Run Free Clinics Conference in Conjunction With the Society of Teachers of Family Medicine. February 2012 Long Beach, Calif
19. Meah YS, Smith EL, Thomas DC. Student-run health clinic: Novel arena to educate medical students on systems-based practice. Mt Sinai J Med. 2009;76:344–356
20. Bogdan RC, Biklen SK Qualitative Research in Education: An Introduction to Theory and Methods. 2006 Boston, Mass Allyn and Bacon
21. Mack N, Woodsong C, MacQueen KM, Guest G, Namey E Qualitative Research Methods: A Data Collector’s Field Guide. 2005 Research Triangle Park, NC Family Health International
22. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3:77–101
23. Sheu L, Lai CJ, Coelho AD, et al. Impact of student-run clinics on preclinical sociocultural and interprofessional attitudes: A prospective cohort analysis. J Health Care Poor Underserved. 2012;23:1058–1972
24. Agrawal JR, Huebner J, Hedgecock J, Sehgal AR, Jung P, Simon SR. Medical students’ knowledge of the U.S. health care system and their preferences for curricular change: A national survey. Acad Med. 2005;80:484–488
25. Patel MS, Lypson ML, Davis MM. Medical student perceptions of education in health care systems. Acad Med. 2009;84:1301–1306
26. Colbert CY, Ogden PE, Lowe D, Moffitt MJ. Students learn systems-based care and facilitate system change as stakeholders in a free clinic experience. Adv Health Sci Educ Theory Pract. 2010;15:533–545
27. Simmons BB, DeJoseph D, Diamond J, Weinstein L. Students who participate in a student-run free health clinic need education about access to care issues. J Health Care Poor Underserved. 2009;20:964–968