Secondary Logo

Journal Logo

Perspectives

Health Systems Science: The “Broccoli” of Undergraduate Medical Education

Gonzalo, Jed D. MD, MSc; Ogrinc, Greg MD, MS

Author Information
doi: 10.1097/ACM.0000000000002815
  • Free

Abstract

With increasing evidence suggesting poor health outcomes in spite of high health care spending, U.S. health systems are transforming care delivery models to better achieve the Quadruple Aim of enhancing patients’ experiences of care, improving population health, lowering cost, and improving providers’ work–life balance.1,2 A critical need in this process is providers’ ability to practice and lead in new care models. These changes require medical schools and residency programs to increase focus on new curricular areas that allow for the training of the next generation of collaboratively effective, systems-based physicians.3–8 Medical educators and health system leaders have made calls for significant reform along the education continuum to catalyze much-needed change within health systems.6,9–13

To prepare medical students to thrive in a 21st-century health care system, many medical schools are aligning health systems science (HSS) curricula with emerging health system needs. Previously, we have described HSS as the third educational pillar that complements the foundational basic and clinical sciences.8 HSS is defined as the “principles, methods, and practice of improving quality, outcomes, and costs of health care delivery for patients and populations within systems of medical care.”14 The HSS pillar provides a comprehensive framework of competencies related to value-based care, population health, interprofessional collaboration, health system improvement, and systems thinking.8,15,16

Medical schools are struggling to allocate resources for, find curricular space for, and identify educators able to lead and teach new HSS coursework.17,18 Perhaps one of the most significant challenges in HSS teaching, however, is students’ mixed receptivity and engagement.19–21 Many students are fully engaged while others are dissatisfied with curricular time dedicated to competencies not perceived as “high yield.”19,22,23 This is compounded when students encounter highly variable support for and role modeling of HSS competencies in learning environment settings.24,25 HSS learning is “broccoli” for some students—they may realize it is good for them and will benefit them down the road, but it may not be palatable during introductory phases, and they may not appreciate it in the moment. As a result, at many medical schools, enthusiasm for HSS may devolve into a position of “lowest performer” in the curriculum, furthering student and faculty dissatisfaction and placing HSS course directors and education leadership in potentially vulnerable positions.

Struggles related to student receptivity of HSS curricula are not well articulated in the literature. In our experience, the struggles are often identified in confidential, personal conversations amongst educators from different medical schools. For the continued transformation of health systems, medical education must promote healthy dialogue about real-time challenges and accelerate its own changes to align with these needs. We represent 2 U.S. medical schools with over 11 years of combined experience in global HSS curricular reform, and our curricula are well beyond initial implementation phases.26 Using the curricular implementation “performance gap” concept as a guide, we explore successes and challenges of student engagement in HSS curricula and call for a reexamination of 5 key issues that may address the broader understanding of underlying challenges of HSS innovations.26,27

Context and Coursework

Both Geisel School of Medicine at Dartmouth and Penn State College of Medicine (PSCOM) have several years of experience building integrated, longitudinal content in HSS. Geisel uses the Health Care Delivery Science title, while PSCOM uses HSS, but the domains, content, and skills are similar (Table 1).14,16,26,28 Both of these longitudinal curricula were built for students to learn the basic concepts in the preclinical years, apply those in the clerkships, and then have a capstone experience near the end of medical school.

Table 1
Table 1:
Description of Health Systems Science Coursework Across All 4 Years at 2 U.S. Medical Schools

Successes of HSS Coursework

The HSS coursework has shown successes at several levels. From a health system and patient perspective, the coursework has created opportunities for incorporating new educators into the academic mission and stimulated bridge building between the medical school and health system. Students working in systems roles are making a difference for patient care (over 1,100 Penn State Health patients positively impacted by student work since 2014) and for system-level changes (more than 60 quality improvement [QI] projects completed by fourth-year students at Geisel and PSCOM over the past 5 years).20,29,30 Additionally, the curricula have positioned our schools positively with regard to the Liaison Committee on Medical Education standards for interprofessional education and service learning and several of the Association of American Medical Colleges (AAMC) Graduation Questionnaire items.31 From a learning perspective, students demonstrate their acquisition of new HSS competencies, spanning social determinants of health to the design of QI projects that impact local microsystems.20,32

Reimagining the Performance Gap in Curricular Change

Our curricular work has gone through the planning, initiation, and implementation phases and is settling into the institutionalization phase.33 As anticipated, our curricula have encountered several challenges.33 Students’ ratings of “course quality” (the primary, readily available metric for short-term evaluation) are well below our respective schools’ mean for other courses and, in some cases, are the lowest scores of any course. Aligned with the “performance gap” (or “implementation dip”), we expected low scores during implementation.27,33,34 The performance gap is a phenomenon seen during curricular change where a decrease in performance occurs after implementation, followed by a rise to “new highs” after several iterations (Figure 1).27,34 The performance gap can theoretically be illustrated using any evaluation metric, such as perceptions or performance-based assessments, but given the lack of other performance metrics in HSS, we have used student perception scores. During initial years, new sessions were piloted, and new educators were being acclimated to the teaching environment. With continuous improvement, sessions were changed, added, or dropped; educators improved their skills and familiarity with the medical school teaching environment; and course directors better framed courses to increase applicability. However, we are well beyond initial trials of curricular pilots and introduction of new content. The first students exposed to initial courses have graduated. Despite the passing of time and significant improvements, challenges remain in student satisfaction. Annually, we have generally found that one-third of students in each class will rate the coursework as (1) poor, (2) average, and (3) good, with slightly higher ratings from upper-year students. This gap between the expected recovery for most curricula and our experience in HSS curricula we term the “HSS performance gap” (see Figure 1B).

Figure 1
Figure 1:
The performance gap in curricular innovations and health systems science (HSS) coursework. Figure 1A depicts the performance gap (described by Eastwood and Louis27 and Bland et al33) that shows the expected performance decline at the start of a curriculum (“implementation dip”), followed by a rise in performance to a status above baseline after a few iterations. The actual performance infrequently reaches the “myth” level of exemplary performance. Figure 1B depicts the authors’ application of HSS curricula to the performance gap concept. An increase in performance is identified with improvements such as modified course sessions and enhanced educators’ skills. Because of underlying system factors (as described in the text), there remains an “HSS performance gap” between actual performance and current status.

From a superficial level, a medical school’s impulse may be to “fix it” through technical means, such as new educators, methods, or evaluations. On a deeper level, though, the HSS performance gap suggests a complex tension within the core educational mission. Root causes and high-leverage solutions likely lie beyond tinkering around the edges of individual sessions and involve a systems-related network of factors that need to be addressed. These changes may lie outside the purview of any one medical school and reflect larger cultural issues in medical education that require time and strategy, but they need to be pursued aggressively to achieve desired outcomes. HSS coursework has unique factors that differentiate it from most other curricula, prompting a reimagining of the “performance gap.” Rather than months to several years of recovery following the implementation dip for most curricula, we believe that HSS coursework will require at least several years for a full recovery and a combination of both local and larger-scale changes in medical education. Based on our experiences, we explore 6 key drivers of the HSS performance gap.

Marginalization of HSS coursework

Before launching our curricula, we performed a qualitative study with medical students to identify pedagogical challenges in HSS coursework.19 Students reported that HSS competencies were important to “become the best doctor possible,” but this was at odds with an educational system driven by board examinations and grades. As a result, students shared significant apprehensions about HSS. Fast forward several years, and we see that our findings have played out as expected. Preclinical students have a strong focus on board examinations and grades, and some students view HSS as competitor “content” or “soft skills,” which limits traction and engagement. The approach at our medical schools has been to integrate HSS early in the curriculum during a time when students are not fully immersed in clinical environments and grappling with the realities of health system issues, which may be limiting additional engagement.

For over 100 years, medical schools have emphasized basic and clinical science competence (i.e., science grades) as the principal predictor of future success. Not until 2013 did the AAMC suggest that medical schools should shift their principal emphasis away from undergraduate science preparation to a more holistic approach that gives equal value to commitment to service, cultural sensitivity, and interpersonal skills.35 This historical context is critical to the conversation about HSS coursework because variability in student engagement may be driven by deeply entrenched expectations about what should be taught and learned in medical schools. Broadening students’ focus to health systems issues may be a larger cultural challenge that requires transformational change in our education system.19,24

Lastly, we believe that the root of some marginalization of HSS coursework is a mismatch between what students envision as HSS and what content is presented in the curriculum. For example, many of our students have a strong interest in health care disparities and advocacy, and as a result, they expect coursework to focus exclusively on these areas (e.g., how to write a policy brief). This is at odds with a more comprehensive HSS framework focusing on underlying concepts of systems, improvement, patient safety, and biostatistics. This creates a challenge when framing HSS material in the classroom and when setting expectations throughout the coursework.

Infancy of HSS as a field

Compared with basic and clinical sciences, HSS is still relatively young. Basic foundational sciences have centuries of empirical evidence on which to draw, and clinical sciences have grown by leaps and bounds over the past century.11,36 The science of how systems, and especially health care systems, are structured, function, fail, and improve is only several decades old. The epistemological foundation of HSS is still being formed, and this can make the material feel more nebulous to students (and also to faculty educators who work to operationalize HSS competencies). We anticipate that as the science of health systems grows, student traction and best teaching methods will follow.

Relative nascence of curricula and educators

Many HSS educators are new to their role and have not previously been acclimated to the medical school environment. Our educators are often system leaders who keep the health system nimble and aligned with desired outcomes.18 For example, educators in our curricula include the chief quality officer, chief operating officer, vice chair(s) for quality, director of community outreach, professors from arts and sciences, and dean. Some educators, particularly those from the QI or patient safety environment, have not traditionally viewed themselves as educators, yet have an inspiring passion for HSS coursework.

Each year, we modify sessions based on student feedback, and some require several iterations to work out the kinks. Each session requires close collaboration between the medical school’s HSS faculty and content experts, many of whom may not be familiar with medical school goals, objectives, and processes and many of whom do not consider themselves medical school faculty. But they are absolutely critical to HSS education, and they come from many parts of the education and health system. Unlike a department of public health sciences, for example, with dedicated faculty with protected time and established “educator” identity, HSS has a fragmented educator network that requires a high degree of alignment by the medical school. These complexities create unique features for classroom-based sessions that are different from other courses, creating a cognitive divide for some students’ expectations. The moving target of faculty and content creates real-time challenges for consistency and quality checks.

Heterogeneity of HSS pedagogies

At the core of HSS education lies a shift away from rote memorization toward application of systems thinking to complex adaptive challenges occurring in health care delivery.37–39 HSS competencies are highly contextualized within a system and a patient construct.40,41 Unlike basic or clinical science learning, such as folate synthesis, Krebs cycle, or pathology of vasculitis, all of which may be conducive to expert-driven lectures, HSS requires active learning and grappling with understanding the interrelatedness of system components.41,42 As a result, mini-lectures, workshops, patient- and community-based panels, interprofessional activities in large-group settings, and group projects are the norm. New medical students may be more accustomed to passive learning methods from college coursework and concurrent medical school classes (e.g., lecture on gastrointestinal bleed). Some students are vocal in their dissatisfaction with such formats, stating that they are “inefficient,” and commenting, “Just tell me what I need to know.”

Tension with students’ perception of their professional role

Through HSS experiences, students learn about the challenges in health systems that are affecting patient health. This also causes frustration with being taught about these realities. Some students are frustrated because they believe that they are not in a position to make changes, that there is “nothing to do about it,” or that there is no clear answer. Alternatively, other students are frustrated because they see clear answers to “fix” systemic problems.43 Along with some faculty mentors, some students make it clear that knowing and improving health systems is not part of their professional role. This challenge is compounded by the fact that being a systems-minded physician requires the development of a new professional identity as much as knowledge acquisition.44–46 Most medical school courses last several weeks, focusing on a single organ system and allowing for a perceived “mastery” of the subject. HSS coursework, however, is a journey toward a new professional identity as a systems citizen—it is hard to see the end when you have only just started.26,46–48 This evolving professional identity is one that requires the traditional physician traits, such as altruism, trustworthiness, and respect for others.45,46 However, it also involves an expanded mindset that necessitates caring for and addressing insufficiencies in the system as obligatory aspects of the professional role.26,49 After the first few years of coursework, students have not yet seen the fruits of their travels and do not know the places they will soon visit. Near the final years of medical school, however, we see our students’ growing appreciation for HSS learning and complexities of the health care system. This is reflected in personal statements where systems experiences are highlighted, and many upper-year students are realizing that these experiences are prized by residency programs when they are discussed at length during interviews.

Unconducive culture to HSS integration

While on clinical rotations, when students see residents, fellows, and attending physicians embody certain behaviors, they may fail to see the “new” professional role that is being sought in HSS curricula.45,46 Students are not always embedded in high-functioning clinical teams, and it can be challenging for a medical school curriculum to create ideal HSS learning sites. Some of our students have asked us, “Why are we being taught something that is impractical or not reality?” Students may hear derogatory comments from physicians about other health care professionals, and they may witness a physician’s inability to participate meaningfully in HSS activities (e.g., a physician’s frustration with insufficient time in clinic to address end-of-life care planning). Students work with many excellent residents, faculty, and health care professionals, but it only takes a few comments about the “real” delivery of care for a student to become disengaged with HSS content. We need to build both educational and clinical systems where the HSS content is fully integrated into all curricular areas and the care delivery system, respectively.25,50,51 Until this occurs, we believe that educators will continue to encounter similar struggles.

Reexamination of Key Issues

We believe that these drivers influencing student receptivity of HSS coursework and the HSS performance gap (Figure 1B) prompt a reexamination of 5 key issues for medical schools (see Table 2 for descriptions and strategies). We hypothesize that addressing these 5 areas will accelerate recovery of the implementation dip and some closure of the HSS performance gap and will facilitate achievement of “new highs” for HSS curricula.

Table 2
Table 2:
Issues and Proposed Strategies to Address Mixed Student Receptivity With Health Systems Science Education

Medical school recruitment processes

If medical education has a social responsibility to prepare the next generation of physicians for practicing in evolving health systems, is it also our responsibility to adapt our admissions processes to achieve this outcome? Many students are fully engaged in HSS material, ask excellent questions, and pursue HSS-related scholarly work with faculty. Those who are not engaged can be vocal about their dissatisfaction with having to “deal with” HSS learning. Is it our responsibility to be recruiting more medical students who will be “early adopters” of HSS education and who will have the potential to fully embrace systems analysis and improvement as a professional responsibility?52 Do students fully understand the focus on HSS when they are entering medical school? Are colleges and universities preparing students ably to enter this new culture of medical school and health care? There are important implications on all sides of these questions, which must be considered as HSS evolves and establishes itself in undergraduate medical education.

Faculty development

Since many HSS educators have not been educated in HSS principles and/or are unfamiliar with the best teaching methods and larger context of the medical school environment, these educators need to be assisted by the medical school and health system to meet learners’ needs. Although requiring significant investment, there is a growing need to provide faculty with education to address their gaps in HSS skills.10,39,53 National conferences for faculty education exist (e.g., Quality and Safety Educators Academy); however, these programs are short immersive programs, devoid of local contextual factors, and focus more narrowly on “QI,” missing several key HSS competency areas.54 The Teachers of Quality Academy at East Carolina University, Health Systems Science Academy at Penn State, and Master of Health Care Delivery Science program at Dartmouth are examples of how local education initiatives can be created to help fill local needs.55 In addition, there is a need for creating educational programs and interventions focused on developing educators’ teaching skills to more fully embrace adult learning theories within the instructional methods they use.18 We believe a rate-limiting step is a transformational change in expectations for educators currently teaching HSS competencies.

Building an HSS academic “home”

In addition to faculty development, we strongly advocate for institutions to develop the faculty and systems to build the science of health systems. Many institutions have created centers that serve as an academic home for these researchers and improvers. The Dartmouth Institute for Health Policy and Clinical Practice, the Mayo Clinic Kern Center for the Science of Health Care Delivery, and the Division of Healthcare Delivery Science at New York University are but a few examples. In the United States, the National Institutes of Health created a National Center for Advancing Translational Sciences, and the Department of Veterans Affairs national VA Quality Scholars program provides training for physicians and doctoral-prepared nurses in the science of health care improvement. Even though these centers and programs use different titles, they are bound together by a common goal of advancing the systematic, scholarly efforts to improve the quality, safety, and value of health care services and raise the profile of HSS in academic settings.56,57 Additionally, publication guidelines like SQUIRE (Standards for Quality Improvement Reporting Excellence) and StaRI (Standards for Reporting Implementation Studies) provide templates for sharing HSS work in the scholarly literature.56,58 Together, these efforts will raise the academic profile of and create an academic “home” for HSS.

Evaluations

Given that HSS is still a nascent field, student satisfaction scores remain a primary “outcome” measure. Current higher-order competency assessments for students performing in clinical environments or through systems-related projects are insufficient.17 If HSS is to become an increasingly established area of undergraduate medical education, then more robust assessments and outcome data are required to further establish the field. Are learners achieving higher scores on HSS-related, knowledge-based board examinations? Is learners’ progression in systems-based practice and QI-related milestones more accelerated with exposure to such curricula? How can we ensure that patient and system outcomes are improved as learners transition into their internships, residencies, and practice? Also, changes to the AAMC Graduation Questionnaire that include additional HSS-related areas can help facilitate a medical school’s focus on HSS education and expand students’ perceptions of learning in these areas. We strongly encourage HSS educators and medical schools to look beyond student satisfaction ratings and continue to develop assessments of HSS knowledge, evaluation of skills, and assessments of learner attitudes while building opportunities for learners to be integrated into meaningful HSS work in health care settings.29,30,59,60 Ultimately, the success of HSS curricula should be reflected in better outcomes for patients, families, and communities and better systems of care delivery.

Transparent collaboration between medical schools

In our collaborations with medical schools implementing HSS curricula, we have learned that their struggles are similar to ours. The literature, however, is not transparent regarding these issues. This is likely due to publication bias of effective interventions and descriptions of pilot projects. For example, preliminary data from the University of California, San Francisco have suggested similar tensions related to HSS education, with students reporting some dissatisfaction with HSS coursework.22 Unfortunately, our best examples come from individual discussions with educators who discuss their “horror stories” of HSS curricula. One educator shared a story from a medical school where he implemented an HSS course in the early 2000s. For 5 years the school battled students’ dissatisfaction and unhappiness, and the school ultimately moved all of the material to an end-of-fourth-year course. As with any developing field, those on the vanguard will receive the most resistance. A true paradigm shift that includes the successful spread of HSS across medical education requires educators to share what works and what does not work, for whom, and under what circumstances. True science demands that we share our successes as well as our missteps so that others may learn from our work and not make similar mistakes.

Conclusion

As health systems evolve, medical education is rapidly adapting education programs to align with patient needs, which includes an increased focus on HSS. Several key challenges to HSS education have been identified, particularly students’ mixed receptivity and satisfaction, which is the primary, readily available metric to use in identifying the “performance gap” in medical education innovation. The performance gap can be further tracked using more objective HSS evaluation metrics as they are developed and implemented in medical schools. If a medical school’s goal is to develop HSS skills and a new professional identity as a systems citizen, then medical schools must grapple with the underlying struggles occurring in medical schools to accelerate change.

Acknowledgments:

The authors thank the students, faculty, and leadership at their medical schools for their contributions to and support of these educational programs.

References

1. Bodenheimer T, Sinsky C. From triple to quadruple aim: Care of the patient requires care of the provider. Ann Fam Med. 2014;12:573–576.
2. Berwick DM, Nolan TW, Whittington J. The triple aim: Care, health, and cost. Health Aff (Millwood). 2008;27:759–769.
3. Thibault GE. Reforming health professions education will require culture change and closer ties between classroom and practice. Health Aff (Millwood). 2013;32:1928–1932.
4. Skochelak SE. A decade of reports calling for change in medical education: What do they say? Acad Med. 2010;85(9)(suppl):S26–S33.
5. Skochelak SE. Commentary: A century of progress in medical education: What about the next 10 years? Acad Med. 2010;85:197–200.
6. Lucey CR. Medical education: Part of the problem and part of the solution. JAMA Intern Med. 2013;173:1639–1643.
7. Frenk J, Chen L, Bhutta ZA, et al. Health professionals for a new century: Transforming education to strengthen health systems in an interdependent world. Lancet. 2010;376:1923–1958.
8. Gonzalo JD, Haidet P, Papp KK, et al. Educating for the 21st-century health care system: An interdependent framework of basic, clinical, and systems sciences. Acad Med. 2017;92:35–39.
9. Combes JR, Arespacochaga E. Lifelong Learning—Physician Competency Development. 2012.Chicago, IL: American Hospital Association.
10. Crosson FJ, Leu J, Roemer BM, Ross MN. Gaps in residency training should be addressed to better prepare doctors for a twenty-first-century delivery system. Health Aff (Millwood). 2011;30:2142–2148.
11. Cooke M, Irby DM, O’Brien BC; Carnegie Foundation for the Advancement of Teaching. Educating Physicians: A Call for Reform of Medical School and Residency. 2010.San Francisco, CA: Jossey-Bass.
12. Ludmerer KM. Let Me Heal: The Opportunity to Preserve Excellence in American Medicine. 2015.Oxford, NY: Oxford University Press.
13. Finocchio LJ, Bailiff PJ, Grant RW, O’Neil EH. Professional competencies in the changing health care system: Physicians’ views on the importance and adequacy of formal training in medical school. Acad Med. 1995;70:1023–1028.
14. Skochelak SE, Hawkins RE; AMA Education Consortium. Health Systems Science. 2016.St. Louis, MO: Elsevier.
15. Ogrinc G, Headrick LA, Mutha S, Coleman MT, O’Donnell J, Miles PV. A framework for teaching medical students and residents about practice-based learning and improvement, synthesized from a literature review. Acad Med. 2003;78:748–756.
16. Starr SR, Reed DA, Essary A, et al. Science of health care delivery as a first step to advance undergraduate medical education: A multi-institutional collaboration. Healthc (Amst). 2017;5:98–104.
17. Gonzalo JD, Caverzagie KJ, Hawkins RE, Lawson L, Wolpaw DR, Chang A. Concerns and responses for integrating health systems science into medical education. Acad Med. 2018;93:843–849.
18. Gonzalo JD, Chang A, Wolpaw DR. New educator roles for health systems science: Implications of new physician competencies for U.S. medical school faculty. Acad Med. 2019;94:501–506.
19. Gonzalo JD, Haidet P, Blatt B, Wolpaw DR. Exploring challenges in implementing a health systems science curriculum: A qualitative analysis of student perceptions. Med Educ. 2016;50:523–531.
20. Gonzalo JD, Graaf D, Ahluwalia A, Wolpaw DR, Thompson BM. A practical guide for implementing and maintaining value-added clinical systems learning roles for medical students using a diffusion of innovations framework. Adv Health Sci Educ Theory Pract. 2018;23:699–720.
21. Butler JM, Anderson KA, Supiano MA, Weir CR. “It feels like a lot of extra work”: Resident attitudes about quality improvement and implications for an effective learning health care system. Acad Med. 2017;92:984–990.
22. Mills LM, Hoffman AB, Khan A, Lai CJ. Integrating health systems science in early undergraduate medical education: Barriers to implementation and lessons learned. MedEdPublish. 2017. doi:10.15694/mep.2017.000204
23. Pines AR, Khurana A, Mastorakos GM, Richter KR, Asprey WL. A prognosis for health systems science courses: Observations from current students. Acad Med. 2018;93:1434–1436.
24. Gonzalo JD, Baxley E, Borkan J, et al. Priority areas and potential solutions for successful integration and sustainment of health systems science in undergraduate medical education. Acad Med. 2017;92:63–69.
25. Improving Environments for Learning in the Health Professions: Recommendations from the Macy Foundation Conference. 2018. New York, NY: The Josiah Macy Jr. Foundation; http://macyfoundation.org/publications/publication/conference-summary-improving-environments-for-learning-in-the-health-profes. Accessed April 20, 2019.
26. Gonzalo JD, Wolpaw T, Wolpaw D. Curricular transformation in health systems science: The need for global change. Acad Med. 2018;93:1431–1433.
27. Eastwood KW, Louis KS. Restructuring that lasts: Managing the performance dip. J Sch Leadersh. 1992;2:212–224.
28. Gonzalo JD, Dekhtyar M, Starr SR, et al. Health systems science curricula in undergraduate medical education: Identifying and defining a potential curricular framework. Acad Med. 2017;92:123–131.
29. Gonzalo JD, Lucey C, Wolpaw T, Chang A. Value-added clinical systems learning roles for medical students that transform education and health: A guide for building partnerships between medical schools and health systems. Acad Med. 2017;92:602–607.
30. Gonzalo JD, Graaf D, Johannes B, Blatt B, Wolpaw DR. Adding value to the health care system: Identifying value-added systems roles for medical students. Am J Med Qual. 2017;32:261–270.
31. Association of American Medical Colleges. Medical School Graduation Questionnaire: 2013 All Schools Summary Report. 2013.Washington, DC: Association of American Medical Colleges.
32. McDermott C, Shank K, Shervinskie C, Gonzalo JD. Developing a professional identity as a change agent early in medical school: The students’ voice [published online ahead of print February 19, 2019]. J Gen Intern Med. doi:10.1007/s11606-019-04873-3
33. Bland CJ, Starnaman S, Wersal L, Moorehead-Rosenberg L, Zonia S, Henry R. Curricular change in medical schools: How to succeed. Acad Med. 2000;75:575–594.
34. Herold DM, Fedor DB. Change the Way You Lead Change: Leadership Strategies That Really Work. 2008.Stanford, CA: Stanford Business Books.
35. Witzburg RA, Sondheimer HM. Holistic review—Shaping the medical profession one applicant at a time. N Engl J Med. 2013;368:1565–1567.
36. Ludmerer KM. Time to Heal: American Medical Education From the Turn of the Century to the Era of Managed Care. 1999.Oxford, UK: Oxford University Press.
37. Cristancho S, Lingard L, Regehr G. From problem solving to problem definition: Scrutinizing the complex nature of clinical practice. Perspect Med Educ. 2017;6:54–57.
38. Colbert CY, Ogden PE, Ownby AR, Bowe C. Systems-based practice in graduate medical education: Systems thinking as the missing foundational construct. Teach Learn Med. 2011;23:179–185.
39. Gonzalo JD, Ahluwalia A, Hamilton M, Wolf H, Wolpaw DR, Thompson BM. Aligning education with health care transformation: Identifying a shared mental model of “new” faculty competencies for academic faculty. Acad Med. 2018;93:256–264.
40. Gonzalo JD, Thompson BM, Haidet P, Mann K, Wolpaw DR. A constructive reframing of student roles and systems learning in medical education using a communities of practice lens. Acad Med. 2017;92:1687–1694.
41. Mann KV. Theoretical perspectives in medical education: Past experience and future possibilities. Med Educ. 2011;45:60–68.
42. Wenger E. Communities of Practice: Learning, Meaning, and Identity. 1998.Cambridge, UK: Cambridge University Press.
43. Eisenstein L. To fight burnout, organize. N Engl J Med. 2018;379:509–511.
44. Cruess RL, Cruess SR, Steinert Y. Amending Miller’s pyramid to include professional identity formation. Acad Med. 2016;91:180–185.
45. Hafferty FW, Levinson D. Moving beyond nostalgia and motives: Towards a complexity science view of medical professionalism. Perspect Biol Med. 2008;51:599–615.
46. Brennan TA. Physicians’ professional responsibility to improve the quality of care. Acad Med. 2002;77:973–980.
47. Gonzalo J. Ensuring health professions students understand the systems of care. https://macyfoundation.org/news-and-commentary/ensuring-students-understand-systems-of-care. Published 2018. Accessed May 7, 2019.
48. Davis CR, Gonzalo JD. How medical schools can promote community collaboration through health systems science education. AMA J Ethics. 2019;21:E239–E247.
49. Senge P. Systems citizenship: The leadership mandate for this millennium. Reflections. 2006;7:1–8. https://www.conservationgateway.org/ConservationPlanning/cbd/guidance-document/key-advances/Documents/Systems%20Citizenship_The%20Leadership%20Mandate%20for%20this%20Millenium.pdf. Accessed May 7, 2019.
50. Headrick LA, Ogrinc G, Hoffman KG, et al. Exemplary care and learning sites: A model for achieving continual improvement in care and learning in the clinical setting. Acad Med. 2016;91:354–359.
51. Ogrinc G, Cohen ES, van Aalst R, et al. Clinical and educational outcomes of an integrated inpatient quality improvement curriculum for internal medicine residents. J Grad Med Educ. 2016;8:563–568.
52. Rogers EM. Diffusion of Innovations. 2003.5th ed. New York, NY: Free Press.
53. Lucas B. Getting the improvement habit. BMJ Qual Saf. 2016;25:400–403.
54. Myers JS, Tess A, Glasheen JJ, et al. The quality and safety educators academy: Fulfilling an unmet need for faculty development. Am J Med Qual. 2014;29:5–12.
55. Baxley EG, Lawson L, Garrison HG, et al. The teachers of quality academy: A learning community approach to preparing faculty to teach health systems science. Acad Med. 2016;91:1655–1660.
56. Ogrinc G, Davies L, Goodman D, Batalden P, Davidoff F, Stevens D. SQUIRE 2.0 (Standards for QUality Improvement Reporting Excellence): Revised publication guidelines from a detailed consensus process. BMJ Qual Saf. 2016;25:986–992.
57. Splaine ME, Ogrinc G, Gilman SC, et al. The Department of Veterans Affairs National Quality Scholars Fellowship Program: Experience from 10 years of training quality scholars. Acad Med. 2009;84:1741–1748.
58. Pinnock H, Barwick M, Carpenter CR, et al.; StaRI Group. Standards for Reporting Implementation studies (StaRI) statement. BMJ. 2017;356:i6795.
59. Gonzalo JD, Dekhtyar M, Hawkins RE, Wolpaw DR. How can medical students add value? Identifying roles, barriers, and strategies to advance the value of undergraduate medical education to patient care and the health system. Acad Med. 2017;92:1294–1301.
60. Moser EM, Fazio SB, Packer CD, et al. SOAP to SOAP-V: A new paradigm for teaching students high value care. Am J Med. 2017;130:1331–1336.e2.
Copyright © 2019 by the Association of American Medical Colleges