Secondary Logo

Journal Logo

Reviews

Social Accountability Frameworks and Their Implications for Medical Education and Program Evaluation: A Narrative Review

Barber, Cassandra MA; van der Vleuten, Cees PhD; Leppink, Jimmie PhD; Chahine, Saad PhD

Author Information
doi: 10.1097/ACM.0000000000003731

Abstract

There have been repeated international calls for medical schools to be socially accountable to the populations they intend to serve. While social accountability is an ideal that many institutions strive toward, measuring it remains a global challenge. With increasing societal demands for greater transparency and accountability, medical schools face growing pressures to produce stronger evidence of their social accountability.1,2

In 1995, the World Health Organization (WHO) defined social accountability as:

[T]he obligation of medical schools to direct their education, research and service activities towards addressing the priority health concerns of the community, region, and/or the nation they have a mandate to serve. The priority health needs are to be identified jointly by governments, healthcare organizations, health professionals and the public.3

Since then, the literature surrounding social accountability has expanded and the number of initiatives has multiplied.4,5 Many medical schools have embedded social accountability policies in their mission statements, program objectives, and strategic plans, and some organizations have included them in formal accreditation processes.5 Yet despite the growing interest, how social accountability is operationalized into measurable attributes remains elusive, making social accountability difficult to evaluate objectively.6

Although various policies and frameworks have been established to assist medical schools in the evaluation of social accountability, their descriptions of socially accountable principles, indicators, and parameters remain predominately conceptual in nature. The WHO’s social accountability definition, above, encompasses the 3 domains of medical education (education, research, and service activities), and this review addresses the educational domain. The purpose of this review is to identify and document common themes and indicators across large-scale social accountability frameworks, using a program evaluation model as an organizational framework. It is intended to facilitate the development of initial operational constructs needed to evaluate social accountability in medical education.

Background

Derived from the verb account, accountability in its simplest form means answerability, the obligation to provide an account and be held responsible for one’s actions.7,8 In education, accountability functions as a system to evaluate institutional effectiveness (i.e., how well institutions meet their goals), holding institutions responsible for results and promoting educational improvement.9–12 This system implies a sense of responsibility, transparency, and public trust, whereby educational institutions are obligated to answer to society for their actions.13,14 While many forms of accountability exist, they all address the following fundamental questions: Who is held to account, for what, to whom, and through what means?7,10,15

All medical schools are accountable to the public, regardless of whether they choose to acknowledge or address this obligation.3 Health professions education programs and any educational institutions responsible for preparing the future health care workforce are accountable to the medical profession; the public (patients, families, communities, and society); their educational products (graduates, service activities, and research activities); and future health care needs. As a form of accountability, social accountability is implicit, explicit, and anticipated, in that medical schools must produce competent graduates prepared to respond to the changing public health care needs within their local communities.16–20

The medical profession has been granted certain responsibilities and privileges by society. Through legislation, regulation, and accreditation, medical schools are entrusted to produce competent physicians who are prepared to meet the needs of society.21,22 This social role carries great responsibilities, signifying the intrinsic social contract between physicians and society.23 This social contract is specifically amplified in countries where medical education is government funded. As a result, medical schools face increasing societal pressures to provide evidence of a positive social return.3,16 Social accountability represents an omnipresent social contract that exists between medicine and society.24–31

Broadly, social accountability implies an entity’s commitment to the society it is intended to serve for its actions, conduct, and performance.32 The WHO’s definition of social accountability remains the most widely accepted internationally. In 2010, the Global Consensus for Social Accountability of Medical Schools reaffirmed this definition, emphasizing that social accountability is a measurable activity:

[A]n action to respond to current and future health needs and challenges in society while working collaboratively with key stakeholders; policy-makers; healthcare organizations; health-insurance providers, health professionals and civil society.19

Within the broader accountability literature, the term accountability is often referred to as a conceptual umbrella13,33,34 and used interchangeably with responsibility, answerability, or effectiveness to portray an image of trust, trustworthiness, or transparency. However, in the medical education literature, the terms accountable, responsible, and responsive are not equivalent. Differences between them are clearly defined within Boelen and Woollard’s social obligation scale.32 Their taxonomy represents a linear progression toward achieving social accountability: responsibility refers to a “state of awareness of duties to respond to society’s needs”; responsiveness refers to “a course of action addressing society’s needs”; and accountability represents a “measurable activity” to provide evidence that programs proactively meet the priority health care needs of society while working alongside key stakeholders to positively impact public health.32

Method

Program evaluation models are widely used in multiple fields to provide comprehensive evaluations of social policies, programs, and interventions.35–39 We conducted a narrative review40 using a program evaluation model as an organizational framework and a systematized process to review large-scale social accountability frameworks as well as journal articles and other documents from the medical education literature. We then synthesized key concepts using a qualitative approach.

Organizational framework

We selected Stufflebeam’s context–input–process–product (CIPP) model as the assessment tool to systematically identify social accountability complex needs, indicators, and outcomes.35 First conceptualized in the 1960s to provide greater accountability in education, this program evaluation model is an internationally used accountability model and widely accepted in medical education.35–37 As depicted in Figure 1, the CIPP model uses evaluation as a method for program improvement and accountability. It consists of 4 interrelated components and incorporates continuous quality improvement feedback loops to be used throughout the evaluation model.35,36,38

F1
Figure 1:
The CIPP (context–input–process–product) evaluation model, adapted from Stufflebeam,35 , 36 , 38 used as the organizational framework for this narrative review of social accountability frameworks in medical education. The solid black arrows represent the linear production function of the CIPP model, whereby educational inputs are transformed into educational products. The broken black arrows represent the continuous improvement feedback loop to be used throughout the model.

In the CIPP model,35–37context refers to background—a needs assessment used to help identify needs, objectives, and/or opportunities of an educational institution. Inputs refer to material and human resources needed for effective functioning of an educational institution. Inputs are used to determine the appropriate course of action(s) required to achieve program goals and objectives. Processes are used to guide the implementation of a program. Products refer to the quality of student learning and its usefulness for the individual and for society. Products are used to measure outcomes. In later iterations of the CIPP model, the product component was divided into 4 subcomponents to assess a program’s impact, effectiveness, sustainability, and transportability.36,38 The CIPP model is dynamic and views education as a production function, whereby educational inputs are transformed to educational outputs. While each component can be evaluated independently, no indicator independently represents an absolute measure of program performance.36,38

Selection and search criteria

Using an iterative process, we searched 5 electronic bibliographic databases and platforms (PubMed, Embase, ERIC, Web of Science, and Google Scholar) as well as the broader World Wide Web (using Google) for social accountability frameworks and peer-reviewed journal articles and documents applicable to medical education. These searches were limited to English-language documents. The searches were first conducted in October 2018 and then repeated on March 31, 2019, to include any more recent documents. Keywords used in the search strategies included social accountability OR responsibility, socially accountable OR responsible, and social policies. These words were searched in combination with medical education, medical schools, medical training programs, and health professions education subject heading terms. A sample database search strategy is provided in Supplemental Digital Appendix 1 at https://links.lww.com/ACADMED/B24.

Inclusion and exclusion criteria

Our focus was social accountability in medical schools. Key English-language policy frameworks and peer-reviewed documents published from 1990 (when the term social accountability explicitly emerged within the medical education literature) through March 2019 were eligible for inclusion. Documents that did not discuss social accountability frameworks were excluded. All documents identified in the searches underwent an inclusion review process by the research team. Two of the authors (C.B. and S.C.) screened all documents identified in the searches. The full research team met frequently to review the documents and come to consensus regarding eligibility requirements. Primary source social accountability frameworks which represented the foundational values, principles, and/or parameters of the attributes medical schools can strive toward to fulfill their social mandate and which were used in subsequent papers to conceptualize social accountability were included in the review. Subframeworks and/or program- or institution-specific documents were excluded as these built upon previously established frameworks and could lack generalizability.

Analysis

Thematic synthesis41–44 was used to describe common and unique elements across the included social accountability frameworks. Thematic synthesis involves the systematic coding of text using an inductive approach to generate themes.43,44 The 3-stage analytical process starts with line-by-line coding of text; followed by the development of descriptive themes, which we characterized using the 4 dimensions of the CIPP model as an organizational framework; and then the generation of analytical themes. Two of the authors (C.B. and S.C.) coded the included documents independently. Resulting themes were reviewed by the 2 coders and discussed within the research team until consensus was reached to ensure coding accuracy and inclusivity.

Results

From the initial sample of 33 documents,3,16,18–20,23,44–70 we selected 4 key large-scale social accountability policy frameworks3,16,18,19 for inclusion in the review (see Table 1 for an overview of the selected frameworks). These 4 primary source documents represent the foundational values, principles, and/or parameters of social accountability in medical education. Additionally, these documents have all been highly cited and used in subsequent papers to conceptualize social accountability. They were also used to inform the Training for Health Equity Network evaluation framework47 as well as various institution-specific education, research, and service activities.

T1
Table 1:
Key Large-Scale Social Accountability Frameworks in Medical Education Included in the Narrative Review

These frameworks include policy, definition, application, and evaluation of social accountability at the local, national, and international levels.71 Although these frameworks differ slightly, they all describe characteristics that can be used toward demonstrating social accountability. Commonalities include responding to local public health needs; working alongside key stakeholders in identifying existing and forthcoming societal public health needs; servicing surrounding communities; addressing physician shortages; increasing diversity within the admissions process to reflect local demographics and geography; producing competent medical professionals; and ensuring the curriculum reflects priority health needs.3,16,18,19

Our thematic synthesis identified 6 themes, including shared values and 5 indicators as they relate to the CIPP evaluation model: context (program objectives), inputs (actions), processes (activities), products (institutional outputs/outcomes), and impacts on societal health. While impact evaluation is a subcomponent of product evaluation in the CIPP model, given the emphasis of social accountability in medical education on impact in practice and improvement in public health, we treated impacts as a separate theme in our analysis. Additionally, we identified subthemes within each theme, as described below and depicted in Figure 2. A selection of quotes to illustrate the themes and subthemes is provided in Table 2.

T2
Table 2:
Themes Identified Across the 4 Large-Scale Social Accountability Frameworks Included in the Narrative Review, Using the CIPP Model as an Organizational Frameworka
F2
Figure 2:
Themes and subthemes that emerged from the thematic synthesis of the narrative review of social accountability frameworks in medical education mapped to the interrelated components of the CIPP (context–input–process–product) evaluation model, used as an organizational framework.35 , 36 , 38 Impact evaluation was added in later iterations of the CIPP model as a subcomponent of product evaluation.36 , 38 Six themes were identified in the 4 included social accountability frameworks3 , 16 , 18 , 19: shared values (inclusive of the 4 core social values) and 5 indicators as they relate to the CIPP evaluation model (context, inputs, processes, products, and impacts). The broken black arrows serve a dual purpose. First, they represent the far-reaching core social values, which extend across all dimensions of the CIPP model. These 4 core social values are intended to guide medical education program activities in education, research, and service across the training continuum.18 Additionally, the broken black arrows represent the continuous improvement feedback loop to be used throughout the model. The solid lines represent the linear production function of the CIPP model, whereby educational inputs are transformed into educational products.

Shared values

All 4 frameworks emphasized the 4 core social values (relevance, quality, effectiveness, and equity).3,16,18,19 These far-reaching values extend across all components of the CIPP model. Generally, the core social values refer to the conceptual ideals and well-intended attributes of social accountability intended to inform context (program objectives), inputs (actions), processes (activities), and products (institutional outputs/outcomes). They are action oriented and grounded in the identification of societal needs. They are intended to guide medical education program activities in education, research, and service across the training continuum.18

The core social values were originally conceptualized in 1995 by the WHO3 as a means to help medical schools evaluate their progress in addressing social accountability and have since been adapted by subsequent frameworks.16,18–20,23,44–70Relevance implies that a medical education program addresses priority health needs or concerns of the population, community, or nation using a systematic approach in education, research, and service activities.3,16,18,19Quality refers to providing individuals with the best possible care that is evidence based, comprehensive, and culturally sensitive.3,16,18,19Effectiveness refers to the utilization of health care resources (costs) and ensuring that the greatest impact on public health is achieved while making the best use of resources.3,16,18,19Equity refers to universal access and striving to ensure that all individuals have access to quality health care.3,16,18,19

The interrelationship between these core social values represents a universal social commitment to “building a health care system that is relevant to the needs of the community or nation and provides high-quality care that is cost-effective and equitable.”3 Medical school activities in education, research, and service as well as health policies must be reflective of these needs—they must relate to, respond to, and anticipate priority health needs of the population.3,16,18,19

In addition to the core social values, 3 of the included frameworks16,18,19 emphasized the value of professionalism as well as the following competencies: ethics, teamwork, cultural competence, leadership, communication, lifelong learning, and evidence-based practice. In the Canadian context, the values of academic freedom and clinical autonomy were also highlighted.16

CIPP model

Context.

Context is the first component in the CIPP model. Recurring subthemes that emerged across the frameworks included mission statements, community partnerships, and active contributions to health care policy.

Institutional or program mission statements, mandates, policies, objectives, and/or goals must reflect the core social values of social accountability and the explicit commitment to meeting societal health needs.3,16,18,19 These statements should be posted publicly and made easily accessible to the general population.16 Additionally, the content and context specificity of a medical school’s mission statement and activities in education, research, and service should be inspired by and aligned with the current and anticipated priority health needs or concerns of the community and/or nation the institution serves.3,19 These mission statements serve as needs assessments and are intended to guide institutions’ education, research, and service activities to demonstrate their social obligation and commitment to society.3

Developing effective community partnerships with local health systems as well as other stakeholders is also important.3,16,18,19 Medical schools are more likely to improve their effectiveness if they work collaboratively with other stakeholders to establish priorities and identify current and future health needs.18 The local community serves as the primary stakeholder of all medical schools.19 Therefore, it is imperative that schools work in partnership with local stakeholders responsible for health care policy, planning, and finance to identify priority health needs as well as services and resources required for optimal patient care.3,16,18,19 Partnerships with affiliated health care organizations, professional groups, governments, consumers, and civil society could facilitate and encourage shared work on health planning, policy development, health care delivery, and evaluation.16

Medical education programs also play an important role in shaping the health care system. Community partnerships would serve as a means for medical schools to actively contribute to health care policy.3,16,18,19 Medical schools should act as catalysts of change and actively contribute to the sustainability and evaluation of health care planning and delivery, and policy development.16

Inputs.

Inputs are actions taken by programs to meet targeted goals. These actions are motivated by institution/program mandates and mission statements, and they reflect the core social values of social accountability. Subthemes across frameworks included diversity and equity in recruitment and selection (students, faculty, and staff) and community population health profiles.

Two frameworks emphasized the importance of diversity and equity in the recruitment and selection of students.18,19 To meet the social commitments embedded within the core social values and mission statements, medical schools must adapt their recruitment and selection policies to increase the diversity of accepted applicants to include individuals from underrepresented populations and disadvantaged groups.18 Students should reflect the demographics of the general population—including race and ethnicity, visible minority or indigenous status, socioeconomic status, gender and sexual orientation, and religious affiliation—and reflect other disadvantaged groups, such as rural and underserved communities.19 Additionally, schools should implement strategic pipelines and/or quotas for underrepresented groups as well as support mechanisms (e.g., financial aid, counseling services) to ensure equal opportunities for socially disadvantaged applicants.19 Medical schools should also ensure that faculty from medicine, health service delivery, and social science divisions are represented and involved in the curriculum and in programmatic decision making.18 Lastly, medical schools should matriculate students who are more likely to practice as generalists, as recommended by the WHO report.3

Another central theme in 3 frameworks was the need for medical schools to identify population needs as well as service gaps of a targeted community and/or nation.3,18,19 Schools can begin to identify these needs through well-defined population health research and the development of a comprehensive community population health profile.3,18 These profiles must reflect the community’s sociodemographic and geopolitical composition as well as population health risks, social determinants of health, and barriers to accessing services.

Processes.

Processes include the entire spectrum of educational activities: curricular content and structure; teaching methods; community-based clinical training opportunities and learning exposures to local populations and underserviced areas; learning assessments; continuing professional development; and evaluation systems. Recurring subthemes that emerged across frameworks included curricular activities as well as community-based clinical training opportunities and learning exposures.

Medical schools must direct their curricular activities toward addressing priority public health needs.3,16,18,19 Curricular content and structure should be approached using a student-centered paradigm and must include the social determinants of health; public health risks; and the geopolitical, sociodemographic, and epidemiological specificities of a population, community, and/or nation.3,19 Additionally, schools’ curricular activities should support lifelong learning opportunities for faulty, graduates, and staff through the availability of robust continuing professional development programs.16,18

Community-based clinical training opportunities and learning exposures should be designed using a population approach.3,19 Medical schools should promote primary care and provide learning opportunities and exposure to primary care practices.3,16,19 Additionally, schools should provide longitudinal community-based learning experiences that are relevant to the community’s health needs.3,19 Lastly, schools should provide learning opportunities in rural health care settings as well as exposure to disadvantaged and underserved groups.3,19

Products.

Product evaluation refers to the usability of a program’s graduates. Recurring subthemes that emerged across frameworks included physician resource planning, quality assurance, and program evaluation and accreditation.

All 4 frameworks emphasized the importance of physician resource planning. Medical schools should be actively involved in determining and educating the right composition of students and in determining the distribution, deployment, and retention of graduates necessary to meet social needs.3,16,18,19 Additionally, schools must ensure local employment opportunities for primary care physicians.18,19

Another central theme was the importance of program evaluation and accreditation.3,16,18,19 Accreditation standards and processes should incorporate social accountability principles.3,16,18,19 Evaluation and accreditation must be conducted at regular intervals,3,19 and the results should be made publicly available and used for institutional improvement.19 Additionally, evaluation and accreditation teams should be widely representative of stakeholders, including policymakers, health professionals, and community members.19

Lastly, the importance of embracing a continuous quality assurance process in education, research, and service delivery was emphasized across all frameworks.3,16,18,19 This process should be transparent and guided using well-defined standards to promote educational improvements.3,19 Additionally, graduate competencies must be assessed regularly and reflect well-defined educational standards to ensure quality of care and that graduates enter practice equipped with the skills required to meet changing public health needs.3,16,18,19

Impacts.

The premise of social accountability requires that the purpose and practices of medical education programs commence in the identification of societal needs and conclude in meeting those needs.18 Impact evaluation is part of product evaluation. A common theme highlighted across frameworks was overall improvement in community health outcomes.3,16,18,19 Another common theme was reduction and prevention of community health risks and morbidity and mortality of community diseases.16,19

To evaluate societal impacts effectively, medical schools must develop standards that span the educational continuum and focus on impacts of graduates in practice.19 They must develop metrics to assess the extent to which their graduates reduce the burden of illness and improve the health of the communities they serve.16 Medical schools must be able to demonstrate that the outcomes of their activities have positive impacts on community health.3,16,18,19 They have an obligation to ensure their graduates have a positive social return on investment to public health by reducing community health risks and the morbidity and mortality of community diseases.16,19

Discussion

This review identified major themes and indicators across 4 large-scale social accountability policy frameworks3,16,18,19 using the CIPP evaluation model.35 The CIPP model has not been used previously in the medical education literature to identify social accountability indicators across policy documents, but this review provides evidence of its utility in the development of initial operational constructs to evaluate social accountability in medical education. The themes explored in the included frameworks are consistent with the broader social accountability literature in medical education. However, the CIPP model provides an evaluation framework for medical education programs to strengthen their accountability systems.35–38 Additionally, this review also inadvertently addresses the fundamental questions of accountability: Who is held to account, for what, to whom, and through what means?7,10,15 These questions are critical to understanding accountability and can be used to help operationalize social accountability frameworks to better evaluate how and in which ways medical schools are socially accountable.

While this review focused primarily on social accountability of medical schools, it is important to acknowledge that social accountability is a dynamic process. It represents a collaborative relationship between citizens, government, training institutions, and health care educators/providers to systematically identify, prioritize, and address societal health needs.3,72 The measurement and systematic evaluation of social accountability in medical schools requires the use of a robust evaluation model to capture its conceptual and operational complexities. While accreditation may address many of these issues, it often serves a different purpose—ensuring medical schools produce competent graduates for the workplace. In this instance, schools are accountable to the accreditors. Canada and Australia have incorporated formal social accountability standards into their accreditation processes as a means to evaluate a medical school’s commitment to addressing the priority health concerns of the population.19,72,73 While this is a positive advancement, we need to continue to think about social accountability outcomes more broadly and establish meaningful relationships between educational inputs, outputs, and impacts.19,73

There is, however, an understudied assumption that medical schools meet societal needs. According to Boelen,72 only 1% of medical schools are socially accountable, whereas 9% of medical schools are socially responsive and 90% are socially responsible. While transparency and accountability initiatives have emerged as a key strategy for improving public services, the relationship between these initiatives and their impacts on public health remains largely unknown.74 This issue is not specific to medical education.75 There is a need to evaluate and demonstrate the social impacts graduates have in practice on communities and establish a link between theory and practice. This demonstration becomes less about providing public displays of good intentions and commitment to social accountability and more about proof of concept.75 A growing body of literature seeks to affirm the progress of individual medical schools toward becoming socially accountable (see Reeve et al76 for a systematic review). Some examples of medical schools’ efforts include widening access through admissions processes,77–82 curricular reforms reflecting social determinants of health,83–86 community-based clinical training opportunities and learning exposures,87–90 and location of learners.91–97

While progress in evaluating social accountability continues to expand in select medical education settings,98 the extent to which social accountability initiatives impact societal health remains largely unknown.99,100 However, a small number of empirical papers associate patient health outcomes with physician training and performance101,102 and some commentaries103–105 emphasize the need to link graduate outcomes with patient impacts using national clinical datasets to better understand the effects medical education programs have on public health needs.

Limitations

This review extends earlier work.3,16,18–20,23,44–70 It does not provide a comprehensive list of all possible social accountability indicators. The themes and indicators presented here are limited to primary source social accountability policy frameworks and are not necessarily inclusive of metrics used to assess quality. This review does not address more recent global health movements, for instance, the growing concerns regarding global health disparities. Additionally, the CIPP model assumes a top-down systems approach to education, whereby educational inputs are turned into products. This review is also primarily on medical education, not other interrelated and interdependent program activities of social accountability (i.e., research and service). Further research is needed to examine these relationships in more detail and determine whether medical schools address and respond to local health needs.

Conclusion

This review links an established program evaluation model and evidence from 4 large-scale social accountability policy frameworks, which may lead to the creation of indicators across the medical education continuum. Program evaluation models provide a systematic and easily understood practical guide for monitoring the progress of an institution toward desired goals and objectives. However, even when medical schools attempt to fulfill their social obligations, there is no guarantee that these actions will positively impact public health.3

The task of evaluating social accountability is complex.65 Most of the previous literature assessing the quality of medical education programs has focused predominantly on inputs and processes.72 As more emphasis is placed on social accountability, it is imperative that we as a community shift our focus from educational inputs and processes to products and impacts. There is a need to establish meaningful relationships between program inputs (who is trained and from where), products (what graduates do in practice, in what medical specialty, and where), and impacts (how graduates’ activities improve population health).32,74 We suggest a way to begin to establish these links is through the use of the CIPP program evaluation model.

References

1. Cook DA. Twelve tips for evaluating educational programs. Med Teach. 2010; 32:296–301
2. Fox J. Social accountability: What does the evidence really say? World Development. 2015; 72:346–361
3. Boelen C, Heck JE. Defining and Measuring the Social Accountability of Medical Schools. Geneva, Switzerland: World Health Organization, 1995. http://whqlibdoc.who.int/hq/1995/WHO_HRH_95.7.pdf. Accessed July 15, 2020
4. Woollard RF. Caring for a common future: Medical schools’ social accountability. Med Educ.2006; 40:301–313
5. Association of Faculties of Medicine of Canada (AFMC).. Future of medical education in Canada (FMEC): A collective vision for MD education. 2010. https://afmc.ca/sites/default/files/pdf/2010-FMEC-MD_EN.pdf. Accessed July 15, 2020
6. Reddy A, Lazreg SA, Phillips RL, Bazemore AW, Lucan SC. Towards defining and measuring social accountability in graduate medical education: A stakeholder study. JGME. 1993; 5:439–445
7. Romzek BS. Dynamics of public accountability in an era of reform. Int Rev Adm Sci. 2000; 66:21–44
8. Adams JE Jr, Kirst MW. New demands and concepts for educational accountability. Murphy J, Louis KS, eds. In: Handbook of Research on Educational Administration. 2nd edSan Francisco, CA: Jossey-Bass, 1999463–489
9. Lessinger L. Every Kid a Winner. Palo Alto, CA: Science Research Associates, 1970
10. Wagner R. Accountability in Education: A Philosophical Inquiry.New York, NY: Routledge, 1989
11. Rothman R. Measuring Up: Standards, Assessment and School Reforms. San Francisco, CA: Jossey-Bass;, 1995
12. Leithwood K, Edge K, Jantzi D. Educational Accountability: The State of the Art.Gutersloh, Germany: Bertelsmann, 1999
13. Fox J. The uncertain relationship between transparency and accountability. Dev Pract. 2007; 17:663–671
14. Muller J, Hernandez F. On the geography of accountability: Comparative analysis of teachers’ experiences across seven European countries. J Educ Change. 2010; 11:307–322
15. Trow M. Trust, markets and accountability in higher education: A comparative perspective. Research and Occasional Papers Series: CSHE.1.96, Berkeley, CA. Center for Studies in Higher Education, University of California, Berkeley, 1996
16. Health Canada. Social Accountability: A Vision for Canadian Medical Schools. Ottawa, ON, Canada: Health Canada, 2001. https://www.afmc.ca/future-of-medical-education-in-canada/medical-doctor-project/pdf/sa_vision_canadian_medical_schools_en.pdf. Accessed July 15, 2020
17. Palsdottir B, Neusy AJ, Reed G. Building the evidence base: Networking innovative socially accountable medical education programs. Educ Health (Abingdon). 2008; 21:177
18. Boelen C, Woollard B. Social accountability and accreditation: A new frontier for educational institutions. Med Educ. 2009; 43:887–894
19. Global Consensus for Social Accountability of Medical Schools. Global consensus for social accountability of medical schools. http://healthsocialaccountability.org. Published December 2010 Accessed July 15, 2020
20. Larkins SL, Reston R, Marre MC, et al. Measuring social accountability in health professional education: Development and international pilot testing of an evaluation framework. Med Teach. 2013; 35:32–45
21. Rourke J. AM last page. Social accountability of medical schools. Acad Med. 2013; 88:430
22. Ho K, Buote D, Jarvis-Selinger S, et al. Achieving social accountability through interprofessional collaboration: The Canadian medical schools experience. J Interprof Care. 2008; 22suppl 14–14
23. Buchman S, Woollard R, Meili R, Goel R. Practising social accountability: From theory to action. Can Fam Physician. 2016; 62:15–18
24. Welie JV. Social contract theory as a foundation of the social responsibilities of health professionals. Med Health Care Philos. 2012; 15:347–355
25. Carrin GJ. Rousseau’s “social contract”: Contracting ahead of its time? Bull World Health Organ. 2006; 84:916–918
26. Norjte N, De Jongh J. Professionalism—A case for medical education to honour the social contract. S Afr J Occup Ther.2017; 47:41–44
27. Lubchenco J. Entering the century of the environment: A new social contract for science. Science. 1998; 279:491–497
28. Cruess SR, Johnston S, Cruess RL. Professionalism for medicine: Opportunities and obligations. Med J Aust. 2002; 177:208–211
29. Cruess SR, Cruess RL. An overview of the origins of the social contract between physicians and society, with expectations and demands on both parties. Virtual Mentor. 2004; 6:185–188
30. Cruess RL, Cruess SR. Expectations and obligations: Professionalism and medicine’s social contract with society. Perspect Biol Med. 2008; 51:579–598
31. Cruess R, Cruess S. Updating the Hippocratic Oath to include medicine’s social contract. Med Educ. 2014; 48:95–100
32. Boelen C, Woollard R. Social accountability: The extra leap to excellence for educational institutions. Med Teach. 2011; 33:614–619
33. Bovens M. Analysing and assessing public accountability. A conceptual framework. European Governance Papers (EUROGOV). 2006;C-06-01:1–37. https://www.ihs.ac.at/publications/lib/ep7.pdf. Accessed July 15, 2020.
34. Mulgan R. “Accountability”:. An ever-expanding concept? Public Administration. 2000; 78:555–573
35. Frye AW, Hemmer PA. Program evaluation models and related theories: AMEE guide no. 67. Med Teach. 2012; 34:e288–e299
36. Stufflebeam DL. The CIPP model for evaluation. Kellaghan T, Stufflebeam DL, eds. In: International Handbook of Educational Evaluation. Kluwer International Handbooks of Education, vol 9Dordrecht, the Netherlands200331–62
37. Chinta R, Kebritchi M, Ellias J. A conceptual framework for evaluating higher education institutions. Int J Educ Manag. 2016; 30:989–1002
38. Stufflebeam DL. The relevance of the CIPP evaluation model for educational accountability. Presented at: Annual meeting of the American Association of School Administrators, February 24, 1971, Atlantic City, NJ
39. Kozica SL, Lombard CB, Hider K, Harrison CL, Teede HJ. Developing comprehensive health promotion evaluations: A methodological review. MOJ Public Heal. 2014; 1:39–48
40. Grant MJ, Booth A. A typology of reviews: An analysis of 14 review types and associated methodologies. Health Info Libr J. 2009; 26:91–108
41. Boyatzis RE. Transforming Qualitative Information: Thematic Analysis and Code Development. London, UK: Sage, 1998
42. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006; 3:77–101
43. Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol. 2008; 8:45
44. Dixon-Woods M, Agarwal S, Jones D, Young B, Sutton A. Synthesising qualitative and quantitative evidence: A review of possible methods. J Health Serv Res Policy. 2005; 10:45–53
45. Dijk S, Glasner J. Student’s toolkit on social accountability in medical schools. The International Federation of Medical Students’ Associations. https://ifmsa.org/social-accountability. Published 2017 Accessed July 15, 2020
46. Ross SJ, Preston R, Lindemann IC, et al. The training for health equity network evaluation framework: A pilot study at five health professional schools. Educ Health (Abingdon). 2014; 27:116–126
47. THEnet: Training for Health Equity Network. THEnet’s evaluation framework for socially accountable health professional education. https://gcsanetwork.files.wordpress.com/2012/02/monograph-print-quality-feb-1.pdf. Published 2011 Accessed July 15, 2020
48. Sandhu G, Garcha I, Sleeth J, Yeates K, Walker GR. AIDER: A model for social accountability in medical education and practice. Med Teach. 2013; 35:e1403–e1408
49. The International Federation of Medical Students’ Associations (IFMSA). IFMSA Policy proposal: Social accountability in medical schools. https://ifmsa.org/wp-content/uploads/2018/03/Social-Accountability-In-Medical-Schools.pdf. Published March 2018 Accessed July 15, 2020
50. Boelen C. World Health Organization. Towards Unity for Health: Challenges and Opportunities for Partnership in Health Development.Geneva, Switzerland:: WHO, 2000. https://www.who.int/hrh/documents/en/TUFH_challenges.pdf. Accessed July 15, 2020
51. Boelen C, Dharamsi S, Gibbs T. The social accountability of medical schools and its indicators. Educ Health (Abingdon). 2012; 25:180–194
52. Boelen C, Pearson D, Kaufman A, et al. Producing a socially accountable medical school: AMEE guide no. 109. Med Teach. 2016; 38:1078–1091
53. General Medical Council. Tomorrow’s Doctors: Recommendations on undergraduate medical education. http://www.ub.edu/medicina_unitateducaciomedica/documentos/TomorrowsDoctors_2009.pdf. Published September 2009 Accessed July 15, 2020
54. Peabody JW. Measuring the social responsiveness of medical schools: Setting the standards. Acad Med. 1999; 748 supplS59–S68
55. Lindgren S, Karle H. Social accountability of medical education: Aspects on global accreditation. Med Teach. 2011; 33:667–672
56. Puschel K, Rojas P, Erazo A, Thompson B, Lopez J, Barros J. Social accountability of medical schools and academic primary care training in Latin America: Principles but not practice. Fam Pract. 2014; 31:399–408
57. Frenk J, Chen L, Bhutta ZA, et al. Health professionals for a new century: Transforming education to strengthen health systems in an interdependent world. Lancet. 2010; 376:1923–1958
58. Biggs JS, Wells RW. The social mission of Australian medical schools in a time of expansion. Aust Health Rev. 2011; 35:424–429
59. World Federation for Medical Education. Basic Medical Education: WFME Global Standards for Quality Improvement. Copenhagen, Denmark: WFME, 2015. https://wfme.org/standards/bme. Accessed July 15, 2020
60. Boelen C, Bandaranayake R, Bouhuijs PAJ, Page GG, Rothman AI. Towards the assessment of quality in medical education. WHO/HRH/92.7. Geneva, Switzerland: World Health Organization, 1992
61. Boelen C. The Five-Star Doctor: An Asset to Health Care Reform?. Geneva, Switzerland: World Health Organization, 1993. https://www.who.int/hrh/en/HRDJ_1_1_02.pdf. Accessed July 15, 2020
62. Association of Faculties of Medicine of Canada (AFMC). The Future of Medical Education in Canada. A Collective Vision for Postgraduate Medical Education in Canada. https://afmc.ca/sites/default/files/pdf/2012-FMEC-MD_EN.pdf. Published 2012 Accessed July 15, 2020
63. Ventres W, Dharamsi S. Socially accountable medical education—The REVOLUTIONS framework. Acad Med. 2015; 90:1728
64. Leinster S. Evaluation and assessment of social accountability in medical schools. Med Teach. 2011; 33:673–676
65. Murray RB, Larkins S, Russell H. Medical schools as agents of change: Socially accountable medical education. Med J Aust. 2012; 196:51
66. Reddy AT, Lazreg SA, Phillips RL Jr, Bazemore AW, Lucan SC. Toward defining and measuring social accountability in graduate medical education: A stakeholder study. J Grad Med Educ. 2013; 5:439–445
67. Preston R, Larkins S, Taylor J, Judd J. Building blocks for social accountability: A conceptual framework to guide medical schools. BMC Med Educ. 2016; 16:227
68. Meili R, Ganem-Cuenca A, Leung JW, Zaleschuk D. The CARE model of social accountability: Promoting cultural change. Acad Med. 2011; 86:1114–1119
69. Goel R, Buchman S, Meili R, Woollard R. Social accountability at the micro level: One patient at a time. Can Fam Physician. 2016; 62:287–290299–302
70. Mullan F, Chen C, Petterson S, Kolsky G, Spagnola M. The social mission of medical education: Ranking the schools. Ann Intern Med. 2010; 152:804–811
71. Gibbs T. Sexy words but impotent curricula: Can social accountability be the change agent of the future? Med Teach. 2011; 33:605–607
72. Boelen C. Why should social accountability be a benchmark for excellence in medical education. Educación Médica. 2016; 17:101–105
73. Boelen C. Adapting health care institutions and medical schools to societies’ needs. Acad Med. 1999; 748 supplS11–S20
74. Woollard B, Boelen C. Seeking impact of medical schools on health: Meeting the challenges of social accountability. Med Educ. 2012; 46:21–27
75. Joshi A. Do they work? Assessing the impact of transparency and accountability initiatives in service delivery. Dev Policy Rev. 2013; 31:s29–s48
76. Reeve C, Woolley T, Ross SJ, et al. The impact of socially-accountable health professional education: A systematic review of the literature. Med Teach. 2017; 39:67–73
77. Ployhart RE, Holtz BC. The diversity–validity dilemma: Strategies for reducing racioethnic and sex subgroup differences and adverse impact in selection. Pers Psychol. 2008; 61:153–172
78. Kravitz DA. The diversity-validity dilemma: Beyond selection—The role of affirmative action. Pers Psychol. 2008; 61:173–93
79. Larkins S, Michielsen K, Iputo J, et al. Impact of selection strategies on representation of underserved populations and intention to practise: International findings. Med Educ. 2015; 49:60–72
80. Young ME, Razack S, Hanson MD, et al. Calling for a broader conceptualization of diversity: Surface and deep diversity in four Canadian medical schools. Acad Med. 2012; 87:1501–1510
81. Girotti JA, Park YS, Tekian A. Ensuring a fair and equitable selection of students to serve society’s health care needs. Med Educ. 2015; 49:84–92
82. Patterson F, Roberts C, Hanson MD, et al. 2018 Ottawa consensus statement: Selection and recruitment to the healthcare professions. Med Teach. 2018; 40:1091–1101
83. Allan J, Barwick TA, Cashman S, et al. Clinical prevention and population health: Curriculum framework for health professions. Am J Prev Med. 2004; 27:471–476
84. Dharamsi S, Ho A, Spadafora SM, Woollard R. The physician as health advocate: Translating the quest for social responsibility into medical education and practice. Acad Med. 2011; 86:1108–1113
85. Kaprielian VS, Silberberg M, McDonald MA, et al. Teaching population health: A competency map approach to education. Acad Med. 2013; 88:626–637
86. Maeshiro R, Johnson I, Koo D, et al. Medical education for a healthier population: Reflections on the Flexner Report from a public health perspective. Acad Med. 2010; 85:211–219
87. Strasser RP. Community engagement: A key to successful rural clinical education. Rural Remote Health. 2010; 10:1543
88. Strasser R, Hogenbirk JC, Minore B, et al. Transforming health professional education through social accountability: Canada’s Northern Ontario School of Medicine. Med Teach. 2013; 35:490–496
89. Worley P, Prideaux D, Strasser R, Magarey A, March R. Empirical evidence for symbiotic medical education: A comparative analysis of community and tertiary-based programmes. Med Educ. 2006; 40:109–116
90. Worley P, Silagy C, Prideaux D, Newble D, Jones A. The parallel rural community curriculum: An integrated clinical curriculum based in rural general practice. Med Educ. 2000; 34:558–565
91. Brooks RG, Walsh M, Mardon RE, Lewis M, Clawson A. The roles of nature and nurture in the recruitment and retention of primary care physicians in rural areas: A review of the literature. Acad Med. 2002; 77:790–798
92. Chan BT, Degani N, Crichton T, et al. Factors influencing family physicians to enter rural practice: Does rural or urban background make a difference? Can Fam Physician. 2005; 51:1246–1247
93. Dunbabin JS, Levitt L. Rural origin and rural medical exposure: Their impact on the rural and remote medical workforce in Australia. Rural Remote Health. 2003; 3:212
94. Easterbrook M, Godwin M, Wilson R, et al. Rural background and clinical rural rotations during medical training: Effect on practice location. CMAJ. 1999; 160:1159–1163
95. Glasser M, Hunsaker M, Sweet K, MacDowell M, Meurer M. A comprehensive medical education program response to rural primary care needs. Acad Med. 2008; 83:952–961
96. Tesson G, Curran V, Pong R, Strasser R. Advances in rural medical education in three countries: Canada, the United States and Australia. Educ Health (Abingdon). 2005; 18:405–415
97. Wilkinson D, Laven G, Pratt N, Beilby J. Impact of undergraduate and postgraduate rural training, and medical school entry criteria on rural practice among Australian general practitioners: National study of 2414 doctors. Med Educ. 2003; 37:809–814
98. Strasser R, Worley P, Cristobal F, et al. Putting communities in the driver’s seat: The realities of community-engaged medical education. Acad Med. 2015; 90:1466–1470
99. Sharma M, Pinto AD, Kumagai AK. Teaching the social determinants of health: A path to equity or a road to nowhere? Acad Med. 2018; 93:25–30
100. Ventres W, Boelen C, Haq C. Time for action: Key considerations for implementing social accountability in the education of health professionals. Adv Health Sci Educ Theory Pract. 2018; 23:853–862
101. Kaplan SH, Griffith JL, Price LL, Pawlson LG, Greenfield S. Improving the reliability of physician performance assessment: Identifying the “physician effect” on quality and creating composite measures. Med Care. 2009; 47:378–387
102. Hong CS, Atlas SJ, Chang Y, et al. Relationship between patient panel characteristics and primary care physician clinical performance rankings. JAMA. 2010; 304:1107–1113
103. Triola MM, Hawkins RE, Skochelak SE. The time is now: Using graduates’ practice data to drive medical education reform. Acad Med. 2018; 93:826–828
104. Teodorczuk A, Yardley S, Patel R, et al. Medical education research should extend further into clinical practice. Med Educ. 2017; 51:1098–1100
105. Chahine S, Kulasegaram KM, Wright S, et al. A call to investigate the relationship between education and health outcomes using big data. Acad Med. 2018; 93:829–832

Supplemental Digital Content

Copyright © 2020 by the Association of American Medical Colleges