Concern about the quality of undergraduate medical education (UME) is growing internationally. There are over 3,000 medical schools in the world, twice as many as there were 20 years ago,1 and international migration among medical students and graduates is increasing.2
Many believe that accreditation systems can address quality concerns by evaluating UME schools and programs against preexisting standards and guiding their improvement. In 2010, the Educational Commission for Foreign Medical Graduates (ECFMG) announced that by 2023, graduation from an appropriately accredited medical school would be required for all physicians seeking to practice in the United States.3 As a result, the World Federation for Medical Education (WFME) began its Recognition Programme,4 in which it evaluates UME-accrediting agencies to determine their level of conformity with its recognition criteria. Only graduates from schools accredited by WFME-recognized agencies would meet the new ECFMG requirement. Additionally, the World Health Assembly incorporated an accreditation recommendation into its Global Strategy on Human Resources for Health: Workforce 2030, which set a goal for all countries to have accreditation mechanisms for health training programs by 2020,5 and the World Medical Association endorsed ongoing development of accreditation systems internationally.6
Much work would be needed to develop and evaluate a global system of UME-accrediting authorities that conformed to WFME recognition criteria. As of September 2018, the Directory of Organizations that Recognize/Accredit Medical Schools (DORA) listed 134 accrediting agencies that cover 117 countries7; however, only 10 of these agencies had been formally recognized by the WFME.8 An additional 68 countries with medical schools have no known accrediting authority.
The creation of new accreditation systems, and enhancement to existing ones, should be based on evidence to ensure more efficient use of resources and more effective quality assurance practices. However, the evidence related to UME accreditation has not been systematically summarized, and concerns linger that, despite growing momentum to implement accreditation around the world, evidence informing UME accreditation practices is limited.9
The goal of this Review was to summarize the state of the evidence related to UME accreditation internationally, describe from whom and where the existing evidence has come, and identify opportunities for further investigation. Therefore, we conducted a scoping review, which is performed when one seeks “to examine the extent, range, and nature of research activity in a given field.”10
Method
Definition of accreditation
The simplest definition of accreditation is the evaluation of programs (e.g., rather than individuals) against preexisting standards. Accreditation is a term that originated in the United States11 and has since been adopted by international organizations, including the WFME. The WFME recognition criteria for accrediting agencies define accreditation as a quality assurance process that includes development and revision of explicit standards for medical schools (or programs), a self-study, a site visit by experts, a report from the site visit team that is submitted to the accreditation agency, the agency’s review of the report, the agency’s summary decision and recommendations for the school, follow-up monitoring, and cyclical review.12 WFME recognition criteria also specify requirements for the agency’s structure, internal policies, and resources. For the purposes of this study, we conceptualized the above as researchable accreditation components (Table 1), which also align with elements described in a general accreditation framework that was recently developed.13
;)
Table 1: Researchable Components of Accreditation Agencies’ Structures and Processesa
Search strategy
In July 2017, we searched Embase, ERIC, PubMed, and Scopus from their respective inceptions and without language restrictions to identify peer-reviewed original or review articles on UME accreditation. We updated the search on January 31, 2018. Our complete search strategy is included in Supplemental Digital Appendix 1 (at https://links.lww.com/ACADMED/A707).
Article inclusion
Each title and abstract was screened in English independently by 2 reviewers on our study team (S.T., C.Z., and/or M.v.Z.) to identify studies that focused on (1) accreditation and (2) UME programs. Articles advanced to full-text review if at least 1 reviewer voted for inclusion. In most cases, articles were excluded because the accreditation described in the article was of an educational program at the graduate or continuing medical education level.
Two members of the study team (S.T., N.N., C.Z., and/or M.v.Z.) reviewed each full text independently to determine whether it represented UME accreditation scholarship. Scholarship was defined as meeting all of Glassick’s criteria14: clear goals, adequate preparation, appropriate methods, significant results, effective presentation, and reflective critique. All of Boyer’s forms of scholarship (i.e., discovery, application, integration, and teaching)15 were eligible for inclusion as long as all of Glassick’s criteria14 were met; however, because the scholarship of teaching was less likely to yield evidence for whether and how accreditation should be implemented, we did not systematically search for accreditation-related curricula. Additionally, we only included articles that explicitly reported the objective of addressing a UME accreditation topic; accordingly, studies were not included when accreditation emerged as a theme or other finding in the results of a study addressing another topic16,17 or when studies were performed by an accrediting agency using agency data but did not address a topic about accreditation processes or outcomes.18,19 If an article explicitly addressed a UME accreditation topic but did not meet all of Glassick’s criteria,14 we categorized it as nonscholarship. For example, narrative reviews, where the methods were unclear or not formally stated,20,21 were accreditation nonscholarship. Journalistic articles or letters to the editor in response to another article were excluded.
For non-English-language full texts, we used the translator function in Microsoft Word 2016 (Microsoft Corporation, Redmond, Washington) to assess them for inclusion. When that provided insufficient clarity, for Spanish-language articles, 1 author (N.N.), who is proficient in Spanish, translated articles to facilitate our group discussion. Articles in Korean and German were reviewed by study team members with individuals who were fluent in each language.
We supplemented our database search with hand searching of references from included UME scholarship and narrative review articles. We discussed findings during regular team meetings until we reached consensus for all article categorizations.
Data extraction and analysis
For all UME accreditation articles (i.e., both scholarship and nonscholarship), we extracted the complete citation, countries of authors (based on their stated affiliation in the article), whether at least 1 author had a clear affiliation with a regulatory authority (i.e., an accreditation agency or organization that has significant influence over accreditation policy or programs [e.g., WFME, ECFMG, the Foundation for Advancement of International Medical Education and Research (FAIMER), national professional organizations]), and the accreditation agency (or agencies) studied or discussed in the article. For UME accreditation scholarship articles, we also extracted the component (or components) of accreditation that was studied (or if accreditation was treated as a single intervention), aspects of the study’s design and methodology, the study’s key findings, and the reported source of funding. For each article, a single author extracted the data, which were verified by the rest of the author team.
We performed descriptive statistics for trends in the number of articles by the locations of authors, locations of accrediting agencies, and publication date. Country region and income classifications were based on the World Bank’s lists for fiscal year 2019,22 which were also used when aggregating low-income, low-middle-income, and upper-middle-income countries into a single (low- or middle-income country [LMIC]) category.
Results
Our search for peer-reviewed UME accreditation articles in July 2017 yielded 2,104 citations (1,298 nonduplicate citations), with 210 articles eligible for full-text review. Our updated search on January 31, 2018, yielded an additional 102 citations (81 nonduplicate citations), with 5 articles eligible for full-text review. This resulted in a total of 2,206 citations (1,379 nonduplicate citations), with 215 articles eligible for full-text review. Of these, 1 article’s full text could not be retrieved via interlibrary loan, and 11 articles proved to be duplicates that had slightly different citations or were published at different times yet had nearly the same text,23–25 leaving 203 unique, accessible articles. Of these, 29 (14%) qualified as UME accreditation scholarship articles, 72 (35%) had a UME accreditation focus but did not meet all of Glassick’s criteria14 for scholarship (i.e., qualified as UME accreditation nonscholarship), and 102 (50%) were excluded because they did not focus on UME accreditation. Hand searching identified 7 additional UME accreditation scholarship and 13 nonscholarship articles, resulting in a total of 36 UME accreditation scholarship26–61 and 85 nonscholarship articles.20,21,24,62–143 A flowchart of our literature search, title and abstract screening, and full-text review is given in Figure 1.
Figure 1: Flowchart showing the study selection process used in a scoping review (through January 31, 2018) to summarize the state of the evidence related to undergraduate medical education accreditation internationally, describe from whom and where the existing evidence has come, and identify opportunities for further investigation. The authors followed the PRISMA extension for scoping reviews guidelines.
153UME accreditation scholarship articles
A complete list of the included UME accreditation scholarship articles is given in Appendix 1.
Agencies, authors, funding sources, and timing.
Across the 36 UME accreditation scholarship articles, 26 national or regional accrediting agencies and the WFME were investigated (see Supplemental Digital Appendix 2 at https://links.lww.com/ACADMED/A707). An accrediting authority from each World Bank region was included in at least 1 study (Figure 2). Accreditation agencies from high-income countries were featured most often (28/60,* 47%), with the United States’ Liaison Committee on Medical Education (LCME) included in 13 (36%) studies. Agencies from low-income countries were not included in any articles. A total of 21 (58%) studies had at least 1 author from the United States or Canada, while 8 (22%) had at least 1 author from an LMIC. Sixteen (44%) studies had at least 1 author affiliated with a regulatory authority. Funding was not mentioned for 23 (64%) studies and reported as no funding for 7 (19%) studies. Of the 6 (17%) studies that reported funding sources, 3 sources were governmental,28,41,58 2 were university-related sources,32,54 and 1 was a nongovernmental organization source.47 Studies appeared more frequently over time, with 5 (14%) published in the 1990s, 8 (22%) in the 2000s, and 23 (64%) in 2010–2018 (Figure 3).
Figure 2: Total number of UME accreditation articles that focus on accrediting agency by the agency’s country income level and region in a scoping review (through January 31, 2018) to summarize the state of the evidence related to UME accreditation internationally, describe from whom and where the existing evidence has come, and identify opportunities for further investigation. The authors used the World Bank’s lists for fiscal year 2019
22 to classify country income level and region. The WFME is also shown, but as it is international, it is not listed by country income level and region. Abbreviations: UME indicates undergraduate medical education; WFME, World Federation for Medical Education.
Total number of undergraduate medical education (UME) accreditation articles by publication year in a scoping review (through January 31, 2018) to summarize the state of the evidence related to UME accreditation internationally, describe from whom and where the existing evidence has come, and identify opportunities for further investigation.
Topics, methods, and outcomes.
Of the 36 scholarship articles, we classified 30 (83%) as scholarship of discovery (i.e., research) and 6 (17%) as scholarship of application; no reviews (i.e., scholarship of integration) were identified that reported formal methods (Appendix 1). Eighteen (50%) articles focused on a single component of accreditation, 10 (28%) examined multiple components, and 9 (25%) studied accreditation as a single intervention, with 1 study looking at accreditation as both a single intervention and as having multiple components.40 Accreditation standards for schools and/or programs were examined in a majority of studies (20; 56%), followed by institutional self-studies (13; 36%).
Of the 30 research studies, 12 (40%) included only analysis of accreditation documents. Surveys were performed in 9 (30%) studies, secondary data analysis in 7 (23%) studies, and interviews or focus groups in 4 (13%) studies. All studies were cross-sectional or retrospective in design.
Additionally, of the 30 research studies, a total of 5 (17%) attempted to determine whether UME accreditation improved educational outcomes.26,38,48,51,55 Three of these compared outcomes between accredited and nonaccredited medical schools.48,51,55 Of those 3, 1 study examined accreditation at the country level, finding that United States Medical Licensing Examination (USMLE) Steps 1, 2 Clinical Knowledge (CK), and 2 Clinical Skills (CS) performance were better among those from schools in Caribbean countries with accreditation versus those from schools in Caribbean countries without accreditation, although in comparisons for non-Caribbean countries, there were only slight differences for Step 2 CS and no differences for Step 1 or 2 CK.51 The other 2 studies examined relationships at the school level in countries where accreditation was voluntary. The first found that in Mexico, first-attempt pass rates for USMLE Step 1 (but not Step 2 CK or 2 CS) were better among those from accredited versus nonaccredited schools, while in the Philippines, first-attempt pass rates for USMLE Steps 1, 2 CK, and 2 CS as well as rates of successful ECFMG certification were better among those from accredited schools.48 The second study showed that residency exam scores and placements in Mexico were better among those from accredited versus nonaccredited schools.55
UME accreditation nonscholarship articles
A complete list of the included UME accreditation nonscholarship articles is given in Supplemental Digital Appendix 3 (at https://links.lww.com/ACADMED/A707).
Agencies, authors, and timing.
Across the 85 UME accreditation nonscholarship articles, 22 national or regional accrediting authorities and the WFME were discussed (see Supplemental Digital Appendix 2 at https://links.lww.com/ACADMED/A707). Agencies from North America (as defined by the World Bank) (46/101,† 46%) and from high-income countries (70/101,‡ 69%) were discussed most frequently (Figure 2), with the LCME being a focus in 32 (38%) articles, the WFME in 19 (22%), and the Commission on Osteopathic College Accreditation (and its predecessor the Bureau of Professional Education) in 12 (14%). Forty-four (52%) articles had at least 1 author from the United States or Canada, 8 (9%) had an Australian author, and 11 (13%) had at least 1 author from an LMIC. Forty-three (51%) articles had at least 1 author affiliated with a regulatory authority. Three (4%) were published in 1969–1979, 1 (1%) in the 1980s, 11 (13%) in the 1990s, 33 (39%) in the 2000s, and 37 (44%) in 2010–2018 (Figure 3).
Topics.
Articles addressed an array of issues. The most common types of articles were those that described the structure and function of national accrediting bodies (24; 28%), those published by schools describing their experiences with accreditation (16; 19%), and those about the WFME (8; 9%) and accreditation and quality (8; 9%).
Discussion
This study is the first known systematic effort to identify and describe the evidence underpinning the world’s UME accreditation systems. We found that while some accreditation components have been addressed by multiple studies, there is considerable potential for more research that can more effectively guide UME accreditation practices by strengthening research methods and studying unaddressed aspects of accreditation.
The quantity of UME accreditation articles has been increasing over time, with about half of all articles having been published in 2010 or later. While this may represent growing interest in accreditation, it also resembles general trends in medical education publication. The annual number of medical education articles nearly doubled from 2000 to 2010144 and has continued to increase so that currently approximately 5,000–6,000 medical education articles are indexed in PubMed each year. Systematic reviews on medical education topics often include larger numbers of studies than our scoping review despite having more focused research questions and search strategies. For example, a systematic review describing the prevalence of depression among medical students pooled 77 research studies in its meta-analysis.145 A systematic review on instruments to measure perceptions of the learning environment found 15 distinct instruments to measure medical student perceptions, which had been used in over 50 research studies.146 In our study, we applied a broad search strategy without setting time or language restrictions, yet we discovered only 36 scholarly articles on UME accreditation, 30 of which constituted research.
The quality of UME accreditation research was varied. Only 13 articles, dispersed over more than 2 decades, reported empiric data collection to address a research question, and all study designs were cross-sectional or retrospective. Indeed, 12 studies were based only on review of existing accrediting agency documents. A stark illustration of how such methods can be problematic is that 1 study, based on existing documents, concluded that the standards for medical schools and the processes of the Medical Council of India (MCI) were comparable to those of the LCME39; however, shortly after this study was published, there were public concerns about corruption in the MCI, and as a result of these concerns, the MCI is now being replaced with a new National Medical Commission.147
While, in general, the quantity and quality of UME accreditation scholarship were limited, we found multiple articles that addressed standards and medical school self-studies. Studies described the acceptability of standards nationally26 and internationally,56 the variation in standards across systems,39,49 and the relationships between the content and formatting of standards and corresponding educational outcomes26,38 and accreditation actions.43,44 There is evidence that describes processes for developing standards,41,57 the importance of aligning UME standards with standards in other stages of the educational continuum,60 and the revisions that may be needed when adapting existing standards to local contexts,28 which could help to guide the creation of new versions of standards. Frameworks for evaluation against accreditation standards34,45 and for conducting a self-study33 have also been published, and studies consistently reported formative benefits from medical school self-studies32,42,46,52–54 accompanied by considerable resource requirements.30,46 Together, these studies provide useful evidence to guide accreditation system development and enhancement, and their methods could be replicated in alternate settings. They also suggest that the content of UME accreditation standards is likely sufficient and appropriate for most contexts and that self-studies are a beneficial aspect of accreditation processes.
Some accreditation components were not addressed by any study and could be prioritized in future research. For example, no studies touched on agency policies and resources, the type or usefulness of formative feedback provided to medical schools, or interval or methods of follow-up after a full accreditation review. Moreover, all components of UME accreditation would benefit from further study. Rigorously evaluating current activities would ensure that accreditation is consistent and fair, contributes to improvements in educational practices and outcomes, and offers a favorable return on investment. Innovations could be implemented with research in mind; for example, experiments could be conducted in nascent accreditation systems using technologies that make data collection easier or that permit remote access to the expertise needed in full accreditation reviews. Further, randomized trials are possible within accreditation systems.148 Robust evidence can also be generated with observational designs,149 which could be facilitated by international data sharing and strengthened by making comparisons across systems. Linking UME accreditation data longitudinally with other data along the educational continuum could further enhance understanding of the components of accreditation that are critical to health care outcomes.150
When there is so much potential to advance UME accreditation scholarship, how can the work be done? Individuals affiliated with regulatory authorities can continue to examine their existing data and share findings with the academic community as they have done in the past. More research led by those who are not affiliated with accrediting authorities would also be beneficial in that it could decrease the chances of intellectual bias, diversify research questions and approaches, and facilitate synergies between policy researchers and policymakers.151 Additionally, the quality of scholarship would surely improve with increased funding.152 Given the investment that goes into accreditation implementation because it is believed to incentivize quality improvement in medical education and ultimately protect the public good, the apparent paucity of funding for accreditation research is disconcerting. Accreditation agencies can fund research related to their policies as the Accreditation Council for Graduate Medical Education has done for studies related to duty hours in postgraduate trainees in the United States.148 Greater investment in accreditation research by local authorities or international agencies advocating for accreditation could generate more evidence that supports rational development and implementation of UME accreditation.
Notably, the strongest evidence that UME accreditation improves outcomes comes from ecologic studies that treated accreditation as a single intervention. Within these studies, there were inconsistent relationships between UME accreditation and outcomes, and there was limited ability to control for confounding variables. Accreditation’s multifaceted nature and the wide variation in accreditation systems across settings further limit conclusions that can be drawn about UME accreditation’s usefulness from existing evidence. Additionally, we found that the number of UME accreditation nonscholarship articles outnumbered UME accreditation scholarship articles by more than 2:1 and that a majority of UME accreditation articles were from high-income settings with a large proportion coauthored by individuals affiliated with regulatory authorities. With a limited evidence base to support UME accreditation and with evidence suggesting that voices advocating for accreditation come primarily from resource-rich settings, there is a risk of accreditation colonialism. The principles of setting standards for UME, having programs evaluate themselves against those standards, and having an external peer review process are likely to be useful in any setting. However, investigation into how these principles are best applied in practice would ensure that they are optimized for their local contexts.
There are important limitations to consider when reviewing our work. First, while we attempted to be inclusive in identifying UME accreditation articles, articles in a foreign language may use a term that was not captured by our search strategies, and some areas may have accrediting activities that go by another name (e.g., the United Kingdom’s General Medical Council [GMC] is listed in DORA and has medical school standards, requires school self-evaluation, and performs school inspections but does not use the term accreditation to describe these activities). However, our database and hand searches found articles in multiple languages and from areas that do not regularly use the term accreditation. Second, the quantitative data we present to summarize our findings are based on our team’s consensus. While we classified articles according to well-accepted criteria, other teams may have arrived at different conclusions, which could influence the precision of our estimates. Third, some may argue that the usefulness of accreditation is self-evident or could be supported by the historical evidence of poorly functioning schools being identified after accreditation activities were initiated (e.g., in the United States in the 1900s or more recently in the Caribbean); however, these anecdotal experiences do not replace the need to scrutinize current practices or to permit peer and public review of evidence that relates to UME accreditation. This is especially true when large amounts of resources are diverted to accreditation activities—and, thus, away from other important activities, such as clinical care—which could have significant unintended consequences.
Conclusions
Assuring and improving the quality of education for future physicians is imperative, yet there is limited evidence to support existing UME accreditation practices or to guide the creation or improvement of accreditation systems. As new accreditation systems are being formed and refined, there is great potential to experiment, investigate, and iterate. More research studies (especially in low-resource settings) and research funding are required to optimize the value that UME accreditation systems provide to medical students, programs, and society.
Acknowledgments:
The authors gratefully acknowledge Dr. Carolyn Park, who provided translations of the Korean-language articles, and Dr. John Boulet, who reviewed our list of references.
References
1. Opalek A, Gordon D. A data model for medical schools and their programs: Structuring data to inform medical regulation worldwide. J Med Regul. 2018;104:5–12.
2. Merçay C, Dumont J-C, Lafortune G; Organization for Economic Cooperations and Development, ed. Changing patterns in the international migration of doctors and nurses to OECD countries. In: International Migration Outlook 2015. 2015:Paris, France: Organization for Economic Cooperations and Development; 105–176.
3. Educational Commission for Foreign Medical Graduates. Requiring medical school accreditation for ECFMG certification—Moving accreditation forward.
https://www.ecfmg.org/forms/rationale.pdf. Published September 21, 2010. Accessed May 30, 2019.
4. World Federation for Medical Education. WFME Recognition Programme.
https://wfme.org/accreditation/recognition-programme. Accessed May 30, 2019.
6. World Medical Association. WMA Declaration of Chicago on Quality Assurance in Medical Education.
https://www.wma.net/policies-post/wma-declaration-of-chicago-on-quality-assurance-in-medical-education. Published October 19, 2017. Accessed May 30, 2019.
7. Foundation for Advancement of International Medical Education and Research. Directory of Organizations that Recognize/Accredit Medical Schools (DORA).
https://www.faimer.org/resources/dora/index.html. Accessed May 30, 2019.
8. World Federation for Medical Education. Countries served by agencies with recognition status (map).
http://wfme.org/accreditation/accrediting-agencies-status. Accessed May 30, 2019.
9. Kulasegaram KM, Tonin P, Houston P, Whitehead C. Accreditation drives medical education. Does evidence drive accreditation? Med Educ. 2018;52:772–773.
10. Thomas A, Lubarsky S, Durning SJ, Young ME. Knowledge syntheses in medical education: Demystifying scoping reviews. Acad Med. 2017;92:161–166.
11. Young KE. Understanding Accreditation: Contemporary Perspectives on Issues and Practices in Evaluating Educational Quality. 1983.San Francisco, CA: Joseey-Bass Publishers.
12. World Federation for Medical Education. Recognition criteria.
http://wfme.org/accreditation/recognition-criteria. Accessed May 30, 2019.
13. Taber S, Akdemir N, Gorman L, van Zanten M, Frank J. A “fit for purpose” framework for medical education accreditation system design. BMC Med Educ. In press.
14. Glassick CE. Boyer’s expanded definitions of scholarship, the standards for assessing scholarship, and the elusiveness of the scholarship of teaching. Acad Med. 2000;75:877–880.
15. Boyer EL. Scholarship Reconsidered: Priorities of the Professoriate: A Special Report. 1990.Stanford, CA: Carnegie Foundation for the Advancement of Teaching.
16. Nair M, Webster P. Education for health professionals in the emerging market economies: A literature review. Med Educ. 2010;44:856–863.
17. Püschel K, Riquelme A, Moore P, Rojas V, Perry A, Sapag JC. What makes a medical school socially accountable in Latin America? A report from a Latin American panel based on a qualitative approach. Med Teach. 2017;39:415–421.
18. Kassebaum DG, Eaglen RH, Cutler ER. The objectives of medical education: Reflections in the accreditation looking glass. Acad Med. 1997;72:647–656.
19. Kassebaum DG, Eaglen RH. Shortcomings in the evaluation of students’ clinical skills and behaviors in medical school. Acad Med. 1999;74:842–849.
20. Boulet J, van Zanten M. Ensuring high-quality patient care: The role of accreditation, licensure, specialty certification and revalidation in medicine. Med Educ. 2014;48:75–86.
21. Davis DJ, Ringsted C. Accreditation of undergraduate and graduate medical education: How do the standards contribute to quality? Adv Health Sci Educ Theory Pract. 2006;11:305–313.
23. Ward D. History of osteopathic medical education accreditation. J Am Osteopath Assoc. 1994;94:920–921.
24. Ward WD, Retz KC. History of osteopathic medical education accreditation. J Am Osteopath Assoc. 1997;97:636–637.
25. [No authors listed.] History of osteopathic medical education accreditation. AOA Editorial Department. J Am Osteopath Assoc. 1988;88:1371–1372.
26. Kassebaum DG, Cutler ER, Eaglen RH. On the importance and validity of medical accreditation standards. Acad Med. 1998;73:550–564.
27. Yang EB. A study on the content validity and factor validity of accreditation standards for medical schools in Korea [in Korean]. Korean J Med Educ. 2002;14:85–97.
28. Ho MJ, Abbas J, Ahn D, Lai CW, Nara N, Shaw K. The “glocalization” of medical school accreditation: Case studies from Taiwan, South Korea, and Japan. Acad Med. 2017;92:1715–1722.
29. McArtor RE, Kliot LA. The conduct and neglect of self-studies in medical education. Eval Heal Prof. 1990;13:204–214.
30. Simpson DE, Golden DL, Rehm JM, Kochar MS, Simons KB. The costs versus the perceived benefits of an LCME institutional self-study. Acad Med. 1998;73:1009–1012.
31. Bezuidenhout MJ. Quality assurance in undergraduate medical education: A guide for accreditation reviews. South Afr J High Educ. 2007;21:427–440.
32. Al Mohaimeed A, Midhet F, Barrimah I, Saleh MN. Academic accreditation process: Experience of a medical college in Saudi Arabia. Int J Health Sci (Qassim). 2012;6:23–29.
33. Chandran L, Fleit HB, Shroyer AL. Academic medicine change management: The power of the Liaison Committee on Medical Education accreditation process. Acad Med. 2013;88:1225–1231.
34. Shroyer AL, Lu WH, Chandran L. Drivers of dashboard development (3-D): A curricular continuous quality improvement approach. Acad Med. 2016;91:517–521.
35. Kassebaum DG, Cutler ER, Eaglen RH. The influence of accreditation on educational change in U.S. medical schools. Acad Med. 1997;72:1127–1133.
36. Kassebaum DG, Eaglen RH, Cutler ER. The meaning and application of medical accreditation standards. Acad Med. 1997;72:808–818.
37. Grant J, Marshall J, Gary NE. Pilot evaluation of the World Federation for Medical Education’s global standards for basic medical education. Med Educ. 2005;39:245–246.
38. Yang EB. A study on the classification validity and predictive validity of accreditation standards for medical schools in Korea [in Korean]. Korean J Med Educ. 2002;14:213–226.
39. Cueto J Jr, Burch VC, Adnan NA, et al. Accreditation of undergraduate medical training programs: Practices in nine developing countries as compared with the United States. Educ Health (Abingdon). 2006;19:207–222.
40. Yang EB. Analysis on the performance and tasks of accreditation system for medical colleges [in Korean]. J Korean Med Assoc. 2008;51:586–592.
41. Yousefy A, Changiz T, Yamani N, Zahrai RH, Ehsanpour S. Developing a holistic accreditation system for medical universities of the Islamic Republic of Iran. East Mediterr Health J. 2009;15:747–756.
42. Sánchez ID, Riquelme AP, Moreno RB, Salas SI, Pertuzé JR, García PC. Description of assessment processes at the medical school of the Pontificia Universidad Católica de Chile: A ten-year experience [in Spanish]. Rev Med Chil. 2010;138:1055–1061.
43. Hunt D, Migdal M, Eaglen R, Barzansky B, Sabalis R. The unintended consequences of clarity: Reviewing the actions of the Liaison Committee on Medical Education before and after the reformatting of accreditation standards. Acad Med. 2012;87:560–566.
44. Hunt D, Migdal M, Waechter DM, Barzansky B, Sabalis RF. The variables that lead to severe action decisions by the Liaison Committee on Medical Education. Acad Med. 2016;91:87–93.
45. Tackett S, Grant J, Mmari K. Designing an evaluation framework for WFME basic standards for medical education. Med Teach. 2016;38:291–296.
46. Blouin D, Tekian A, Kamin C, Harris IB. The impact of accreditation on medical schools’ processes. Med Educ. 2018;52:182–191.
47. van Zanten M, Norcini JJ, Boulet JR, Simon F. Overview of accreditation of undergraduate medical education programmes worldwide. Med Educ. 2008;42:930–937.
48. van Zanten M, McKinley D, Durante Montiel I, Pijano CV. Medical education accreditation in Mexico and the Philippines: Impact on student outcomes. Med Educ. 2012;46:586–592.
49. van Zanten M, Boulet JR, Simon FA. Flexner’s global influence: Medical education accreditation in countries that train physicians who pursue residency in the United States. Acad Med. 2010;85:324–332.
50. Borgos JE. Using principal-agent theory as a framework for analysis in evaluating the multiple stakeholders involved in the accreditation and quality assurance of international medical branch campuses. Qual High Educ. 2013;19:173–190.
51. van Zanten M. The association between medical education accreditation and the examination performance of internationally educated physicians seeking certification in the United States. Perspect Med Educ. 2015;4:142–145.
52. Borges Oquendo LC, Rubio Olivares DY, Alberto Pichs García LA. Management of quality at the “General Calixto García” medical school [in Spanish]. Rev Cuba Educ Medica Super. 2016;30:580–590.
53. Villamor MA, Reyes ALT. Accreditation of medical education: Perspectives of stakeholders. Health Sci J. 2016;5:25–31.
54. Alrebish SA, Jolly BC, Molloy EK. Accreditation of medical schools in Saudi Arabia: A qualitative study. Med Teach. 2017;39(suppl 1):S1–S7.
55. Vázquez Martínez FD, Ortiz-León MC. Acreditación y resultados de la educación médica en México. Educ Méd. 2017;19:333–338.
56. van Zanten M, Boulet JR, Greaves I. The importance of medical education accreditation standards. Med Teach. 2012;34:136–145.
57. Galukande M, Opio K, Nakasujja N, et al. Accreditation in a sub Saharan medical school: A case study at Makerere University. BMC Med Educ. 2013;13:73.
58. Abdalla ME. Suggested new standards to measure social accountability of medical schools in the accreditation systems. J Case Stud Accredit Assess. 2014;3:1–25.
59. Abdalla ME. Social accountability of medical schools: Do accreditation standards help promote the concept? J Case Stud Accredit Assess. 2014;3:1–14.
60. Andrew SE, Oswald A, Stobart K. Bridging the continuum: Analysis of the alignment of undergraduate and postgraduate accreditation standards. Med Teach. 2014;36:804–811.
61. Hays R. The potential impact of the revision of the Basic World Federation Medical Education Standards. Med Teach. 2014;36:459–462.
62. Miskowicz-Retz KC, Williams A. New colleges of osteopathic medicine: Steps in achieving accreditation. J Am Osteopath Assoc. 2008;108:121–125.
63. Williams A, Miskowicz-Retz KC. Evolution of colleges of osteopathic medicine: A discussion of the COCA’s “substantive change” policies. J Am Osteopath Assoc. 2009;109:128–134.
64. Williams A, Miskowicz-Retz KC. Colleges of osteopathic medicine: The process of continuous evaluation. J Am Osteopath Assoc. 2010;110:144–148.
65. Ching LM, Burke WJ. Osteopathic distinctiveness in osteopathic predoctoral education and its effect on osteopathic graduate medical education. J Am Osteopath Assoc. 2011;111:581–584.
66. Getz R, Frait S. Osteopathic education: The link between mission and accreditation. Int J Osteopath Med. 2014;17:48–51.
67. Williams A, Miskowicz-Retz KC. New colleges of osteopathic medicine, branch campuses, and additional locations—What is the difference? J Am Osteopath Assoc. 2015;115:242–250.
68. Bury G. Accreditation of medical schools in Ireland: Standards and procedures. Med J Malaysia. 2005;60(suppl D):11–19.
69. Yoshioka T, Nara N. International accreditation of medical school towards quality assurance of medical education [in Japanese]. Rinsho Shinkeigaku. 2013;53:1136–1138.
70. Egan RL. Recent events of special interest to medical education. JAMA. 1985;254:1556–1564.
71. Shomaker TS. The role of accreditation in medical education. Hawaii Med J. 2001;60:211–212.
72. Wilson EA. Formative accreditation: Complying with Liaison Committee on Medical Education (LCME) standards. Plan High Educ. 2007;36:23–27.
73. Severyanova L, Lazarev A. Recognition of higher medical institutions in Russia. Med J Malaysia. 2005;60(suppl D):71–74.
74. Azila NM, Tan CP. Accreditation of medical schools: The question of purpose and outcomes. Med J Malaysia. 2005;60(suppl D):35–40.
75. Mahmud Mohd MN. The role of the Malaysian Medical Council in medical education. Med J Malaysia. 2005;60(suppl D):28–31.
76. Hillen HFP. Quality assurance of medical education in the Netherlands: Programme or systems accreditation? Tijdschr Med Onderwijs. 2010;29:73–79.
77. Schirlo C, Heusser R. Quality assurance of medical education: A case study from Switzerland. GMS Z Med Ausbild. 2010;29:80–86.
78. Lai CW. Experiences of accreditation of medical education in Taiwan. J Educ Eval Health Prof. 2009;6:2.
79. Kassebaum DG. Origin of the LCME, the AAMC-AMA partnership for accreditation. Acad Med. 1992;67:85–87.
80. Schwarz MR. Liaison Committee on Medical Education. Past successes, future challenges. JAMA. 1992;268:1091–1092.
81. Verheyden CN. Medical accreditation in the United States: What for? Plast Reconstr Surg. 2016;138:1367–1370.
82. Cordova JA, Aguirre E, Hernández A, et al. Assessment and accreditation of Mexican medical schools. Med Educ. 1996;30:319–321.
83. Rosselot E. Accreditation of medical schools. Purpose, development and projections of a process (corrected) [in Spanish]. Rev Méd Chil. 2001;129:935–943.
84. Berlin A. Tips for surviving the GMC Quality Assurance of Basic Medical Education Procedures. Clin Teach. 2006;3:233–237.
85. Middleton AL. LCME accreditation: Role of students. Hawaii Med J. 2009;68:169–170.
86. Maccarrick G, Kelly C, Conroy R. Preparing for an institutional self review using the WFME standards—An international medical school case study. Med Teach. 2010;32:e227–e232.
87. Sakai DH, Kasuya RT, Smerz RW, Ching N. Medical school hotline: Liaison Committee on Medical Education (LCME) accreditation and the John A. Burns school of medicine: What medical student teachers should know. Hawaii J Med Public Health. 2013;72:242–245.
88. Miller B, Dzwonek B, McGuffin A, Shapiro JI. From LCME probation to compliance: The Marshall University Joan C Edwards School of Medicine experience. Adv Med Educ Pract. 2014;5:377–382.
89. Sakai DH, Kasuya RT, Fong SF, et al. Medical school hotline: Liaison Committee on Medical Education accreditation: Part I: The accreditation process. Hawaii J Med Public Health. 2015;74:311–314.
90. Fong SF, Kramer K, Sakai DH, et al. Medical school hotline: Liaison Committee on Medical Education accreditation: Part II: The graduation objectives. Hawaii J Med Public Health. 2015;74:378–381.
91. Fong SF, Omori JS, Sakai DH, et al. Medical school hotline: Liaison Committee on Medical Education accreditation, part III: Educational program content, curriculum management, and student assessment. Hawaii J Med Public Health. 2015;74:412–414.
92. Fong SF, Sakai DH, Kasuya RT, et al. Medical school hotline: Liaison Committee on Medical Education accreditation, part IV: Pre-clerkship education. Hawaii J Med Public Health. 2016;75:48–51.
93. Hamilton JD, Vandewerdt JM. The accreditation of undergraduate medical education in Australia. Med J Aust. 1990;153:541–545.
94. Len KA, Riklon S, Anegawa LJ, et al. Medical school hotline: Liaison Committee on Medical Education accreditation, part V: Clerkship education. Hawaii J Med Public Health. 2016;75:78–81.
95. Smerz RW, Fong SF, Sakai DH. Medical school hotline: Liaison Committee for Medical Education accreditation, part VI: Academic, career, personal counseling/student health and well-being. Hawaii J Med Public Health. 2016;75:140–143.
96. Lee WK. Medical school hotline: Liaison Committee on Medical Education accreditation, part VII: Diversity/pipeline programs and partnerships. Hawaii J Med Public Health. 2016;75:200–202.
97. Sakai D, Kasuya R, Fong S, Asano I. Medical school hotline: Liaison Committee on Medical Education accreditation, part VIII: Faculty development. Hawaii J Med Public Health. 2016;75:278–280.
98. Blanchette PL, Takanishi DM Jr, Connolly KK. Medical school hotline: Liaison Committee on Medical Education accreditation, part IX: Strengths and challenges at the John A. Burns School of Medicine (JABSOM). Hawaii J Med Public Health. 2016;75:345–347.
99. Connolly KK, Blanchette PL, Hedges JR. Medical school hotline: John A. Burns School of Medicine receives continued full accreditation by the Liaison Committee on Medical Education (LCME) for the maximum accreditation period of eight years. Hawaii J Med Public Health. 2017;76:287–289.
100. Blumberg P. Multidimensional outcome considerations in assessing the efficacy of medical educational programs. Teach Learn Med. 2003;15:210–214.
101. McLaughlin SA, Hobgood C, Binder L, Manthey DE; SAEM Undergraduate Education Committee for 2004–2005. Impact of the Liaison Committee on Medical Education requirements for emergency medicine education at U.S. schools of medicine. Acad Emerg Med. 2005;12:1003–1009.
102. MacCarrick GR. A practical guide to using the World Federation for Medical Education (WFME) standards. WFME 1: Mission and objectives. Ir J Med Sci. 2010;179:483–487.
103. Maccarrick G. A practical guide to using the World Federation for Medical Education standards. WFME 2: Educational program. Ir J Med Sci. 2010;179:489–491.
104. Hamilton JD. Establishing standards and measurement methods for medical education. Acad Med. 1995;70(7 suppl):S51–S56.
105. MacCarrick GR. A practical guide to using the World Federation for Medical Education Standards. WFME 3: Assessment of students. Ir J Med Sci. 2011;180:315–317.
106. [No authors listed.] International standards in medical education: Assessment and accreditation of medical schools’—Educational programmes. A WFME position paper. The Executive Council, The World Federation for Medical Education. Med Educ. 1998;32:549–558.
107. Karle H; WFME task force on defining international standards in basic medical education. Report of the working party, Copenhagen, 14–16 October 1999. Med Educ. 2000;34:665–675.
108. van Niekerk JPD, Christensen L, Karle H, Lindgren S, Nystrup JJ. WFME global standards in medical education: Status and perspectives following the 2003 WFME World Conference. Med Educ. 2003;37:1050–1054.
109. van Niekerk JP. WFME global standards receive ringing endorsement. Med Educ. 2003;37:585–586.
110. Karle H. Global standards and accreditation in medical education: A view from the WFME. Acad Med. 2006;81(12 suppl):S43–S48.
111. Karle H. World Federation for Medical Education Policy on international recognition of medical schools’ programme. Ann Acad Med Singapore. 2008;37:1041–1043.
112. Karle H; Executive Council, World Federation for Medical Education. International recognition of basic medical education programmes. Med Educ. 2008;42:12–17.
113. Rezaeian M, Jalili Z, Nakhaee N, Shirazi JJ, Jafari A. Necessity of accreditation standards for quality assurance of medical basic sciences. Iran J Public Health. 2013;42(suppl 1):147–154.
114. Stevens DP. Three questions for the LCME. Acad Med. 2000;75:960–961.
115. Breen K, Frank I, Walters T. Australian Medical Council: A view from the inside. Intern Med J. 2001;31:243–248.
116. Barón Maldonado M. La razón de ser de la Acreditación de las Facultades de Medicina. Educ Méd. 2003;6:12–13.
117. Al Alwan I. Is accreditation a true reflection of quality? Med Educ. 2012;46:542–544.
118. Al-Shehri AM, Al-Alwan I. Accreditation and culture of quality in medical schools in Saudi Arabia. Med Teach. 2013;35(suppl 1):S8–S14.
119. Ahn E, Ahn D. Beyond accreditation: Excellence in medical education. Med Teach. 2014;36:84–85.
120. Barzansky B, Hunt D, Moineau G, et al. Continuous quality improvement in an accreditation system for undergraduate medical education: Benefits and challenges. Med Teach. 2015;37:1032–1038.
121. Blouin D, Tekian A. Accreditation of medical education programs: Moving from student outcomes to continuous quality improvement measures. Acad Med. 2018;93:377–383.
122. Schwörer B, Wissing F. Private medical education in Germany [in German]. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz. 2018;61:148–153.
123. Simpson I, Lockyer T, Walters T. Accreditation of medical training in Australia and New Zealand. Med J Malaysia. 2005;60(suppl D):20–23.
124. Field MJ. Medical school accreditation in Australia: Issues involved in assessing major changes and new programs. J Educ Eval Health Prof. 2011;8:6.
125. Geffen L, Cheng B, Field M, Zhao S, Walters T, Yang L. Medical school accreditation in China: A Sino-Australian collaboration. Med Teach. 2014;36:973–977.
126. Pumerantz P. The accreditation process for colleges of osteopathic medicine. J Am Osteopath Assoc. 1975;75:225–227.
127. Hosseini F, Einollahi B, Zand RH, Nazaran FS, Niaei AM, Avarzamani MN. A review of defining standards process in basic medical education accreditation in Mexico and WFME. Med Educ J Iran. 2002;1:147–149.
128. Goic A. Development of traditional careers at new universities. The case of medicine [in Spanish]. Rev Med Chil. 1995;123:99–107.
129. Rosselot E. Professional quality assurance. A new ethic frame for the practice of medicine [in Spanish]. Rev Med Chil. 1999;127:1375–1383.
130. Rosselot E, Norero C, Hanne C, Mateluna E. Professional formation and its accreditation in medicine. A paradigm to sustain public confidence [in Spanish]. Rev Med Chil. 2002;130:585–589.
131. van Zanten M, Parkins LM, Karle H, Hallock JA. Accreditation of undergraduate medical education in the Caribbean: Report on the Caribbean Accreditation Authority for Education in Medicine and Other Health Professions. Acad Med. 2009;84:771–775.
132. Wood DL, Hahn MB. Accreditation standards of osteopathic and allopathic medical schools: Could they affect educational quality? Acad Med. 2009;84:724–728.
133. Leinster S. Role of accrediting bodies in providing education leadership in medical education. J Health Spec. 2014;2:2–5.
134. Cane MA. Restrictive practices in accreditation of medical schools: An antitrust analysis. S Calif Law Rev. 1978;51:657–694.
135. Kassebaum DG. Accreditation, curricular change, and other disturbances of the steady state. Physiologist. 1994;37:56–59.
136. Kassebaum DG. Achieving better institutional self-study. Acad Med. 1998;73:925–927.
137. Pumerantz P. The self-study process. J Am Osteopath Assoc. 1975;75:227–230.
138. Barzansky B. Comparison of accreditation practices and standards of US schools of medicine and veterinary medicine. J Vet Med Educ. 2004;31:120–125.
139. Simon FA, Aschenbrener CA. Undergraduate medical education accreditation as a driver of lifelong learning. J Contin Educ Health Prof. 2005;25:157–161.
140. Tekian AS, Al Ahwal MS. Aligning the SaudiMED framework with the National Commission for Academic Accreditation and Assessment domains. Saudi Med J. 2015;36:1496–1497.
141. Hays R, Baravilala M. Applying global standards across national boundaries: Lessons learned from an Asia-Pacific example. Med Educ. 2004;38:582–584.
142. Lindgren S, Karle H. Social accountability of medical education: Aspects on global accreditation. Med Teach. 2011;33:667–672.
143. Sweet S. Accreditation at colleges of osteopathic medicine. J Am Osteopath Assoc. 2001;101:675–676.
144. Lee K, Whelan JS, Tannery NH, Kanter SL, Peters AS. 50 years of publication in the field of medical education. Med Teach. 2013;35:591–598.
145. Puthran R, Zhang MW, Tam WW, Ho RC. Prevalence of depression amongst medical students: A meta-analysis. Med Educ. 2016;50:456–468.
146. Colbert-Getz JM, Kim S, Goode VH, Shochet RB, Wright SM. Assessing medical students’ and residents’ perceptions of the learning environment: Exploring validity evidence for the interpretation of scores from existing tools. Acad Med. 2014;89:1687–1693.
147. ET Bureau. Niti Aayog Panel recommends replacing MCI with National Medical Commission. Economic Times.
http://economictimes.indiatimes.com/news/economy/policy/niti-aayog-panel-recommends-replacing-mci-with-national-medical-commission/articleshow/53407195.cms. Updated July 27, 2016. Accessed May 30, 2019.
148. Bilimoria KY, Chung JW, Hedges LV, et al. National cluster-randomized trial of duty-hour flexibility in surgical training. N Engl J Med. 2016;374:713–727.
149. Frieden TR. Evidence for health decision making—Beyond randomized, controlled trials. N Engl J Med. 2017;377:465–475.
150. van Zanten M, Boulet J. Developing a research agenda to support medical education accreditation. May 19–20, 2018; Poster presented at: The World Health Professions Regulation Conference 2018; Geneva, Switzerland.
151. Clancy CM, Glied SA, Lurie N. From research to health policy impact. Health Serv Res. 2012;47(1 pt 2):337–343.
152. Reed DA, Cook DA, Beckman TJ, Levine RB, Kern DE, Wright SM. Association between funding and quality of published medical education research. JAMA. 2007;298:1002–1009.
Reference cited in Figure 1 only
153. Tricco AC, Lillie E, Zarin W, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and explanation. Ann Intern Med. 2018;169:467–473.
Appendix 1 Summary of the 36 UME Accreditation Scholarship Articles From a Scoping Review (Through January 31, 2018)
