Of the many U.S./Canadian and international organizations that regulate medicine and medical education, I have particular respect for the Educational Commission for Foreign Medical Graduates (ECFMG; Philadelphia, Pennsylvania). My esteem for this organization is based on specific interorganizational collaborations in which I participated with the ECFMG over a decade ago and my recent participation in analyses of the impact of their assessment practices.
Founded in 1956, it is a relatively new organization compared with sibling bodies in the United States and Canada (e.g., the National Board of Medical Examiners [NBME] and the Medical Council of Canada [MCC]). The ECFMG quickly established procedures to validate foreign graduates’ existing medical credentials, and in 1958, in collaboration with the NBME, it successfully began administering a medical sciences examination and an English proficiency test. Following the merger with the Commission on Foreign Medical Graduates in 1974, the ECFMG assumed its roles of conducting research about international medical graduates (IMGs) and monitoring the visa sponsorship of exchange visitors in the United States. In 1986, the medical education community witnessed the ECFMG’s introduction of primary-source verification of IMGs’ medical credentials (i.e., confirming reported medical degrees or other qualifications with the original educational institution conferring the credential). Subsequently, with advice on design and piloting from my colleagues at the MCC, the ECFMG incorporated objective structured clinical examinations into its certification processes. Its most significant development to date has been the creation of the Foundation for the Advancement of International Medical Education and Research (FAIMER) in 2000. All of these innovations were guided by the ECFMG’s core values of collaboration and accountability. To illustrate this commitment to evidence and accountability, the FAIMER-based research staff have published a tracking study that confirmed the quality of the ECFMG’s certification assessment of IMGs based on these graduates’ subsequent performance in practice in the United States.1 Ensuing articles provided further evidence of the validity of the ECFMG’s certification strategies using successful IMGs’ practice outcomes. In sum, the ECFMG is akin to a regulator that guides IMGs and offers them a valid and reliable certificate of future capability.
Thus, I welcomed the opportunity to comment on Tackett’s timely review2 in this issue. He examines the potential consequences of the ECFMG board’s 2010 decision to change the standards and procedures for credentialing and certifying IMGs. Beginning in 2023, all IMGs applying for ECFMG certification must have graduated from a medical school that has been formally accredited by a recognized authority. Furthermore, that authority must follow standards and procedures that are comparable to those of the Liaison Committee on Medical Education or to the globally accepted criteria of the World Federation for Medical Education (WFME).
What triggered the ECFMG’s decision? Tackett describes the irony that by the start of the new millennium, the world was facing a major increase in the number of medical schools—as its population plateaued. The increase in number and the quality of these schools’ graduates concerned U.S. and Canadian workforce policymakers and regulatory authorities. In time, thousands of U.S. and Canadian students were admitted to the new schools, often in the Caribbean. By 2010, the ECFMG board had reason to reassess the situation and consider policy incentives to promote higher-quality medical education. Over the subsequent eight years, the ECFMG, its partner the WFME, and the international medical education community have encouraged a worldwide adoption of the accreditation requirement.
Data cited by Tackett suggest that the progress has been variable and has evolved more slowly than might have been expected. Tackett clarifies that significant adoption took place in countries where WFME and FAIMER guided and supported the process but that, regrettably, two-thirds (n = 6,700) of the ECFMG applicants in 2015 were educated in 119 countries that do not currently have a recognized accrediting authority. Their medical schools have been significant sources of IMGs migrating to the United States for years. Because IMGs constitute approximately one-quarter of the U.S. physician workforce, Tackett postulates that the change in ECFMG requirements could have a negative impact on the total number of IMGs applying to enter the United States. In turn, postgraduate medical education (PGME) programs and, ultimately, health institutions will face a smaller pool of qualified IMGs. Yet, Tackett notes, certain countries or regions may have little motivation to act on the directive. The loss of newly graduated physicians to the United States might not be a preferred outcome.
Tackett postulates three basic options for the ECFMG: (1) to go forward with the policy change, (2) to abandon it, or (3) to adjust or delay it. He gives little attention to other tools or strategies that the ECFMG or other regulatory bodies might consider in resolving this dilemma. Therefore, I will address two questions. To start, given the ECFMG’s historic role as a protector of the U.S. public, what policy or measurement options could the ECFMG consider in managing this situation? Secondly, given the complexity of medical workforce planning, what should be the ECFMG’s future role or contributions?
Management or Measurement Options for the ECFMG to Consider
The goal of the ECFMG’s 2010 announcement is to continue “efforts to enhance protection of the public.”3 Are other means of achieving this goal possible? Here, I propose three different options: (1) measurement-based policy strategies, (2) rational planning models since clearly defined goals exist, and (3) frameworks designed either to guide policymaking in poorly defined contexts or to meet broad social goals.
Measurement-based policy options: Content or statistical adjustments versus screening strategies
At the outset, alternatives could have included adding skills or competencies to the certification blueprint. They would have addressed known areas of weakness in certified IMGs’ subsequent clinical performances. That plan would have required a major redesign of parts of the blueprint for the United States Medical Licensing Examination with the NBME. Another solution might have been adding an adjustment of “one standard error of measurement” to the final passing score of the assessment procedures. That change would have prevented IMGs with borderline passes from squeaking through. In an era of competency-based and criterion-referenced examinations, however, this adjustment to the passing score was not advisable. Whatever the options, the ECFMG decided to add one more screen to the certification processes. The concept of screening for public protection must be thought of as a series of slices of Swiss cheese. No single slice is perfect; each has a set of holes. If there are several slices and if the holes are small and not similarly aligned, then the missing competency or skill of concern should be caught in the series of well-planned certification slices. The added slice was formal accreditation of international medical schools.
Defined goals with rational linear models
Approaches designed to improve the monitoring and management of a situation are available for any organization that has a clear conception of the desired outcomes and impacts. One such approach is the logic model. The logic model is linear and forces the organization to preidentify the problem, the goals (along with written objectives), the required activities (inputs and resources), the methods, the outputs, and—in due course—the desired outcomes and impacts. The key is that these elements are definable and concrete and, hence, measurable. Various formats for the logic model are available to the ECFMG depending on its priorities and concerns regarding the current results.
Guiding models for complex situations or social programs with broad goals
Before 2023, the ECFMG has to ask, Have we adequately solved the precipitating problem? If not solved to its satisfaction, what are its current options? In the event that the original decision was intended to deal with a problem that is not well defined and for which available tools are less certain, the Stacey matrix can serve as guide.4 In simple terms, the Stacey matrix permits problem solvers or policymakers to plot a decision-making situation on two axes: degree of agreement on the what and degree of certainty on the how. To illustrate, if both the requirements (the what) and the needed technology (the how) are well known, then ordinary linear management is sufficient. However, if neither the requirements nor the technology solutions are known or clear, then situations are complicated and extraordinary management skills and analyses are needed. If the ECFMG felt a high degree of certainty about the problem (e.g., poor performance of IMGs and U.S. IMGs from nonaccredited schools), but less certainty about how to reach the preferred outcome, it would have had to make a judgment call. If that was the case, the ECFMG’s decision is comparable to a vision with no detailed plan against which to monitor success. The point is that when the degrees of agreement and certainty are not high, imperfect decisions are made and judgments or politics drive the day.
If the organization is hesitant about the underlying causes of IMG shortcomings and remains uncertain about feasible solutions, innovation is an option. However, to inform that decision, another framework can be helpful: contribution analysis (CA).5 CA involves surveying and interviewing multiple stakeholders. CA need not be adopted in full scale, but modest versions could provide information about the potential effects of the 2010 policy on the many layers of stakeholders operating in the IMG and PGME environments. The information available from CA inquiries could offer the ECFMG insights for internal policymaking. It might also inform the ECFMG’s discussions with the physician workforce network about side effects from the 2010 decision. To that end, discussions and debate with key stakeholders and experts may yield a breakthrough pathway to a better how.
Looking Ahead: Integrating Domestic and International Strategies
The ECFMG has had many successes: mergers, collaborations with other assessment authorities, the introduction of innovative practices based on evidence, and, most recently, improving international educational practices through FAIMER. These are strong signals for its many stakeholders and for academia. The organization has had far-reaching effects in health care in the United States and, lately, on educational practices across the world. The recent integration of the FAIMER board and the ECFMG Board of Trustees tells stakeholders that the ECFMG’s basic policies for domestic and international operations will be preserved. The ECFMG will continue to protect the public through its certification practices, to offer quality support to IMGs coming to the United States, and to improve (through FAIMER) educational practices around the world. Importantly, FAIMER researchers will continue to publish quality outcomes studies and important tracking reports on IMG migration patterns. However, observers should expect the joint ECFMG/FAIMER board to review and consider updating its policies. That review should include an assessment of the impact of the 2010 decision and a careful consideration of all international programs, especially in light of the changing directives around immigration policies in the United States. Because the ECFMG is supported by a strong research staff, stakeholders and academia should anticipate evidenced-based analyses and policy decisions.
Meanwhile, to address Tackett’s concerns, the ECFMG must continue to monitor and analyze the flows of IMGs into the United States, based on the pre-2023 accreditation policy. While workforce modeling is not the ECFMG’s job, the commission is a key subsystem linked with other stakeholders, analysts, and administrators, which are, in turn, nested within a network of professional organizations and the health care industry. In keeping with its record of collaboration and by making IMG datasets readily available, the ECFMG can guide policymakers seeking better workforce solutions for the United States.
Building on Tackett’s concerns, I have identified management strategies that are currently available to the ECFMG and all authorities. All signs indicate that the current ECFMG leadership is outcomes oriented. Extensive interviewing of all stakeholders would offer major insights for future policies. Such insights would allow the ECFMG to contribute data to inform workforce priorities and guide the PGME community, but ECFMG’s primary responsibility remains to assess each IMG’s suitability for future training and practice opportunities.