Secondary Logo

Journal Logo

Medical Education in the Caribbean: Variability in Educational Commission for Foreign Medical Graduate Certification Rates and United States Medical Licensing Examination Attempts

van Zanten, Marta; Boulet, John R.

Section Editor(s): Woods, Majka PhD; Bowen, Judy MD

doi: 10.1097/ACM.0b013e3181b38d1c
Predictive Validity
Free

Background The purpose of this study was to investigate demographic characteristics and performance outcomes of Caribbean-educated physicians.

Method The study included almost 20,000 Caribbean-educated students/graduates who registered for an exam leading to Educational Commission for Foreign Medical Graduates (ECFMG) certification during the study period. We summarized United States Medical Licensing Examination attempts and ECFMG certification rates (1996 to January 21, 2009), by country of medical school.

Results The proportions of females and non-U.S. citizens attending medical schools in the Caribbean are increasing. Average exam attempts for certified individuals ranged, by country of medical school, from 1.19 to 2.84 for Step 1, from 1.20 to 2.13 for Step 2 Clinical Knowledge, and from 1.01 to 1.42 for Step 2 Clinical Skills/ECFMG Clinical Skills Assessment. Approximately 14,000 (74.2%) individuals achieved certification, and success rates ranged, by country, from 19.1% to 91.5%.

Conclusions There is significant variability in the average number of exams taken and ECFMG certification rates of Caribbean-educated physicians.

Correspondence: Marta van Zanten, FAIMER, 3624 Market Street, Philadelphia, PA 19104; e-mail: (mvanzanten@faimer.org).

Graduates of international medical schools (IMGs) currently constitute approximately 25% of all physicians in residency training and in practice in the United States.1 IMGs fill an important role in the health care system in the United States, as they are more likely than graduates of U.S. medical schools (USMGs) to practice in underserved communities and specialize in primary care disciplines.2 The ECFMG certification process ensures the readiness of IMGs to enter Accreditation Council for Graduate Medical Education-accredited training programs in the United States.

Whereas IMGs are educated at institutions around the world, a large number study at medical schools located in the Caribbean. Although some medical schools in the region, such as those located in Cuba, Jamaica, and Haiti, focus on educating domestic students who remain in their countries to practice, numerous other Caribbean schools recruit, almost exclusively, international students who intend to pursue residency training and ultimate licensure in the United States.

The increasing number of Caribbean physicians seeking residency and practice opportunities in the United States, the large number of schools in the region, many of which have been recently established, and the lack of a uniform system of quality assurance oversight have led to concerns regarding the quality of the education provided at these institutions.3 Numerous studies during the past 30 years have documented that, on average, graduates of Caribbean schools seeking to enter graduate training programs in the United States do not perform as well as USMGs or graduates of international schools located outside of the Caribbean on qualifying exams and other measures of ability.4–6 In part because of these concerns, medical education in the Caribbean is often viewed as a uniform educational entity, despite evidence of wide variability of medical school curricula, selection processes, and performance of students. In our recent study of almost 30,000 students who registered for an exam leading to ECFMG certification between 1993 and 2007, first-time United States Medical Licensing Examination (USMLE) pass rates ranged, by country of medical school, from 19.4% to 84.4% for Step 1 Basic Science, from 26.3% to 79.7% for Step 2 Clinical Knowledge (CK), and from 60.6% to 97.2% for Step 2 Clinical Skills (CS) or the previous requirement, the Clinical Skills Assessment (CSA) (offered from 1998 to 2004).7

The purpose of this current study was to follow up on previous investigations documenting variability of student performance by including additional outcome measures. To better understand profiles of internationally trained physicians entering the U.S. workforce, demographic trends of the student population in the Caribbean were studied. In our previous study,7 we described Caribbean medical school characteristics and examined average first-attempt pass rates on the USMLE by country of medical school. In the current study, for those individuals who achieved ECFMG certification, we analyzed average number of attempts required to obtain a passing score on the USMLE Step 1, Step 2 CK, and Step 2 CS/CSA. Finally, ultimate ECFMG certification rates were analyzed by country of medical school. Unlike individual performance on specific exams, certification rate data provide a rough marker of the quality of the medical education programs in these countries, at least with respect to preparing individuals for entry into U.S. graduate medical education programs. Current ECFMG policy allows for the reporting of exam performance and certification rate data at the country level.

Back to Top | Article Outline

Method

There are currently 55 open medical schools located in the Caribbean listed in the International Medical Education Directory (IMED) (https://imed.faimer.org). A medical school is listed in IMED after the Foundation for Advancement of International Medical Education and Research receives confirmation from the ministry of health or other appropriate agency in the country where the medical school is located that the medical school is recognized by the ministry or other suitable agency. In addition to currently operating schools, IMED also contains historical records, including schools that have closed or moved to other locations.

Among other requirements, an individual must pass USMLE Step 1, Step 2 CK, and Step 2 CS/CSA to be eligible for ECFMG certification. We analyzed a cohort of Caribbean-trained medical students and graduates who first registered for an exam leading to ECFMG certification between 1996 and 2005, inclusive. To better understand demographic trends across the study period, separate analyses were done in five-year blocks (1996–2000 and 2001–2005). We then followed this cohort of individuals through until January 21, 2009. For those individuals who achieved ECFMG certification, the average number of attempts to pass the required exams, summarized over the study period by country of medical school, is provided. December 31, 2005, was used as the ending date for initial registration for an exam, and January 21, 2009, was used as the ending date for achieving certification. This strategy allowed those individuals who registered for their first exam between 1996 and 2005 adequate time to complete their degree, fulfill all subsequent exam requirements, and achieve certification. This study was exempt from IRB review. Participants have acknowledged that their data would be used for research purposes, personal identifying information has been removed from examinee records, and only group-level results are reported.

Back to Top | Article Outline

Results

Student demographics and medical schools

Between 1996 and 2005, 19,436 individuals who attended, or graduated from, medical schools in the Caribbean entered the ECFMG database by registering for an exam leading to ECFMG certification. Overall, approximately one third of Caribbean-educated registrants were non-U.S. citizens (non-USIMGs), 37.2% were female, and the mean age at initial exam registration was 29.5 years. Among registrants, 29.5% self-reported their ethnicity as white, 28.3% as other or no reply, 19.3% as Asian, 14.2% as Hispanic/Latino, 8.0% as black/African American, and 0.7% as Native American/Hawaiian Islander.

Across the study period, the number of Caribbean-educated registrants has been increasing. In 1996, there were 1,771 registrants; in 2005, there were 2,801 registrants. This growth has occurred for both U.S. citizens (USIMGs) and non-USIMG groups. The proportion of females has risen over the years from 33.4% in 1996–2000 to 40.0% in 2001–2005. The proportion of those registrants self-reporting a native language other than English has decreased over the study period. In the first five-year block, 43.0% indicated that English was not their native language, and this percentage decreased to 33.7% in the second five-year block. The average age of registrants has remained relatively constant.

The Caribbean countries with the highest numbers of registrants were Dominica (n = 4,645; 24%), Grenada (n = 4,227; 22%), and the Netherlands Antilles (n = 3,599; 19%). Across the two five-year periods (1996–2000 and 2001–2005), we observed an increase in registrants from schools located in every country except Cuba, Saint Lucia, and Jamaica.

Back to Top | Article Outline

Exam performance

Exam performance, described as the average number of attempts made by students/graduates to pass the USMLE Steps 1, 2 CK, and 2 CS/CSA, by country of medical school, are presented in Table 1. For all graduates who eventually achieved certification, the average number of attempts for Step 1 was 1.61 (minimum 1, maximum 13), Step 2 CK 1.42 (minimum 1, maximum 14), and Step 2 CS/CSA 1.11 (minimum 1, maximum 5). Average exam attempts for certified graduates ranged, by country of medical school, from 1.19 to 2.84 for Step 1, from 1.20 to 2.13 for Step 2 CK, and from 1.01 to 1.42 for Step 2 CS/CSA. Based on country-level mean attempts (n = 16), the SDs for Step 1, Step 2 CK, and Step 2 CS/CSA were .5, .3, and .2, respectively.

Table 1

Table 1

Back to Top | Article Outline

Certification rates

Internationally educated physicians are required to achieve ECFMG certification before securing residency training positions in the United States. Because many students at Caribbean schools take Step 1 after completing basic science instruction and take Steps 2 CK and CS (or, previously, CSA) closer to graduation, we defined the study cohort as individuals who first registered for an exam between 1996 and 2005, and we allowed these individuals an additional number of years (up until January 21, 2009) to complete the subsequent exam requirements and achieve certification. Across the Caribbean, and over the entire study period, 74.2% of individuals who registered for an exam leading to ECFMG certification between 1996 and 2005 eventually achieved certification. Overall, 80.0% of USIMGs and 62.5% of non-USIMGs who attended schools in the Caribbean achieved certification. For the study cohort, the proportion of non-USIMGs achieving certification has increased. Of the cohort of non-USIMGs who first registered for an exam between 1996 and 2000, 57.8% achieved certification, whereas 66.5% of non-USIMG registrants from 2001 to 2005 achieved certification. For USIMGs who first registered for an exam in the five-year block 1996–2000, 83.7% were certified, compared with 77.3% who registered for their first exam in the second time period (2001–2005). Table 1 provides ECFMG certification rates across the study period. These rates range, by country of medical school, from 19.1% to 91.5%.

Back to Top | Article Outline

Discussion

The purpose of this research was to follow up on our previous study7 that described variability of medical education in Caribbean medical schools and the performance (first-attempt pass rates on USMLE) of their students/graduates. Given the growing role of Caribbean-trained doctors in the United States, and their importance in providing primary care,2 it is important to examine the characteristics of these individuals and their performance on qualifying examinations. Depending on the quality of education provided, and other individual factors, certification exam attempt data provide an additional measure by which to examine and quantify variability of student performance and the quality of medical education programs in the region. Ultimate certification rates, although influenced by many factors, some unrelated to ability, can also provide a marker for quality.

The number of students attending schools in the Caribbean and registering for exams leading to ECFMG certification has increased over the study period. Although there have been a small number of school closures in the region, there has also been a large increase in the number of schools opening. Eighteen Caribbean schools listed in IMED opened in the year 2000 or later. This expansion represents 33% growth, during an eight-year period, in the number of recognized and operating schools in the region. In addition to the increase in the number of schools, the escalation in ECFMG exam registrants over the study period may also be attributable to an increase in class size at some of the established medical schools, and a recently imposed requirement, in some programs, of passage of USMLE Step 1 for progression and/or graduation.

A number of notable shifts have occurred in the student demographics at Caribbean schools. While the overall number of medical students/graduates from the Caribbean registering for an exam leading to ECFMG certification has been on the rise for both the USIMG and non-USIMG cohorts, there has been a relative increase in the proportion of non-USIMGs registrants and, within this cohort, an increased likelihood that an individual from this group will achieve ECFMG certification. Possible reasons for this shift include the greater international exposure of Caribbean schools, especially outside the United States, and their ability to offer non-U.S. citizens opportunities to complete clinical clerkships in the United States. Whereas the overall number of females in the study sample is less than the number of males, the proportion of females has increased during the study period. This shift in gender distribution parallels trends in the U.S. medical school population.8

Consistent with our previous study,7 wide variability in performance, as measured by average number of exam attempts and ultimate ECFMG certification, was found by country of medical school. For exam attempts, the variability is greatest for Step 1 and more uniform for Steps 2 CK and CS/CSA. Students/graduates attending medical schools in countries with schools that typically enroll large numbers of students who register for exams leading to ECFMG certification (e.g., Dominica, Grenada, the Netherlands Antilles) tend to require fewer attempts on USMLE to achieve a passing score. Students from schools located in Barbados, Jamaica, and Trinidad and Tobago also require fewer exam attempts, on average. Although these results are only based on those students/graduates who achieve ECFMG certification, they may reflect student selection practices, core educational instruction, and test preparation activities. It should be noted, however, that according to comparisons with USMGs, Caribbean medical students/graduates, on average, perform less well on USMLE.5

The variability of ultimate ECFMG certification rates is likely influenced by a number of factors, including differences in students’ financial resources and/or access to sufficient loans throughout their education, motivation to enter training programs in the United States, and varying levels of attrition from medical school or the ECFMG certification process after an unsuccessful exam attempt. Similar to the results of average exam attempts, students/graduates from schools that enroll large numbers of students registering for USMLE tend to have higher certification rates compared with schools with lower numbers of registrants. Despite these external factors, the extreme variability in ultimate certification rates also suggests that the quality of medical education programs in the Caribbean differs across the region.

Although differences in exam attempts and ultimate ECFMG certification rates are substantial, these performance results should be interpreted with caution. Most Caribbean countries have more than one school, with varying numbers of students per school and with potential differences in admission requirements, qualifications and quality of teaching staff, and resources available. Country-level data for USMLE exam attempts and ultimate ECFMG certification rates are presented for the entire study period block, obscuring any potential changes over time. Also, because some students/graduates taking the USMLE may intend to practice in a country other than the United States, their motivation to complete the ECFMG certification process may not be strong. To minimize this potential bias, we only analyzed exam attempts for students who ultimately achieved ECFMG certification. Additionally, the student attrition rates at these schools, both prior to registration for an exam and during the certification process, are unknown. Finally, registrants from schools educating primarily domestic students are a self-selected group that may differ, both in demographic characteristics and ability, from the country’s total student population.

The number of individuals studying medicine in the Caribbean and seeking training positions and licensure in the United States has been growing over the past decade and is expected to continue to increase. Given this group’s role in providing primary care in the United States, it is important to study their characteristics, performance, and likelihood of meeting the requirements to be eligible for residency training. Results of this study reveal some interesting demographic trends and baseline data indicating wide variability in exam performance (average number of attempts) and ultimate ECFMG certification rates by country of medical school. Further research is necessary to determine how the basic science education at individual medical schools and clinical training experiences, which can often vary widely across and within medical schools, can impact student success. Moreover, the influence of newly developing and ongoing national and regional accreditation practices should be explored.9 These investigations are necessary to determine the relationship between medical education and performance outcomes and the ultimate impact on the health of populations served by these physicians.

Back to Top | Article Outline

References

1 Brotherton SE, Etzel SI. Graduate medical education, 2007–2008. JAMA. 2008;300:1228–1243.
2 Hing E, Lin S. Role of International Medical Graduates Providing Office-based Medical Care: United States, 2005–2006. Hyattsville, Md: National Center for Health Statistics; 2009. National Center for Health Statistics Data Brief 13.
3 Johnson DA. Prospects for a national clearinghouse on international medical schools. J Med Licensure Discipline. 2008;94:7–11.
4 Weinberg E, Bell AI. Performance of United States citizens with foreign medical education on standardized medical examinations. N Engl J Med. 1978;299:858–862.
5 Boulet JR, Cooper RA, Seeling SS, Norcini JJ, McKinley DW. U.S. citizens who obtain their medical degrees abroad: an overview, 1992–2006. Health Aff (Millwood). 2009;28:226–233.
6 Norcini JJ, Anderson MB, McKinley DW. The medical education of United States citizens who train abroad. Surgery. 2006;140:338–346.
7 van Zanten M, Boulet JR. Medical education in the Caribbean: Variability in medical school programs and performance of students. Acad Med. 2008;83:S33–S36.
8 Barzansky B, Etzel SI. Medical schools in the United States, 2007–2008. JAMA. 2008;300:1221–1227.
9 van Zanten M, Boulet JR. Accreditation of undergraduate medical education in the Caribbean: Report on the Caribbean Accreditation Authority for Education in Medicine and Other Health Professions (CAAM-HP). Acad Med. 2009;84:771–775.
© 2009 Association of American Medical Colleges