Skip Navigation LinksHome > February 2011 - Volume 86 - Issue 2 > Medical Education in the Caribbean: A Longitudinal Study of...
Academic Medicine:
doi: 10.1097/ACM.0b013e3182045efe
Examination Performance

Medical Education in the Caribbean: A Longitudinal Study of United States Medical Licensing Examination Performance, 2000–2009

van Zanten, Marta; Boulet, John R. PhD

Free Access
Article Outline
Collapse Box

Author Information

Ms. van Zanten is research associate, Foundation for Advancement of International Medical Education and Research (FAIMER), Philadelphia, Pennsylvania.

Dr. Boulet is associate vice president for research and data resources, FAIMER, Philadelphia, Pennsylvania.

Correspondence should be addressed to Ms. van Zanten, FAIMER, 3624 Market Street, Philadelphia, PA 19104; telephone: (215) 823-2226; fax: (215) 386-3309; e-mail: mvanzanten@faimer.org.

First published online December 16, 2010

Collapse Box

Abstract

Purpose: Caribbean-educated physicians play an important role in the United States. The purpose of this study was to investigate medical school development in the Caribbean, concentrating on performance trends of Caribbean-educated physicians over the past decade.

Method: The study included 26,495 Caribbean-educated students/graduates who took one or more exams leading to Educational Commission for Foreign Medical Graduates (ECFMG) certification between January 1, 2000, and December 31, 2009. The authors describe medical school development in the region, discuss recent student demographic trends, and summarize first-attempt success rates, by year and country of medical school, on the United States Medical Licensing Examination (USMLE) Step 1 (Basic Science), Step 2 (Clinical Knowledge [CK]) and Step 2 (Clinical Skills), or the previously required ECFMG Clinical Skills Assessment.

Results: Sixty-one medical schools operated in the Caribbean from 2000 to 2009, of which 56 are currently open, and approximately 10 new schools are in development. The number of Caribbean-educated ECFMG registrants has grown every year, from 1,510 in 2000 to 4,000 in 2009. USMLE Step 1 and Step 2 CK first-attempt pass rates generally increased over the study period. Performance trends by country varied widely.

Conclusions: Medical education programs in the Caribbean and the number of Caribbean-educated ECFMG registrants continue to expand, with students/graduates showing performance improvement in some areas. Performance findings should be interpreted with caution, because of limitations the authors explain.

Trends in U.S. demographics and potential reform of the health care system are expected to increase the demand for physician services, mainly in primary care fields.1 Graduates of international medical schools (IMGs) currently constitute approximately 25% of all physicians in residency training and in practice in the United States.2 IMGs play an important role in the health care system in the United States because they are more likely than graduates of U.S. medical schools (USMGs) to practice in underserved communities and specialize in primary care disciplines.3 The Educational Commission for Foreign Medical Graduates (ECFMG) certification process ensures the readiness of IMGs to enter U.S. graduate training programs approved by the Accreditation Council for Graduate Medical Education; successful completion of the ECFMG process is a prerequisite to IMGs' subsequent licensure and unsupervised practice.

Although IMGs are educated in countries all over the world, many attend schools located in the Caribbean region. Of the 10,406 physicians certified by the ECFMG in 2009, 2,639 (25.4%) graduated from institutions located in the Caribbean. Although some medical schools in the region focus on educating domestic students who remain in their countries to practice, numerous other Caribbean schools recruit, almost exclusively, international students who intend to pursue residency training and ultimate licensure in the United States.

The increasing number of Caribbean-educated physicians seeking residency and practice opportunities in the United States, the large number of recently established schools in the region, and the lack of a uniform system of quality assurance oversight have led to concerns regarding the quality of the education provided at these institutions.4,5 Studies from over 30 years ago,6,7 in addition to more recent investigations,8,9 have documented that, on average, graduates of Caribbean schools seeking to enter graduate training programs in the United States do not perform as well as USMGs or graduates of international schools located outside of the Caribbean on qualifying exams and other measures of ability and skill. Though medical education in the Caribbean is often viewed as a uniform educational entity, there is evidence of wide variability in resources, curricula, selection processes, and performance of students. In our 2008 study10 of almost 30,000 Caribbean-educated students/graduates who began the ECFMG certification process between 1993 and 2007, first- time United States Medical Licensing Examination (USMLE) pass rates ranged, by country of medical school,

* from 19.4% to 84.4% for Step 1 Basic Science,

* from 26.3% to 79.7% for Step 2 Clinical Knowledge (CK), and

* from 60.6% to 97.2% for Step 2 Clinical Skills (CS) or the previously required ECFMG Clinical Skills Assessment (CSA).

In our 2009 study11 of almost 20,000 Caribbean-educated students/graduates who eventually achieved ECFMG certification, average exam attempts ranged, by country of medical school,

* from 1.19 to 2.84 for Step 1,

* from 1.20 to 2.13 for Step 2 CK, and

* from 1.01 to 1.42 for Step 2 CS/CSA.

Approximately three-quarters of students/graduates who took at least one USMLE exam ultimately achieved ECFMG certification. Certification rates by country of medical education ranged from 19.1% to 91.5%.11 The wide range of performance across Caribbean countries suggests that the quality of educational programs in the region is not equivalent.

Our present study, reported here, expands on previous investigations by examining trends in exam performance by year (2000–2009). Our earlier work grouped data across years, obscuring shifts in performance over time. Because of the dynamic nature of medical education in the Caribbean, such as the opening of new schools, changes in existing schools' admission standards, resources, or curricula, and implementation of new exam eligibility policies, it is important to examine trends in performance data across years. This strategy more accurately depicts current performance of students/graduates and allows for more precise monitoring of trends in the examination performance of a large number of physicians who typically seek entry to U.S. graduate training programs. In addition to changes in exam performance data, we document the sharp increase in the number of medical schools and growth in the number of ECFMG registrants from existing schools over the decade we studied.

Back to Top | Article Outline

Method

Among other requirements, an IMG must pass USMLE Step 1, Step 2 CK and Step 2 CS, or the previously required CSA, to be eligible for ECFMG certification. We analyzed exam performance of a cohort of 26,495 Caribbean medical students/graduates who first registered with the ECFMG between January 1, 2000 and December 31, 2009. Data on trends in the number of medical schools in the region were gathered. We also sought demographic data describing students/graduates, again using ECFMG registration information. Finally, to more accurately portray changes in performance across time, USMLE first-attempt pass rates were calculated for each year from 2000–2009, by country of medical school. (For the purposes of this analysis, Step 2 CS data were combined with ECFMG CSA data; these data are referred to as Step 2 CS/CSA. The CSA was offered from 1998 to 2004; Step 2 CS has been administered since 2004. The two exams are very similar.)

This study was exempt from IRB review. ECFMG registrants have acknowledged that their data would be used for research purposes. Personal identifying information has been removed, and only group-level results are reported. Current ECFMG policy allows for the reporting of aggregate exam performance at the country level.

Back to Top | Article Outline

Results

Medical schools

As of June 1, 2010, there were 56 recognized and operating medical schools located in the Caribbean listed in the International Medical Education Directory (IMED) (https://imed.faimer.org). Over the past decade, an additional five IMED-listed Caribbean medical schools have closed. Of the total of 61 schools, 22 (36.1%) started instruction in the year 2000 or later. Over the past three years, IMED staff have received requests from five new Caribbean schools to be included in the directory. As of May 2010, these pending applications had not met all IMED requirements. Our recent search of media reports produced information on an additional six Caribbean medical schools in development that have not yet applied for IMED listing. (A medical school is listed in IMED after the Foundation for Advancement of International Medical Education and Research receives confirmation from the ministry of health, or other appropriate agency in the country where the medical school is located, that the medical school is recognized by the agency. IMED data are current as of June 1, 2010.)

Back to Top | Article Outline
Student demographics

Between January 1, 2000 and December 31, 2009, 26,495 Caribbean-educated students/graduates registered for one or more exams leading to ECFMG certification. Summarized over the decade, 42% (n = 11,145) were women, and the mean age of all the students/graduates at the time of first exam registration was 28.9. English was the native language of 69% (n = 18,235) of the Caribbean-educated registrants, and 90% (n = 23,838) attended a medical school where English language instruction is provided.

As the number of medical schools in the Caribbean increased, the number of students/graduates registering for an exam leading to ECFMG certification also grew in each year of the study period. The number of ECFMG registrants from Caribbean medical schools has grown every year during the past decade, an overall expansion of 265% between 2000 and 2009. Figure 1 displays the number of registrants each year, from 2000 (1,510 registrants) to 2009 (4,000 registrants).

Figure 1
Figure 1
Image Tools
Back to Top | Article Outline
Exam performance

For all Caribbean-educated students/graduates who took Step 1 during the study period, yearly first-attempt pass rates ranged from 55.5% in the year 2002 to 69.2% in the year 2009. Step 2 CK first-attempt pass rates ranged from 67.3% (2004) to 77.8% (2001), and Step 2 CS/CSA ranged from 81.3% (2007) to 89.0% (2005). Figure 2 displays the average USMLE first-attempt pass rates for Caribbean-educated students/graduates by year. U.S. student/graduate performance is also provided for comparison. An inspection of Figure 2 shows that Caribbean performance on Step 1 and Step 2 CK shows a decrease during the first half of the decade, followed by a gradual increase. For Step 2 CS/CSA, Caribbean performance was relatively steady between 2000 and 2005, followed by a decreasing trend until 2008 and a slight increase in 2009.

Figure 2
Figure 2
Image Tools

USMLE performance by country of medical school, presented as first-attempt pass rates by year (2000–2009), is provided in Appendix 1. There are a variety of patterns observed for Step 1 performance over the past decade. In some countries, such as Antigua and Barbuda, Cuba, and Saint Lucia, there has been a relatively steady increase in first-attempt pass rates. In other countries, such as the Cayman Islands, Dominica, and the Netherlands Antilles,* with some exceptions in individual years, the general trend has been a steady increase in performance from 2000 to 2008, and a slight decrease in 2009.

The pattern for Step 2 CK performance by country also shows some interesting trends. Antigua and Barbuda showed a decrease in the first half of the decade, followed by a sharp increase every year since 2005. Performance by students/graduates from Cuba, Saint Kitts and Nevis, and Trinidad and Tobago has generally been on the rise across the decade. Countries such as Aruba, Barbados, Dominica, Dominican Republic, Grenada, Jamaica, and Saint Lucia all showed an increase in the year 2009.

Although there were a variety of trends in Step 2 CS/CSA performance by country across the decade, countries that educate large numbers of students who take the USMLE (i.e., Dominica and Grenada) showed relatively constant (i.e., unchanging) performance over the study period.

Back to Top | Article Outline

Discussion

The purpose of our research was to follow up on previous studies10,11 that described variability of medical education in Caribbean medical schools, student populations, performance of students/graduates, and ultimate ECFMG certification rates. Such information is important because, as mentioned earlier, there is an anticipated need for more physicians in the United States in the coming decades, which means that Caribbean-educated doctors are expected to continue to play an important role in U.S. health care.

The development of new schools in the Caribbean shows no signs of abating. Although a small number of schools in the region have closed within the past decade, there has also been a large increase in the number of new schools commencing instruction, and approximately 10 additional schools are in development. Although it is unknown whether the plans for all of these new schools will ultimately come to fruition, it is clear that the expected demand for physicians may continue to present opportunities for the establishment and expansion of educational programs in the Caribbean. However, depending on the growth of U.S. medical school programs and limits on the number of residency positions, the need for additional international medical education programs, at least for supplying physicians to the United States, is not certain.

As our findings indicate, the number of ECFMG registrants from Caribbean medical schools has grown steadily over the last decade. This increase is likely due to a number of factors, including several new schools opening in the region and an enlargement in class size at some institutions.

Despite this trend, various dynamics may influence future growth in the Caribbean. For example, the recent increase in class size at many U.S. medical schools, together with the overall growth in the number of U.S. MD-granting and DO-granting institutions, may provide opportunities for potential Caribbean students to study domestically. According to a recent investigation of U.S. citizen applicants to U.S. MD-granting, U.S. DO-granting, and foreign institutions, 62% of the applicants to foreign schools (mostly located in the Caribbean) had also applied to a U.S. school.12 In addition, many schools in the Caribbean do not have access to U.S. federal loan programs. Because of the current difficulty for U.S. citizens to secure private loans, many schools are almost entirely reliant on students who self-finance their education, students who obtain loans via enrollment in concurrent online MBA or other graduate programs, or non-U.S. citizens (e.g., Canadians) who are able to obtain educational loans. However, there has been recent discussion within the U.S. Department of Education (DOE) aimed at modifying the eligibility criteria for international schools qualifying for federal student loans.13

Because little is known about the federal return on investment of IMGs borrowing federal money and attending foreign medical schools that are currently eligible for these loans, Congress recently mandated that the Government Accountability Office (GAO) study the performance of IMGs educated at participating schools and other aspects of foreign medical education. In June 2010 the GAO released the Report to Congressional Committees entitled Foreign Medical Schools: Education Should Improve Monitoring of Schools That Participate in the Federal Student Loan Program.14 In the report, the GAO makes several recommendations to the DOE, including (1) collecting and disseminating enhanced consumer information, such as aggregate student debt levels and graduation rates, (2) requiring participating foreign medical schools to submit aggregate institutional USMLE pass rate data to the DOE and the verification of these data, and (3) modifying the USMLE pass rate threshold.

Although the expansion of U.S. institutions and student loan issues will mainly affect U.S. citizens studying in the Caribbean, increasing numbers of non-U.S. citizens will likely continue to seek education opportunities in the region. Our recent study of ECFMG certification rates showed that the proportion of Caribbean-educated non-U.S. citizens achieving certification grew from 57.8% of those individuals who initiated the certification process to 66.5% over the study period.11 An investigation of the growth in Canadian citizen ECFMG registrants revealed an increase from 79 in 1993 to 580 in 2008, the majority of whom attended Caribbean schools.15 Reasons for this overall increase in international students attending Caribbean schools and then seeking ECFMG certification include the greater exposure of Caribbean schools within the international community, partnerships between international and Caribbean schools, and the ability of many Caribbean schools to offer non-U.S. citizens clinical clerkship opportunities within the United States.

The performance of Caribbean-educated IMGs, measured by first-attempt pass rates on the USMLE, shows some interesting shifts over the past decade. Similar to the results of our previous studies,10,11 there was wide variability across countries. For Step 1 and Step 2 CK, after a drop in pass rates during the first half of the study period, due in part to increases in the pass/fail cut points for Step 1 (2001 and 2007) and Step 2 CK (2003 and 2007), there has been a steady improvement in performance since 2004 when performance data for all countries are averaged. Interestingly, for three of the countries that educate large numbers of students seeking ECFMG certification (Dominica, Cayman Islands, and the Netherlands Antilles), Step 1 first-attempt pass rates dropped slightly in 2009. This performance shift could be influenced by the recent expansion of domestic education opportunities and a corresponding change in the pool of Caribbean applicants. Performance on Step 2 CS/CSA was relatively steady from 2000 to 2005, followed by a decrease and then a slight upward trend in the past year. This pattern is also likely influenced, in part, by increases of the passing cut point for CSA in 2002 and for Step 2 CS in 2006 and 2007.

Although this is our first study to detail Caribbean performance results by year, these findings should still be interpreted with caution. Most Caribbean countries have more than one school, with variations in numbers of students and resources. The student enrollment and attrition rates at these schools are unknown. Schools that require Step 1 for progression to clinical training may prohibit students who performed poorly in the first two years from taking the exam, potentially biasing the results. Post-Step 1 attrition across the exam series in numbers of students taking the USMLE could be due to a number of factors, including the artificial time constraints of the cohort used in the study, students' departure from medical studies during the clinical years of the educational program, variation in motivation to complete the USMLE for students seeking to practice in countries other than the United States, or transfer to another medical school in another country. Finally, registrants from schools educating primarily domestic students are a self-selected group that may differ from the country's total student population.

Back to Top | Article Outline

Summing Up

The number of schools in the Caribbean and individuals seeking training positions and licensure in the United States continues to increase. This study reveals wide variation in performance across countries in the region, and some meaningful exam success trends across the past decade, including an overall increase in Step 1 and Step 2 CK pass rates since 2004. However, a number of new initiatives within individual countries and at regional or international levels may influence the wide divergence in performance across countries in the Caribbean. For example, the ECFMG has recently announced that effective in 2023, physicians applying for ECFMG certification will be required to graduate from a medical school that has been appropriately accredited (www.ecfmg.org/accreditation). To satisfy this requirement, an applicant's medical school must be accredited through a formal process that uses criteria comparable to those established for U.S. MD-granting medical schools by the Liaison Committee on Medical Education or that uses other globally accepted criteria, such as those put forth by the World Federation for Medical Education. This ECFMG initiative and other plans to enhance medical education around the world will likely continue to improve the knowledge and skills of IMG physicians who intend to practice in the United States. As noted in previous investigations,16 further research following these Caribbean-educated physicians into residency and practice, and how they perform as physicians, is warranted.

Back to Top | Article Outline

Funding/Support:

None.

Back to Top | Article Outline

Other disclosures:

None.

Back to Top | Article Outline

Ethical approval:

Not applicable.

Back to Top | Article Outline

References

1Lakhan SE, Laird C. Addressing the primary care physician shortage in an evolving medical workforce. Int Arch Med. 2009;2:14.

2Brotherton SE, Etzel SI. Graduate medical education, 2008–2009. JAMA. 2009;302:1357–1372.

3Hing E, Lin S. Role of International Medical Graduates Providing Office-Based Medical Care: United States, 2005–2006. National Center for Health Statistics Data Brief 13. Hyattsville, Md: National Center for Health Statistics; 2009.

4Eckhert NL. Private schools of the Caribbean: Outsourcing medical education. Acad Med. 2010;85:622–630. http://journals.lww.com/academicmedicine/Abstract/2010/04000/Perspective__Private_Schools_of_the_Caribbean_.21.aspx. Accessed October 13, 2010.

5Johnson DA. Prospects for a national clearinghouse on international medical schools. J Med Licensure Discipline. 2008;94:7–11.

6Relman AS. Americans studying medicine abroad: The distressing facts. N Engl J Med. 1978;299:887–889.

7Weinberg E, Bell AI. Performance of United States citizens with foreign medical education on standardized medical examinations. N Engl J Med. 1978;299:858–862.

8Norcini JJ, McKinley DW, Boulet JR, Anderson MB. Educational Commission for Foreign Medical Graduates certification and specialty board certification among graduates of the Caribbean medical schools. Acad Med. 2006;81(10 suppl):S112–S115. http://journals.lww.com/academicmedicine/Fulltext/2006/10001/Educational_Commission_for_Foreign_Medical.28.aspx. Accessed October 13, 2010.

9Norcini J, Anderson MB, McKinley DW. The medical education of United States citizens who train abroad. Surgery. 2006;140:338–346.

10van Zanten M, Boulet JR. Medical education in the Caribbean: Variability in medical school programs and performance of students. Acad Med. 2008;83:S33–S36. http://journals.lww.com/academicmedicine/Fulltext/2008/10001/Medical_Education_in_the_Caribbean__Variability_in.9.aspx. Accessed October 13, 2010.

11van Zanten M, Boulet JR. Medical education in the Caribbean: Variability in Educational Commission for Foreign Medical Graduate certification rates and United States Medical Licensing Examination attempts. Acad Med. 2009;84(10 suppl):S13–S16. http://journals.lww.com/academicmedicine/Fulltext/2009/10001/Medical_Education_in_the_Caribbean__Variability_in.5.aspx. Accessed October 13, 2010.

12Jolly P, Garrison G, Boulet JR, Levitan T, Cooper RA. Three pathways to a physician career: Applicants to U.S. MD and DO schools and U.S. citizen applicants to international medical schools. Acad Med. 2008;83:1125–1131. http://journals.lww.com/academicmedicine/Fulltext/2008/12000/Three_Pathways_to_a_Physician_Career__Applicants.14.aspx. Accessed October 13, 2010.

13U.S. Department of Education: National Committee on Foreign Medical Education and Accreditation (NCFMEA). Report to the U.S. Congress by the National Committee on Foreign Medical Education and Accreditation Recommending Institutional Eligibility Criteria for Participation by Certain Foreign Medical Schools in the Federal Family Education Loan Program. http://www2.ed.gov/about/bdscomm/list/ncfmea-dir/reporttocongress2009.pdf. Accessed September 20, 2010.

14U.S. Government Accountability Office. Foreign Medical Schools: Education Should Improve Monitoring of Schools That Participate in the Federal Student Loan Program. http://www.gao.gov/new.items/d10412.pdf. Accessed September 20, 2010.

15Opalek A, Boulet JR, Dauphinee DW. Canadian citizen graduates of international medical schools: specialty choice and rate of GME entry. Paper presented at: AAMC Physician Workforce Research Conference; April 30, 2009; Washington, DC.

16Boulet JR, Cooper RA, Seeling SS, Norcini JJ, McKinley DW. U.S. citizens who obtain their medical degrees abroad: An overview, 1992–2006. Health Aff (Millwood). 2009;28:226–233.

Back to Top | Article Outline
Appendix 1 Cited Here...
Appendix 1
Appendix 1
Image Tools
Appendix 1, continue...
Appendix 1, continue...
Image Tools
Appendix 1, continue...
Appendix 1, continue...
Image Tools

*On October 10, 2010, as this report was being edited, the Netherlands Antilles dissolved politically. Two of the five islands became independent countries, and the remaining three islands joined the Netherlands. Cited Here...

© 2011 Association of American Medical Colleges

Login

Article Tools

Images

Share