Specialty Certification Status, Performance Ratings, and Disciplinary Actions of Internal Medicine Residents

Abstract

Purpose 

Little is known about the attrition of physicians trained in internal medicine (IM). The authors sought to examine career paths, disciplinary actions, and American Board of Medical Specialties (ABMS) certification status of IM residents.

Method 

Three datasets were combined to study 66,881 residents in Accreditation Council for Graduate Medical Education–accredited IM residency programs nationwide from 1995 to 2004. Group differences (among an American Board of Internal Medicine [ABIM]-certified cohort; an ABMS-certified cohort (but not ABIM-certified); and a noncertified cohort) in IM residency performance ratings, specialty certification status, year of initial IM training, and medical board disciplinary actions were examined. Analyses included chi-square tests, analysis of variance, pairwise comparisons, and logistic regressions.

Results 

Ninety-five percent of IM residents obtained ABIM certification; 1.6% received ABMS certification in another specialty; 3.4% received no ABMS specialty certification, of which 74.3% have a current medical license; and 66.6% self-reported IM as their primary specialty. During residency, the ABIM cohort performed better than those who never obtained ABIM certification. Disciplinary actions were lowest for the ABIM cohort (1.2%), 2.4% for the ABMS cohort, and highest and more severe for the noncertified cohort (6.0%).

Conclusions 

Only 5% of IM residents do not achieve IM certification. IM resident attrition minimally impacts physician supply, though those without certification appear to contribute disproportionately to poor physician performance indicators. Improved tracking of the U.S. physician workforce could aid policy makers in predicting manpower shifts in certain specialty areas, both during and after residency training.

In the coming decade, the primary care physician workforce will be inadequate to meet the health care needs of patients in the United States.^1,2 Millions of individuals newly insured through the Affordable Care Act and the expansion of health care delivery models will increase the demand for primary care.³ A 2004 survey of American Board of Internal Medicine (ABIM)-certified internists found that general internists leave the practice of internal medicine (IM) at a higher rate (21.0%) than those practicing a subspecialty of IM (4.0%).⁴ Those who reported leaving IM remain largely engaged in clinical practice (79.0%) but in another specialty.⁴ Little is known about their early career paths or their unique characteristics. In a 2006 editorial, Sox⁵ called for improved tracking of physician training and career paths in order to better predict the future physician workforce.

The 2000 Institute of Medicine report To Err Is Human: Building a Safer Health System highlighted the need for the United States to improve the quality of patient care and develop a consistent approach to ensure physicians’ competence throughout their careers.⁶ From a regulatory perspective, state medical boards in the United States have a primary responsibility to protect the public and do so by issuing an undifferentiated license to practice medicine. They are also empowered by statutory law to discipline physicians who fail to uphold the values of the profession.^7,8 While state licensure eligibility typically requires a minimum of one year of graduate medical education (GME) training for graduates of U.S. medical schools (and usually three years of GME for international medical graduates), American Board of Medical Specialties (ABMS) certification is a voluntary process, specific to a medical discipline. The ABIM is one of the ABMS’s 24-member boards; its certification requires successful completion of Accreditation Council for Graduate Medical Education (ACGME)-approved IM residency training. Residency program directors must attest that residents have mastered the six ACGME/ABMS core competencies (patient care, medical knowledge, practice-based learning and improvement, interpersonal and communication skills, professionalism, and systems-based practice) to be eligible to take the ABIM certification examination.⁹ The ABIM confers board certification status when the physician passes its certification examination. Although voluntary, certification is often used by health plans to select physicians, and by hospitals to issue practice privileges.¹⁰ Board certification has also been linked with better processes and outcomes of care.¹¹ Disciplinary action by state medical boards—such as a reprimand or licensure restriction, suspension, or revocation—against ABIM-certified internists correlates with performance during IM residency training. Better scores on measures of professionalism during residency training and on the ABIM certification examination correlate with fewer disciplinary actions.¹²

One element of the physician workforce that is not well understood is the training and career paths of IM residents. Our study examines the practice specialties of physicians in the decade following the start of training in a U.S. IM residency ACGME-accredited program. We also explore performance characteristics during and after training, including the incidence of state medical board disciplinary actions among physicians who were ABIM certified, those who were ABMS but not ABIM certified, and those who never certified with the ABMS.

Method

Subjects

The study included all allopathic physicians whose first year of training in a U.S. ACGME-accredited IM residency program occurred between 1995 and 2004 and who received at least one set of ABIM resident evaluation ratings. We selected trainees through 2004 to observe outcomes over time, including career paths and disciplinary actions. To validate the data, we combined the ABIM’s dataset of 74,699 physician records with data maintained at the Federation of State Medical Boards (FSMB). The FSMB provided up-to-date information on state medical licenses, physician disciplinary actions, and ABMS board certification status in other specialties; 74,635 records (or 99.9%) successfully matched. We were able to match all but 602 records automatically using the ABMS Identifier or the United States Medical Licensing Examination Identifier (USMLE ID); we matched 602 records using a combination of name, sex, date of birth, last four digits of a Social Security number, country of birth, medical school name, medical school graduation year, USMLE ID, and American Medical Association Identifier. Inconsistencies with respect to typographical errors, transposed names or numbers, name changes, or incorrect information were resolved manually by two raters (including L.M.D.). Sixty-four records could not be matched and were excluded. We also excluded a total of 3,730 osteopathic physicians. As well, 3,433 individuals who completed one year or less of categorical IM, primary care IM, or preliminary or transitional IM training were excluded because some non-ABIM specialties require an initial year of general training and many fulfill this requirement in IM, with no intent of entering IM. Finally, we excluded 591 individuals because their first year of training in IM occurred before 1995. Thus, the overall study population was 66,881. Physicians’ names and the last four digits of their Social Security number were removed from the dataset, and we assigned a unique identifier to deidentify the data. AMA Physician Masterfile self-reported specialty practice data were purchased by the ABIM for this study. We obtained state medical license status and state disciplinary actions from the FSMB database as of April 2013. Physicians were classified into three cohorts: those who completed training in IM and received ABIM board certification (ABIM cohort), those who received board certification from an ABMS specialty other than the ABIM (ABMS cohort), and those who never received ABMS board certification in any specialty (noncertified cohort) (Table 1).

Table 1: Demographic Characteristics of 66,881 Residents in Internal Medicine Residency Programs, 1995–2004, by Certification Status, From a Study of Specialty Certification Status, Performance Ratings, and Disciplinary Actions^a

Measures

We included the following demographic variables: sex, age at medical school graduation, type of medical school (U.S./Canadian or international), initial year of IM training, state medical license, and board certification status. We used performance measures obtained during and after IM residency training, number of attempts before passing the IM certification examination, IM certification examination score for the first attempt (made equivalent across years through a linear equating process, with scores ranging from 200 to 800¹³), number of IM residency programs attended, number of months in IM residency, and self-reported primary specialty practice area.

Disciplinary actions by state medical boards were included to address issues such as substance abuse, fraudulent billing, failure to fulfill minimum continuing medical education credits, and unprofessional conduct.¹⁴ We categorized disciplinary actions into three groups based on the severity of the licensing board action: very severe, when a license was revoked, suspended, denied, or surrendered; somewhat severe, when conditions were imposed, a license was restricted, or the physician was placed on probation; and less severe, when a reprimand occurred, continuous medical education was required, a fine was imposed, or an administrative or other formal action was taken.

We examined the ABIM’s standardized Residents’ Annual Evaluation Summary ratings in the first year of training submitted annually to the ABIM by residency program directors. Because components of the rating form changed over time, our analyses included only the two components that did not change during the study interval: medical knowledge and professionalism. Each component has descriptive anchors and is rated on a nine-point scale with 1 to 3 being unsatisfactory, 4 to 6 being satisfactory, and 7 to 9 being superior.¹² We considered medical knowledge and professionalism ratings as both have provided distinct contributions to models in prior studies.¹⁵

Analyses

To evaluate differences among groups for categorical variables, we performed chi-square tests. Analysis of variance (ANOVA) was used to compare means of continuous variables. Pairwise compar isons examined differences between the ABIM and ABMS cohorts and between the ABIM and noncertified cohorts. Logistic regression modeled the association between certification status using the following independent demographic- and performance-related variables: sex, age at medical school graduation, type of medical school, year of initial IM residency training, number of unique IM residency programs, presence of disciplinary actions, and ratings on medical knowledge and professionalism. We excluded variables related to number of months in IM residency training and holding a medical license, as certification status is dependent on both of these. Year of initial IM residency training was included in the model to account for differences over time. All two-way interactions with year of initial IM training were assessed, and we adjusted the model for multiple comparisons. We conducted all analyses using SAS 9.3 statistical software (SAS Institute Inc., Cary, North Carolina), and we were blinded to the physicians’ identities.

Results

Ninety-five percent (63,543) of residents who completed at least their first year of training in an ACGME-approved IM residency program between 1995 and 2004 obtained ABIM certification in IM (ABIM cohort). Of the remaining 5.0% of residents, 1.6% (1,051) received board certification from an ABMS specialty board other than the ABIM (ABMS cohort), while 3.4% (2,287) did not receive ABMS board certification in any specialty (noncertified cohort) (Table 1).

Demographic characteristics

Table 1 shows that a higher proportion of physicians in the noncertified cohort were male compared with the ABIM and ABMS cohorts (63.9% [1,462] versus 59.4% [37,710], 59.7% [625], P < .001). On average, physicians in the ABMS cohort and the noncertified cohort were older upon graduating medical school (mean age of 27.3 [4.0] years and 29.2 [5.3] years, respectively, compared with 26.3 [3.3] for the ABIM cohort, P < .001). A higher proportion of physicians in the ABIM and ABMS cohorts, compared with the noncertified cohort, held a state medical license at the time of the study (96.4% [61,225], 96.8% [1,017] versus 74.3% [1,699], P < .001) or had ever held a license (99.0% [62,904], 99.5% [1,046] versus 84.5% [1,933], P < .001). All pairwise comparisons were statistically significant for the ABIM versus the noncertified cohort, and most were significant for the ABIM versus the ABMS cohort with the exception of sex, and currently or ever holding a medical license.

Practice type

We examined whether residents certified by the ABIM went on to certify in a subspecialty of IM or another ABMS specialty, and how non-board-certified physicians self-reported their specialty. About half (51.1%; 31,049) of the ABIM-certified population went on to subspecialize in IM disciplines. Of the ABIM cohort, 7.6% (4,839) obtained another non-IM ABMS specialty certification. In the ABMS cohort, the most frequent specialty certification boards were psychiatry/neurology (21.3%; 224), pediatrics (19.9%; 209), anesthesiology (9.9%; 104), and radiology (9.0%; 95). In the noncertified cohort, self-report data showed that 66.6% (1,524) claimed their primary specialty as IM, 9.5% (217) claimed a subspecialty of IM, and 13.6% (313) claimed a non-IM specialty area.

Performance

Table 2 shows that a greater percentage of physicians in the ABIM and ABMS cohorts remained in the IM training program into which they matriculated compared with physicians in the noncertified cohort (94.9% [60,294] and 95.6% [1,005], respectively, versus 90.8% [2,118], P < .001). Those in the ABIM cohort had a higher average equated IM certification examination score than individuals who were not ABIM certified (497.5 [80.0] for the ABIM cohort versus 319.2 [56.0] for the ABMS cohort and 286.9 [58.8] for those with no certification, P < .001). The ABIM cohort also had a lower mean number of attempts on the ABIM certification examination before passing than physicians in the other two cohorts (1.1 [0.5] versus 2.0 [1.6] and 3.5 [2.6], P < .001). A lower percentage of the ABIM cohort received disciplinary action by a state medical board over the next decade compared with the ABMS cohort and the noncertified cohort (1.2% [747] versus 2.4% [25] and 6.0% [136], P < .001). Furthermore, the severity of disciplinary actions progressively increased across the three cohorts. The ABIM cohort had more of the less severe disciplinary actions (32.3%; 241) and fewer very severe disciplinary actions (29.9%; 223), while the noncertified cohort had the fewest less severe disciplinary actions (19.9%; 27) and the most very severe disciplinary actions (51.5%; 70).

Table 2: Performance Assessments of 66,881 Residents in Internal Medicine Residency Programs, 1995–2004, by Certification Status, From a Study of Specialty Certification Status, Performance Ratings, and Disciplinary Actions^a

ABIM Residents’ Annual Evaluation Summary ratings

Table 2 also shows that the ABIM cohort had higher average IM Residents’ Annual Evaluation Summary ratings from program directors than did physicians from the other two cohorts. This was true for ratings of medical knowledge in the first year of residency (6.3 [1.2] versus 5.8 [1.2] and 5.3 [1.1], P < .001) and for ratings of professionalism in the first year of residency (6.9 [1.1] versus 6.5 [1.2] and 6.3 [1.3], P < .001).

We also examined program director ratings from IM residency for the noncertified cohort broken down by those eligible for certification and those who were not (n = 2,287). Those deemed eligible to sit for the IM certification examination—based on completing an ACGME-accredited IM training program and program performance—either never passed or never sat for the examination (n = 1,575). Others were not eligible to sit for the examination (n = 712). ABIM-eligible candidates tended to have slightly lower ratings than the non-ABIM-eligible participants in medical knowledge (5.2 [1.1] versus 5.3 [1.2], P = .009) and slightly higher ratings than the non-ABIM-eligible participants in professionalism (6.4 [1.3] versus 6.2 [1.3], P = .006), in the first year of residency.

Regression analysis

We performed logistic regression analyses to determine which demographic and performance variables were most associated with the likelihood of a given certification status. Specifically, we compared the ABIM cohort with the ABMS cohort, and the ABIM cohort with the noncertified cohort (Table 3). There were no significant two-way interactions with year of initial IM residency training. When comparing the ABMS and ABIM cohorts, most variables in the model were statistically significant predictors of certification status, with the exception of sex (OR: 0.97 [0.86, 1.10]), number of unique IM residency programs (OR: 0.76 [0.57, 1.02]), and year of initial residency training (OR: 0.99 [0.97, 1.01]). In the comparison of the noncertified and ABIM cohorts, all variables were found to be significant predictors of certification status except for professionalism ratings (OR: 0.98 [0.93, 1.02]). The noncertified cohort was more likely to be male, have graduated from an international medical school, have attended more than one IM residency program, have received a disciplinary action, have a lower medical knowledge rating, and constitute a higher proportion of the population in the earlier years of the study.

Table 3: Logistic Regression Analysis Predicting Certification Status of 66,383 Residents in Internal Medicine Residency Programs, 1995–2004, From a Study of Specialty Certification Status, Performance Ratings, and Disciplinary Actions

Discussion

This research builds on prior work which studied factors during IM residency training that predict disciplinary actions of ABIM-certified practicing physicians.¹² The current cohort study includes not only those who were ABIM certified but also those who entered an IM residency, with the likely intention of completing their training. We found that 95% of residents who completed at least one year of training in an ACGME-accredited IM program between 1995 and 2004 became ABIM certified, which is a very low attrition rate compared with the 19.5% cumulative risk for attrition reported for 2007–2008 general surgery residents.¹⁶

Tracking physicians with IM residency training in the United States throughout their professional careers is complicated because physician workforce data are not easily obtained from any one source, and data sources are not always independently verifiable. We matched ABIM data to physician data from three additional major sources—the FSMB, ABMS, and the AMA Masterfile—to understand certification status and career paths. Three predictors of no certification were attending more than one IM residency training program, being rated lower in medical knowledge performance measures, and receiving more disciplinary actions from state medical boards. Disciplinary actions were five times more frequent and more often serious among those who did not obtain specialty board certification compared with the ABIM cohort.

The proportion of physicians with a disciplinary action was also higher for the ABMS cohort compared with the ABIM cohort, but it was much lower than for the noncertified cohort. There is some evidence that disciplinary action rates differ by specialty, which may help explain the cohort differences in the study.¹⁷ Compared with IM, disciplinary actions were more frequent among physicians in family practice, obstetrics–gynecology, and psychiatry, while actions for physicians in pediatrics and radiology were less common.¹⁷ Some residents who left IM may have done so because of professionalism lapses related to pursuing training in a specialty for which they were not well suited, though this inference is speculative. We did not conduct a disciplinary action analysis of residents who studied IM after transferring into the field from another specialty, to see if the effect was comparable. Additional research is needed to analyze disciplinary actions in other specialty areas to determine whether they generalize to prior study findings. Subsequent research could explore associations between varying rates of disciplinary actions and patient outcomes, recognizing that the vast majority of physicians in all three cohorts were never disciplined by a state licensing board.

This study does not address the issue of physicians who originally certified but are not maintaining their certification through the maintenance of certification program of the ABMS and its specialty boards. In the future it may be useful to explore whether there is a higher rate of workforce attrition and disciplinary actions among those who do not maintain their certification, and how their patient outcomes compare to outcomes for those who maintain certification. As the ACGME moves forward in seeking a single accreditation system for the GME of all allopathic and osteopathic physicians, an analysis of whether the findings in our study are applicable to osteopathic physicians may offer a more complete picture of the physician workforce in the United States.

This study has several limitations. First, data related to examination performance were based only on those who took the examination, so there were a number of physicians in the non-ABIM cohorts without examination scores. However, we did not use these data to draw conclusions about performance differences but instead used program director ratings, where the data were more complete. Second, we did not have access to data to inform us about physicians currently practicing outside of the United States. Third, there was limited information about the noncertified cohort, especially those who were never, or are no longer, practicing medicine. Fourth, we used self-reported data on specialty designation from the AMA Masterfile, and do not know how routinely it is updated. Fifth, because we followed physicians who trained a decade ago to allow for career decisions and disciplinary events to take place, our findings might not generalize to current trainees.

A structured approach for evaluating readiness for unsupervised practice in a specific discipline of medicine, such as specialty board certification, should increase the likelihood that physicians are equipped with appropriate skills to provide optimum patient care. A more robust workforce dataset should enable ACGME and program directors to better understand issues regarding changes in specialty preference as well as performance measures. More complete data would allow for more effective coaching of residents as they select career paths, and more informed remediation strategies. An appreciation of these parameters across all specialties may help program directors improve residency training programs, guide potential or existing trainees, and improve patient care. Likewise, policy makers could better predict manpower shifts in certain specialty areas, both during and after residency training, and use board certification as a potential marker of higher performance and fewer disciplinary actions in practice.

Acknowledgments: The authors thank Ann Kupinski, formerly at the American Board of Internal Medicine (ABIM), and Frann Holmes from the Federation of State Medical Boards (FSMB), who contributed to data collection and management.