Secondary Logo

Journal Logo

Does Performance on the American Board of Physical Medicine and Rehabilitation Initial Certification Examinations Predict Future Physician Disciplinary Actions?

Kinney, Carolyn L. MD; Raddatz, Mikaela M. PhD; Sliwa, James A. DO; Clark, Gary S. MD; Robinson, Lawrence R. MD

American Journal of Physical Medicine & Rehabilitation: December 2019 - Volume 98 - Issue 12 - p 1079–1083
doi: 10.1097/PHM.0000000000001250
Original Research Articles
Free

Objective The aim of the study was to determine the relationship between performance on the American Board of Physical Medicine and Rehabilitation primary certification examinations and the risk of subsequent disciplinary actions by state medical boards over a physician's career. The hypothesis is that physicians who do not pass either or both of the two initial specialty certification examinations are at higher risk of disciplinary action from a state medical licensing board.

Design This is a retrospective cohort study that analyzed board certification examination data from all physicians who completed physical medicine and rehabilitation residency between 1968 and 2017.

Results Matching examination and license data were available for 9889 physical medicine and rehabilitation physicians, who received a total of 547 disciplinary action reports through the Federation of State Medical Boards. The results showed a significant correlation between failing an American Board of Physical Medicine and Rehabilitation certification examination and the risk of subsequent disciplinary action by a state medical board. Failure to pass either the written (Part I) or oral (Part II) examination increased the risk of subsequent disciplinary action by 5.77-fold (P < 0.0001, 95% confidence interval = 4.07–8.18).

Conclusions Physicians in physical medicine and rehabilitation who do not pass initial certification examinations and become board certified are at higher risk of disciplinary action from a state medical licensing board throughout their careers.

From the American Board of Physical Medicine and Rehabilitation (CLK, MMR); Department of Physical Medicine and Rehabilitation, Mayo Clinic, Phoenix, Arizona (CLK); Department of Physical Medicine and Rehabilitation, Northwestern University Feinberg School of Medicine/Shirley Ryan AbilityLab, Chicago, Illinois (JAS); Department of Physical Medicine and Rehabilitation, Case Western Reserve University/Metro Health Medical Center, Cleveland, Ohio (GSC); and Division of Physical Medicine and Rehabilitation, University of Toronto, Toronto, Ontario, Canada (LRR).

All correspondence should be addressed to: Carolyn L. Kinney, MD, American Board of Physical Medicine and Rehabilitation, 3015 Allegro Park Ln SW, Rochester, MN 55902.

CLK and MMR are employees of the American Board of Physical Medicine and Rehabilitation.

Financial disclosure statements have been obtained, and no conflicts of interest have been reported by the authors or by any individuals in control of the content of this article.

Online date: June 20, 2019

There is no published literature on the frequency of medical license disciplinary actions (DAs) involving physicians who practice Physical Medicine and Rehabilitation (PM&R). However, when these DAs occur, the consequences can be significant. Disciplinary actions can result from breaches in medical practice regulations or violations of medical ethics and professionalism standards impacting patient care safety.1,2 The consequences of these actions may include a number of state medical licensing board sanctions, ranging from fines or additional continuing medical education, to reprimands, license restrictions, and license revocations.3 Such state licensing board DAs are reported regularly to the American Board of Physical Medicine and Rehabilitation (ABPMR), as they are to the other 23 member boards of the American Board of Medical Specialties, via the DA notifications system (DANS) of the Federation of State Medical Boards (FSMB).3,4 The FSMB represents the 70 medical boards in the United States, its territories, and the District of Columbia.3 The DANS system categorizes DAs by severity of action and the type of violation reported. The following 11 categories constitute the classification of reasons for DA: substance use, malpractice, fraud, inappropriate prescribing, criminal actions, impairment, license/board violation, records violations, failure to supervise, unprofessional conduct, and other inappropriate activity; each category also has a number of added descriptors.3

A DA is typically the end point of a series of events that can adversely impact patient care. Identifying early risk factors for subsequent DA could perhaps lead to early interventions reducing such actions and improving patient care. Such interventions could include additional training or practice options in less risky environments. Previous authors have reported that performance in medical school and residency can predict the chances of subsequent DAs.5,6 A few recent studies in other medical specialties have examined the ability of certification examinations to predict the chances of subsequent DAs.5,7 There is no literature regarding the ability to predict DAs in physiatrists.

The ABPMR is one of 24-member boards of the American Board of Medical Specialties, with a primary mission of improving the quality of health care through setting professional and educational standards for medical specialty practice and certification.4 For the ABPMR, this mission is accomplished through a process of certification and maintenance of certification (MOC) that fosters excellence and encourages continuous learning.8

Primary board certification in PM&R is achieved after successful completion of required PM&R training in an Accreditation Council for Graduate Medical Education-approved residency, and passing two initial certification examinations, a written examination (Part I), and an oral examination (Part II).9

These examinations are designed to test a candidate's knowledge (Part I) and its application to patient care (Part II).10,11 The ABPMR has previously reported the performance of these assessments, both of which have high validity in measuring their design attributes.12,13

Recently, board certification, both primary certification and MOC, have come under increased scrutiny, with questions arising as to the value of these processes to measure actual practice performance and patient care outcomes.14

The purpose of this study is to (a) determine the rate of DAs for PM&R physicians and investigate the correlation of DAs with sex, practice subspecialty, and medical school training location (United States vs. international) and (b) investigate the relationship between performance on the ABPMR primary certification examinations and the risk of subsequent DAs by state medical boards during a physiatrist's career. The hypothesis tested is that physicians who do not pass either or both of the two initial certification examinations are at higher risk of DAs from a state medical licensing board. A secondary goal of the study is to determine whether either of the two initial ABPMR certifying examinations is more predictive of physicians receiving future DAs.

Back to Top | Article Outline

METHODS

The study was a retrospective cohort study completed with data obtained from ABPMR (certification examination data) matched with data provided by the FSMB on DAs for PM&R physicians.

This study analyzed data from all physicians who completed a PM&R residency between 1968 and 2017, and all data were deidentified. For analysis of the association between certification examinations and presence of a DA, we included only physicians for whom there also existed a pass/fail result for both the Part I and Part II Examinations.

Records of individuals who completed PM&R training were sent to FSMB for matching with the DANS database and categorized into those with/without a DA. Because prior literature has demonstrated an influence of sex and site of medical school training,5,7 effects of sex and US medical graduate (USG) versus international medical graduate (IMG) status were examined in the larger group.

The subset of individuals with pass/fail examination results was analyzed using Cox proportional hazards regression analysis to study the relationship of DAs with the ABPMR certification examination results and the risk of DA's associated with sex and medical school training location (USG vs. IMG).

The study received institutional review board approval as a minimal risk study from Advarra institutional review board (Protocol# SAIRB-17-0023 IRB# 201701718).

Back to Top | Article Outline

RESULTS

The ABPMR identified 15,925 physician records that qualified for study inclusion, of which the FSMB was able to match 15,077 (94.7%) based on the American Board of Medical Specialties ID, name, and birth date. Of these 15,077 physicians with license data, 9889 physicians (65.6%) had pass/fail data for Part I and Part II. This subgroup of 9889 subjects was used for the comparison analyses of examination results.

Of the initial pool of 15,077 subjects, there were 547 (3.6%) DANS reports received through the FSMB for PM&R physicians from 1969 to 2017. For the 10 yrs from 2008 to 2017, the average rate for disciplinary reports for PM&R physicians was 0.65% per year. Table 1 lists the basis of DA for those physicians with DANS reports. Many physicians had more than one basis of reporting; hence, the total (1330 codes) exceeds 547.

TABLE 1

TABLE 1

Both sex and site of training were associated with risk of a DA (Table 2). Of the initial pool of 15,077 candidates, most (65.7%) were male. Men were overrepresented in physiatrists with DAs (82.6%) compared with women. Disciplinary actions were reported in 4.56% of male PM&R physicians and 1.83% of women PM&R physicians in the study set, which is a statistically significant association (χ2 = 72.34, P < 0.0001). There were 3864 candidates who were IMGs, which represented 25.6% of the total pool. This group was overrepresented in those with DAs (5.1%), which is a statistically significant association (χ2 = 33.26, P < 0.0001) compared with USGs with DAs (3.1%).

TABLE 2

TABLE 2

In the Cox proportional hazards regression analysis, the 9889 physicians with examination scores available were examined for effects of sex, USG vs. IMG, and certification examination results (Table 3). Demographic factors increasing risk for medical license actions included male sex, with 2.29 times higher risk (P < 0.0001), and international medical school graduation, with 2.11 times higher risk (P < 0.0001).

TABLE 3

TABLE 3

As demonstrated in Table 3, failing the Part I and/or Part II Examination carried a higher risk of DAs than either male sex or IMG status. Failure to pass Part II after passing Part I, and therefore not achieving board certification, elevated the risk for future DA to more than four times the reference group. Physicians who failed to pass either examination had more than a five-fold increased risk for future medical license DAs. Those who ultimately passed both examinations but required multiple attempts to pass the Part I Examination, had a higher risk for DA (although this did not reach statistical significance). Conversely, requiring multiple attempts to pass Part II statistically increased risk nearly two-fold (P < 0.0001). Physicians who required multiple attempts to pass both primary certification examinations had more than a three-fold increased risk for licensure actions. Scores on the Part I and Part II Examinations are consistent with this finding, with scores in the lowest (fourth) quartile corresponding to increased risk for DAs (P < 0.0001) (Fig. 1, Table 4). Interestingly, PM&R physicians additionally certified in the subspecialty of pain medicine are overrepresented among PM&R physicians with DANS reports (Table 5).

FIGURE 1

FIGURE 1

TABLE 4

TABLE 4

TABLE 5

TABLE 5

Back to Top | Article Outline

DISCUSSION

This study demonstrates a significant correlation between failing an ABPMR certification examination and risk of subsequent DA by a state medical board. Failing both parts of the examination, or just Part II, dramatically raises the risk of subsequent DA, by 5.8- and 4.3-fold, respectively. Physicians who ultimately pass the examinations but require multiple attempts on both Part I and Part II have a greater than 3-fold risk of a DA compared with those who pass both examinations on their first attempt. The association of DAs with failing the certification examinations is much stronger than with either sex or country of medical school training. This correlation is present even with quartile scores on the Part I and Part II Examinations, with a significantly increased risk for DAs noted with lower (fourth) quartile examination performance (Fig. 1, Table 4).

This study is in agreement with reports from other medical specialties examining medical license DAs.7 An increased rate of physician DAs has been reported to be higher for IMGs and for physicians not passing the primary board certification examinations in internal medicine.6,15

As seen in this study, previous reports have documented the higher rate of DAs associated with male sex.2,6,7,15 The lower rate of DAs in female physicians, as seen in this study, has also been previously reported and parallels the lower incidence of malpractice claims for this group.15,16 Possible explanations have centered on more effective communication skills in female physicians, resulting in fewer patient complaints.16,17 A recent ABPMR analysis of data from the Part II Examination confirms the scores of female physicians in the domain of interpersonal communication were higher than scores for men, and similar findings have been reported for the United States Medical Licensing Examination Step 2 Clinical Skills Communication and Interpersonal Skills scores.18,19 Effective communication is known to decrease risk for complaints in physician patient interactions, and the relative strength of female physicians in this domain may underlie their reduced risk of medical license DAs.16

Inconsistencies in the literature are noted on data for United States versus international medical school graduation across specialties with respect to DAs, with IMGs having a significantly higher risk (more than twofold) of DAs in this study, but not in studies from other specialties.2,7 In the study by Kohatsu et al.,15 a multispecialty analysis of license actions in one state concluded that certain physician characteristics were associated with increased likelihood of medical board DAs, specifically male sex, lack of board certification, increasing age, and international medical school education. In a study of anesthesiologists by Zhou et al.,7 American medical school graduates had a higher risk for DAs when analyzed in the Cox proportional hazards models, although the cumulative incidence rate of license actions did not differ significantly between USGs and IMGs.7 A deeper analysis of region/country of training would likely help in further understanding these results for PM&R but is beyond the scope of this study. Although we explored sex and site of training (United States vs IMG) as co-founders because of prior literature cited previously, there could be a number of other co-founders who have an impact on the risk of DAs as well, which were not explored in this study.

Lack of board certification is consistently reported to be associated with risk for medical license DAs.5–7,15 Zhou et al.7 reported that anesthesiologists, who have a two-phase certification examination process similar to that of PM&R, who passed only the written certification examination had a similar risk for future DAs as those who did not pass either examination; however, the predictive validity of the written examination alone was not assessed in that study. In our study, only the pass/fail dichotomous performance on the Part II (oral) examination significantly separately impacted risk for future DAs.

The predictive ability of examination scores for future DAs has been previously reported.6,20 These studies confirm the predictive validity of knowledge assessments for future DAs, including both pass/fail dichotomous outcomes as well as score-dependent analyses,1,6,21 with this predictive value being independent of other measures of professionalism.1,22 Given data indicating many DAs occur in the category of “incompetence” or “negligence,” it is possible that increased medical knowledge confers improved ability to diagnose and treat appropriately.2

The ABPMR has reviewed and analyzed the domains assessed by its Part I and Part II Examinations.12,13 Although both examinations assess knowledge and have significant correlation for assessing the domains of problem-solving and patient management, the Part II Examination more uniquely evaluates the domains of interpersonal/communication skills and systems-based practice.9,13 This was also the finding in Zhou et al.,7 for the oral certification examination administered to anesthesiologists.

Certification examinations serve as a proxy for measuring attributes and skills physicians apply to patient care. Although we are unable to state what specific attributes measured by the ABPMR certification examinations are responsible for the ability of the examinations to predict future DAs, we are able to state physician performance through the ABPMR certification process does correlate with future state license DAs. In addition, the domains assessed more specifically by the Part II Examination, interpersonal/communication skills and systems-based practice, may explain the increased risk of DAs in those who fail the Part II Examination. The ABPMR process for initial certification requires passing the Part I Examination to be eligible to take the Part II Examination. Therefore, our score data for Part II represent a subset of physicians who have passed the Part I Examination. This could explain the greater hazards ratio for Part I as compared with Part II because those candidates already made it through one step of the certification process.

Perhaps not surprisingly, PM&R physicians with subspecialty certification in Pain Medicine are at significantly higher risk for DAs compared with other PM&R subspecialty diplomates. The procedural nature of pain medicine practice may confer additional risk for DAs. Because many of the DANS reports relate to opioid prescribing practices, this group of physicians has particular practice risk, and additional education about this risk may be beneficial during fellowship training.

Our study has several limitations. First, this study does not establish causality; it only demonstrates a significant association. Our study did not include an analysis of the type of DA (category) in relation to board certification performance, and as such, we are not able to correlate the risk of failing the certification examinations with specific types of DAs. The numbers of reports in each subgroup are too small for these additional analyses. In addition, we did not account for career changes that may have occurred in some of the physicians studied, either through changing to and from nonclinical types of practice or through leaving medical practice entirely; these career changes may have impacted the data, ranging from increasing exposure to license actions to eliminating the risk completely. The type and duration of practice were not included in our analysis, and therefore, we cannot comment on whether certain practice characteristics (inpatient vs. outpatient, rural vs. urban, academic vs. private practice) confer their own risk for DAs, independent of board certification examination performance. Furthermore, the overall rate of DA reports for PM&R physicians (3.6% of total physicians attempting board certification) reflects a variety of time exposures, with some physicians having been in practice for many years, and others having only recently graduated. Finally, because our data are specific to PM&R, we cannot generalize our results to other medical specialties.

Future research would be useful to address whether MOC testing is also similarly predictive of subsequent DA. Ultimately, it would be of great interest to explore whether there are any interventions that can modify the risk of DA. Can individuals, once they become aware of their risk stratification, do anything to reduce their risk of DA?

Back to Top | Article Outline

CONCLUSIONS

This study supports our hypothesis that PM&R physicians who do not pass the initial certification examinations to become board certified are at higher risk of DA from a state medical licensing board. This finding was true for the overall dichotomous pass/fail examination outcome but was also true for needing multiple examination attempts to eventually successfully achieve ABPMR Board Certification. Passing the Part II Examination was independently (even after passing the Part I Examination) predictive of lower risk for future DAs, suggesting certain domains better assessed by the Part II Examination, such as interpersonal/communication skills and system-based practice, contribute significantly to this predictive validity.

Physician specialty board certification should relate to outcomes valued by the public consumers of health care. Initial board certification in PM&R is associated with a lower risk of state medical board DAs throughout a physician's career.

Back to Top | Article Outline

ACKNOWLEDGMENTS

The authors thank Diane Bur, Kevin Randleman, and Kim Van Brunt for their excellent technical contributions to this study.

Back to Top | Article Outline

REFERENCES

1. McDonald F, Duhigg L, Arnold G, et al: The American Board of Internal Medicine maintenance of certification examination and state medical board disciplinary actions: a population cohort study. J Gen Intern Med 2018;33:1292–8
2. Morrison J, Wickersham P: Physicians disciplined by a state medical board. JAMA 1998;279:1889–93
3. U.S. Medical Regulatory Trends and Actions. 2018. Federation of State Medical Boards. Available at: https://www.fsmb.org/siteassets/advocacy/publications/us-medical-regulatory-trends-actions.pdf. Accessed November 30, 2018. Pg. 19–21
4. About ABMS. Available at: https://www.abms.org/about-abms/faqs/ Accessed November 30, 2018
5. Lipner R, Young A, Chaudhry H, et al: Specialty certification status, performance ratings, and disciplinary actions of internal medicine residents. Acad Med 2016;91:376–81
6. Papadakis MA, Arnold GK, Blank LL, et al: Performance during internal medicine residency training and subsequent disciplinary action by state licensing boards. Ann Intern Med 2008;148:869–76
7. Zhou Y, Sun H, Culley D, et al: Effectiveness of written and oral specialty certification examinations to predict actions against the medical licenses of anesthesiologists. Anesthesiology 2017;126:1171–9
8. ABPMR. Available at: https://www.abpmr.org/About Accessed November 30, 2018
9. Driscoll S, Geis C, Raddatz M, et al: Predictors of performance on the American Board of Physical Medicine and Rehabilitation Maintenance of Certification Examination. PM R 2019;10:1361–7
10. ABPMR. Available at: https://www.abpmr.org/Primary/PartI Accessed November 30, 2018
11. ABPMR. Available at: https://www.abpmr.org/Primary/PartII# Accessed November 30, 2018
12. Raddatz M, Robinson LR: Demonstrating construct validity of the American Board of Physical Medicine and Rehabilitation Part I Examination: an analysis of dimensionality. PM R 2017;9:985–9
13. Chiodo A, Raddatz M, Driscoll SW, et al: Should There Be A Part II ABPMR Examination? A Psychometric Inquiry, Function and Rehabilitation. PM R 2019; doi: 10.1002/pmrj.12126
14. Lipner RS, Hess BJ, Phillips RL Jr: Specialty board certification in the United States: issues and evidence. J Contin Educ Health Prof 2013;33(suppl 1):S20–35
15. Kohatsu ND, Gould D, Ross LK, et al: Characteristics associated with physician discipline: a case–control study. Arch Intern Med 2004;164:653–8
16. Taragin MI, Wilczek AP, Karns E, et al: Physician demographics and the risk of medical malpractice. Am J Med 1992;93:537–42
17. Jefferson L, Bloor K, Birks Y, et al: Effect of physicians' gender on communication and consultation length: a systematic review and meta-analysis. J Health Serv Res Policy 2013;18:242–8
18. Cuddy MM, Swygert KA, Swanson DB, et al: A multilevel analysis of examinee gender, standardized patient gender, and United States Medical Licensing Examination Step 2 clinical skills communication and interpersonal skills scores. Acad Med 2011;86:S17–20
19. Driscoll S, Raddatz MM, Robinson LR, et al: Is there evidence of gender bias in the oral examination for initial certification by the American Board of Physical Medicine & Rehabilitation? Am J Phys Med Rehabil 2019;98:512–5
20. Papadakis MA, Teherani A, Banach MA, et al: Disciplinary action by medical boards and prior behavior in medical school. N Engl J Med 2005;353:2673–82
21. Cuddy M, Young A, Gelman A, et al: Exploring the relationships between USMLE performance and disciplinary action in practice: a validity study of score inferences from a licensure examination. Acad Med 2017;92:178–85
22. McCoy CP, Stenerson MB, Halvorsen AJ, et al: Association of volume of patient encounters with residents' in-training examination performance. J Gen Intern Med 2013;28:1035–41
Keywords:

Board Certification; Licensure; Physical Medicine and Rehabilitation; Clinical Competence; Disciplinary Action

Copyright © 2019 Wolters Kluwer Health, Inc. All rights reserved.