Ries, Andrew MD, MPH; Wingard, Deborah PhD; Gamst, Anthony PhD; Larsen, Catherine MPH; Farrell, Elizabeth; Reznik, Vivian MD, MPH
Faculty are the cornerstones of academic medicine. It is important but difficult to attract and retain the best faculty because of the many pressures facing both individuals and institutions. For many clinical faculty, after long years of training, often with significant educational debt and the social demands of personal and family life, the added pressures of teaching and scholarship make the financial sacrifices in academic medicine difficult to justify in the face of the private practice lure. For research faculty, the competition to secure grant funding and publishing in high-quality journals may make the barriers to academic success seem insurmountable. For institutions that have invested substantially in young faculty, replacing those who leave prematurely is a costly proposition.
Facilitating the success of promising young faculty can have a significant effect on an institution’s future. Data from the Association of American Medical Colleges (AAMC) indicate that the 10-year retention rate of 4,279 first-time assistant professors hired from 1981 to 1997 was 43%.1 Nurturing junior faculty though organized faculty development and mentoring programs may be important in fostering academic success,2–4 but critically evaluating the benefits of such programs is difficult because of the absence of well-defined methods for quantifying academic success for the variety of faculty job descriptions in academic medicine. As a result, there is a paucity of published research on academic success and faculty retention in academic medicine.
The National Center of Leadership in Academic Medicine (NCLAM) at the University of California, San Diego (UCSD) is an innovative, seven-month, faculty development program for junior faculty (assistant professors) in health sciences that includes professional development workshops, academic strategic career planning, individualized academic performance counseling, mentoring with a senior faculty member focused on a professional development contract, and network building with other faculty.5 It is open to all junior faculty, and selection for the limited number of positions is made by the program’s Leadership Council based on candidate applications and recommendations by department chairs.
The purpose of this study was to develop and demonstrate the usefulness of quantitative methods for evaluating the retention and academic success of junior faculty at one school of medicine.
A database assembled for a previous analysis catalogued available information for all new assistant professors hired in health sciences at UCSD between July 1, 1988, and December 31, 2005.4 It included information on initial hire date, gender, ethnicity, date of separation, and participation in a formal junior faculty development program, NCLAM, that was established at UCSD in 1998. Because the purposes of the current study were to evaluate (1) potential confounders in our previous analyses and (2) subsequent academic success of this cohort of newly hired assistant professors, we used the existing database for these analyses.
At UCSD, all assistant professors must be promoted to the associate rank by the end of an eight-year probationary period from the date they are hired. They undergo regular academic review every two years. After four years, they undergo a formal “appraisal” of their performance to date and likelihood of future promotion. Departments are expected to review all assistant professors for promotion after six years, although the actual promotion review may be delayed to the seventh or, sometimes, eighth year on request with appropriate justification.
All assistant professors are eligible to apply for the NCLAM program, but they are encouraged to do so after one to two years on the faculty. Selection preference is often given to repeat applicants not chosen the first time. In our experience, despite the limited number of available positions, very few assistant professors who wish to participate and apply are not accepted sometime during their eight-year probationary period. There are typically 25 to 30 applicants for 16 to 18 positions in each NCLAM class.
At UCSD, NCLAM is the only formal faculty development program for junior faculty. Faculty development activities for non-NCLAM participants are primarily the responsibility of individual departments.
Most assistant professors at UCSD are appointed in the salaried adjunct professor series for researchers or the health sciences clinical professor series for clinicians. Some researchers are appointed in the ladder rank (professor) or professor in residence series, and some clinical scholars in the professor of clinical “X” series. We included faculty in all five of these salaried professorial series at UCSD in the analyses.
All analyses in this study were performed under approval by the UCSD human research protections program.
Evaluation of faculty retention
To adjust for possible confounding effects of temporal changes related to when a faculty member was hired, academic series and primary job responsibilities, and home department on the effects of the faculty development program on faculty retention, we matched each participant from the first eight NCLAM classes (1999–2006) with two nonparticipants in the newly hired assistant professor database who were closest to the NCLAM participant on the following characteristics, in order of priority: (1) gender, (2) academic series (research versus clinical primary job description), (3) initial academic rank/step (academic experience when hired), (4) hire date (within 12 months, to control for temporal changes in internal and external institutional variables), and (5) department.
To maximize identification of appropriate matches, when exact matches were not available we allowed substitutions for closely related academic series and departments. Academic series substitutions for salaried faculty considered to have a primary research focus included ladder rank, in residence, and adjunct professor titles. Substitutions for those considered to have a primary clinical focus included the salaried clinical and clinical “X” professor series. For smaller departments, we allowed substitutions within the following related groupings: (1) anesthesiology, ophthalmology, orthopedic surgery, reproductive medicine, surgery (surgical disciplines), (2) pediatrics, family and preventive medicine, neurosciences (medical disciplines), (3) pathology, radiology (laboratory support disciplines), and (4) cellular and molecular medicine, pharmacology (basic science disciplines). Because of their large size, we did not allow substitutions for the departments of medicine and psychiatry.
We constructed Kaplan–Meier survival curves to characterize retention at UCSD from the initial hire date until date of separation or the end of the eight-year probationary period for the NCLAM participants and matched set of non-NCLAM participants. We censored all faculty in the cohort remaining at UCSD on December 31, 2006, the last date for data analyses. Because of the small number of faculty from underrepresented ethnic groups, we pooled all faculty self-designated as Hispanic, African American, and American Indian into one group of underrepresented minorities (URM). We coded all others as majority.
We evaluated the univariate effects of gender, ethnicity, and participation in NCLAM by Kaplan–Meier curves, compared with log-rank tests. To evaluate possible gender differences on faculty retention, we performed a multivariate analysis using the Cox proportional hazards technique including the independent variables of NCLAM participation, gender, and an interaction term of NCLAM × gender. Similarly, we evaluated the possibility of differential retention of URM faculty with a Cox model including the independent variables of NCLAM participation, URM status, and an interaction term of NCLAM × URM. We assumed that a differential effect of the NCLAM program on either gender or URM status in these multivariate analyses would be indicated by statistical significance for the interaction term.
Evaluation of academic success
All assistant and associate professors at UCSD undergo formal academic review every two years; all full professors are reviewed every three or four years. To quantify academic success, one of only two reviewers (coauthors E.F. and V.R.) examined each faculty member’s most recent academic review file for the number of items listed in five categories of academic success. We chose these two reviewers because of their experience and familiarity with academic files at UCSD. The five categories of academic success, selected for review based on available information in the academic review files, were
1. Leadership and Professional Activities: administrative positions; committee member or chair/cochair at the department, school, campus, regional, national, or international level; member or chair/cochair of a research grant review committee; member of a journal editorial board;
2. Honors and Awards;
3. Contracts and Grants: research contracts and grants as principal or coinvestigator;
4. Teaching and Mentoring: course director, number of mentees/advisees; and
The two reviewers then piloted the review process in several files to ensure that the data were available and could be extracted, and to resolve questions identified by the two reviewers and any inconsistencies in their review processes. After the pilot phase, we made minor modifications to the data collection form.
To evaluate the effects of the faculty development program on subsequent academic success, we created a second matched set among faculty in the database who were still at UCSD as of December 31, 2009. To allow for a sufficient period of time to quantify academic success (i.e., evaluation for promotion to the associate rank by the end of the eight-year probationary period for assistant professors), we included only NCLAM participants from the first five classes (1999–2003). Because of the smaller number of eligible faculty in the database, we matched only one non-NCLAM participant with each NCLAM participant using the same strategy that we used for the retention analyses.
Because of the small sample size and nonnormal distribution of data within the various categories of academic success, we used box plots to display the differences between the matched sets of NCLAM and non-NCLAM participants. We evaluated statistical significance between the two groups using the Wilcoxon signed-rank test.
There were 122 assistant professor participants in the first eight classes of the NCLAM program (1999–2006). Among these, two appropriate non-NCLAM matches were identified for 89, only one match for 24, and no match for 9 (who were excluded from the analyses). Therefore, the matched-pair analyses included a total of 113 NCLAM (59 females, 54 males) and 202 matched non-NCLAM participants (104 females, 98 males). Of note, significantly more NCLAM participants (18/113) than nonparticipants (8/202) were classified as URM (P < .001 by chi-square).
Figure 1 presents the overall Kaplan–Meier survival curves for the NCLAM and matched non-NCLAM participants. Overall, retention of NCLAM participants was significantly greater than non-NCLAM participants (P = .04). At the end of their eight-year probationary period, 76 of the 113 (67%) NCLAM participants versus 113 of the 202 (56%) matched nonparticipants were still on the faculty at UCSD.
There were no significant effects of the covariates for gender or URM status in the multivariate Cox proportional hazards analyses for either the overall effects of gender (P = .95) or URM status (P = .49), or for the interaction terms (P = .64 and .55, respectively).
For the analysis of academic success, 47 of the 79 (59%) NCLAM assistant professor participants in the first five classes (1999–2003) were still at UCSD as of December 31, 2009. For these, we identified 1:1 matches for 29 individuals from assistant professor non-NCLAM participants still at UCSD. For the other 18 participants, no appropriate match was available.
Figure 2 presents box plot results of the paired comparisons of the 29 NCLAM and 29 non-NCLAM participants on the basis of the five criteria created for evaluating academic success. These results suggest a consistent trend in favor of the NCLAM participants in each of the five categories of academic success. In the large majority of pairs, the NCLAM participant had more items in each category than the matched non-NCLAM participant. However, only in the leadership and professional activities category did the difference between the NCLAM and matched non-NCLAM participants reach statistical significance (P = .03).
We have described the development and use of innovative methods for quantifying and evaluating both retention and academic success of new junior faculty in academic medicine. Such methods may be helpful in stimulating more quantitative and systematic approaches to research in these understudied areas. Given the difficulty and expense of recruiting and retaining high-quality faculty in academic medicine in this data-driven era of limited resources, such quantitative approaches may be helpful in (1) more precisely describing trends in faculty retention, (2) better understanding how criteria for academic success are being applied, (3) critically evaluating existing and new approaches to foster retention and academic success, and (4) justifying the costs of personnel and programs. We hope that the experience and results of this study will help encourage others to pursue work in this field as a legitimate avenue of academic research.
The multiple professional responsibilities and pressures for faculty in academic medicine pose serious challenges to their willingness to remain in academic medicine and their subsequent academic success if they do. Academic advancement may require excellence in clinical practice in addition to superior teaching, research, and university or professional service. Long periods of education and training, often with substantial associated debt, create personal and financial pressures that lure potentially successful faculty away from academic medicine.
It has been suggested that organized faculty development and mentoring programs, particularly for junior faculty, can have an important influence on faculty retention and, ultimately, career success in academic medicine.2–4 At UCSD, the NCLAM program was established in 1998 through seed funding from the Office of Women’s Health of the U.S. Department of Health and Human Services.3 The UCSD NCLAM program, described in previous publications,4–6 was institutionalized as a formal mentoring and skill development program for selected junior faculty and has been run continuously since its inception.
In a previous publication, we reported that retention of all junior faculty at UCSD was significantly improved after institution of the NCLAM program, and faculty who participated in NCLAM had significantly better retention at UCSD after eight years, the usual maximum term of the probationary period for assistant professors in the University of California.4 However, as noted in that article, these findings could have been influenced by other factors contemporary with implementation of the junior faculty development program, particularly in a new medical school like UCSD that experienced considerable growth and change during the time of those analyses. In addition, in choosing participants from applicants for the junior faculty development program, there may have been selection bias for individuals who were aware of, interested in, and motivated to participate in such programs.
By extending the previous work with the use of matched sets, the results of the current study address limitations in the previous publication and provide a critical reevaluation and important confirmation of the benefits of an organized junior faculty development program on faculty retention in academic medicine. When matched sets controlled for gender, hire date, academic series, and department, junior faculty who participated in the junior faculty development program at UCSD early in their career were more likely to remain at UCSD beyond their eight-year probationary period as assistant professors (and required promotion to associate professor rank).
The use of survival analysis techniques is an innovative way to model faculty retention over time. Although similar to results published previously from this same cohort, the restriction in this analysis to matched sets significantly strengthens the evidence that a faculty development program improves faculty retention. Our previously published study analyzed data from all newly hired assistant professors across an 18-year period that encompassed the time period both before and after the NCLAM program was implemented. This 18-year period represents a critical period of growth for a relatively young medical school that had been in existence for only 37 years at that time. Thus, in the previous analyses we were not able to adequately control for important possible confounders, including temporal changes in medicine in general and in the environment at UCSD in particular, gender, academic series (clinical versus research primary responsibilities), and department/medical discipline. In the present survival analysis, the significantly greater retention of the junior faculty NCLAM participants was maintained even when the participants were compared with nonparticipants hired at the same time who were matched on the basis of hire date, gender, academic series, and department/discipline.
Also of note, the observed retention/promotion of new junior faculty at UCSD who participated in the faculty development program (67%) was significantly higher than for non-NCLAM participants (56%, see Figure 1). Both of these figures are higher than the AAMC data on (1) 10-year retention for all U.S. medical schools of 43% for first-time assistant professors remaining at the same medical school1 and (2) promotion rates for new assistant professors of 33%.7
It should be noted that improved eight-year retention of new assistant professors does not necessarily equate to greater success in academic medicine. However, at UCSD the eight-year probationary period after the initial hire date is the maximum time allowed for all assistant professors to be promoted to the associate professor rank. In general, successful junior faculty do not leave the institution of their first faculty appointment before being promoted. In our experience, assistant professors who leave during their probationary period do so for nonacademic positions. After promotion, however, it is not uncommon for successful faculty to be recruited to other academic institutions. Because we did not have information about the reasons faculty left UCSD, we believed that censoring the data at eight years would provide the most appropriate indicator of academic success for assistant professors at UCSD.
Integral to career decisions of faculty to remain in academic medicine are satisfaction with their positions and success in the academic pursuits for which they were initially hired.8,9 Given the variety, complexity, and likely change and evolution of job descriptions over time for such faculty, it is important to better understand factors that influence academic success. Further research in this area is much needed and will require better methods for quantifying and evaluating success. The methods developed in this study are a definite step in that direction, and the results illustrate how this method can be used to evaluate the effect of the junior faculty development program on subsequent academic success. These results indicate that, compared with non-NCLAM participants who were also successful and still on the UCSD faculty at the time of the data analyses, there was a consistent trend of greater academic success for the NCLAM participants, with the greatest (and only statistically significant) effect on the category of leadership and professional activities.
In addition to improving faculty retention, a robust faculty development program that emphasizes the acquisition and refinement of academic skills should, if successful, augment academic success for participants. Given the variety of job descriptions and pathways to success in academic medicine, it is somewhat difficult to quantify academic success. In a recent publication on minority faculty resilience, Cora-Bramble and coworkers10 used measures of publications, grants, and promotion as indicators of academic productivity in 74 racial and ethnic minority faculty in academic medicine. They reported that two of seven subscales of a personal resilience questionnaire (“Flexible: Social” and “Positive: The World”) were correlated with academic productivity. Reed and coworkers used measures of peer-reviewed publications, dates of academic promotion, and number of leadership appointments to evaluate gender differences in academic productivity and leadership for 25 women clinical faculty compared with 50 male physicians matched by appointment date and career track.11 They reported that publication rates of women increased and actually exceeded those of men later in their careers, but women held fewer leadership positions.
To evaluate academic success in this study, we developed an innovative method to quantify academic accomplishments in five different areas recognized in the regular academic review process at UCSD. On the basis of the previous survival analyses and recognizing that NCLAM participants were more likely to remain at UCSD (and in academic medicine), we decided to identify a matched comparison group of successful non-NCLAM participants who also remained at UCSD beyond the period for promotion after the eight-year probationary period for assistant professors. The results of this analysis indicate a consistent trend in favor of the NCLAM participants in each of the measured areas of academic success. However, only in the leadership and professional activities category did the difference between the NCLAM and matched non-NCLAM faculty reach statistical significance. Although the other categories of success were not statistically significant with this small sample of 29 pairs, we believe that the consistent trend in favor of the NCLAM participants in all five categories is encouraging and may suggest that this type of approach to quantify academic success is deserving of further study.
Selecting a comparison group of successful nonparticipants, who were also promoted successfully to the associate rank, should bias the results against the benefits of the faculty development program and make it harder to identify the added value of an early career faculty development program. Highly qualified faculty are likely to succeed and achieve promotion regardless of whether they participate in an organized faculty development program. Many such faculty have access to their own individual mentors and advisors who can facilitate their academic success. In fact, one might argue that an organized program might have its greatest effect on those junior faculty without strong individual support networks of mentors and advisors.
It is interesting to note that the one category of academic success that was significantly different for the faculty development program participants was leadership and professional activities. This is the one area in which an institutionally based mentoring program might, in fact, exert more influence than individual, department-based mentors. At UCSD, the NCLAM program includes specific information about the overall organization, provides direct contact with institutional leaders, and encourages participants to become involved in institutional governance and local and national professional service.
Although we believe that the current study confirms and extends the results of our previous publication4 regarding the benefits of the NCLAM faculty development program on faculty retention and addresses several limitations of the previous analyses, we acknowledge some limitations to this work. Without a randomized design, there is likely to be a selection bias for NCLAM participants who may differ from nonparticipants in ways not controlled for in these analyses. Although open to all junior faculty, the NCLAM program has a limited number of available positions each year. NCLAM participants choose to apply and are selected by the NCLAM Leadership Council.
Another limitation is related to the measures of academic success developed for this study based on available information in faculty academic review files and categories that we believe represent identifiable dimensions of academic success. We viewed this as an initial effort to try to quantitate success. Clearly, additional work is needed to evaluate the validity and reliability of this approach. Also, true measures of faculty “success,” both for individuals and for institutions, may involve more than just those reflected in these particular categories.
In summary, we believe that the innovative methods developed and used in this study provide useful models for better quantifying and evaluating factors that influence both retention and academic success for faculty in academic medicine. In addition, the results of the analyses conducted using these methods further document the potential benefits of an organized junior faculty development program on retention and subsequent promotion. The trends evident in these data also suggest a positive influence on leadership and academic success.
Funding/Support: The work in this manuscript was partially supported by the U.S. Public Health Service, Office of Women’s Health (#282-98-0051), UC San Diego Comprehensive Research Center in Health Disparities (CRCHD), National Institute of Minority Health and Health Disparities (#P60 MD000220), and the UC San Diego Hispanic Center of Excellence (HRSA Grant #D34HP18954).
Other disclosures: None.
Ethical approval: All analyses were performed under approval by the University of California, San Diego human research protections program.
Disclaimer: The contents of this manuscript are solely the responsibility of the authors and do not represent the official views of the Office of Women’s Health, Health Resources Services Administration, or the National Institutes of Health.