Dyrbye, Liselotte N., MD, MHPE; Wittlin, Natalie M., MS; Hardeman, Rachel R., PhD, MPH; Yeazel, Mark, MD, MPH; Herrin, Jeph, PhD; Dovidio, John F., PhD; Burke, Sara E., PhD; Cunningham, Brooke, MD, PhD; Phelan, Sean M., PhD, MPH; Shanafelt, Tait D., MD; van Ryn, Michelle, PhD, MPH

Medical student depression is an important ethical concern for medical educators. Beyond the associated individual suffering it causes, depression among medical students can lead to a number of professional and personal consequences, including declines in academic performance, thoughts of dropping out of medical school, and suicide.^1–3 Authors of a recent systematic review and meta-analysis estimated the prevalence of depression or depressive symptoms among medical students to be 27.2%, suggesting that depression is 2.2 to 5.2 times more common among medical students than similarly aged individuals in the general U.S. population.⁴ This risk occurs despite matriculating medical students entering medical school with a lower prevalence of depressive symptoms than similarly aged college graduates who pursued other careers.⁵

Given the high prevalence of depression symptoms,⁴ medical educators and others have advocated primary prevention strategies,^{1 , 6} and several screening instruments are now available.^{7 , 8} Although the Liaison Committee for Medical Education (LCME) requires medical schools to have a student wellness program available for all students,⁹ a targeted approach that provides additional support to the students who have the highest risk for developing depression may allow for both better support of students at greatest risk and optimal resource allocation. Previous longitudinal studies evaluating depression among U.S. medical students have been single-institution studies and have primarily focused on the first year of medical school.^10–16 Few studies have followed students into year two^{17 , 18}; and only two studies, both conducted over two decades ago, longitudinally assessed students at several time points across all four years.^{1 , 19} To our knowledge, no previous study has explored the relationship between the risk of developing depression symptoms during medical school and individual characteristics at the time of matriculation and medical school factors.

To address this gap in the literature, we used the longitudinal Medical Student Cognitive Habits and Growth Evaluation Study (CHANGES) database to conduct a comprehensive evaluation of individual and medical school factors associated with an increased risk of developing depression symptoms at the end of year four (Y4) of medical school. Our aim was to develop a prognostic index that stratified the risk of developing depression symptoms (Depression-PI).

Method

The design, setting, and data collection methods for CHANGES have been previously reported.^20–26 Briefly, in 2011, 4,732 of 5,823 invited first-year medical students (81%) attending a stratified random sample of 49 U.S. medical schools consented to participate in the longitudinal CHANGES study and completed the year one (Y1) online questionnaire. The investigators invited these 4,732 medical students via e-mail to complete a follow-up online questionnaire at the end of Y4 of medical school. Students received an incentive for each questionnaire they completed. University of Minnesota and the Mayo Clinic institutional review boards approved the study.

Survey measures

The surveys included items assessing demographic characteristics (age, gender, race, ethnicity, relationship status, parental status [whether students have children], place of birth [whether students are U.S. born]) that are known to be associated with medical students’ mental health,^{2 , 20 , 27–29} as well as their socioeconomic status (parental income and debt from student loans). We did not collect information on the amount of accrued undergraduate or medical school debt, or on income from working while in school. We dichotomized age on the basis of the median age of U.S. medical students.³⁰ We used established scales (described immediately below) to measure students’ depression symptoms, stress, coping behaviors, and social support.

Depression symptoms, stress, coping behaviors, and social support.

We assessed depression symptoms at Y1 and Y4 using the Patient-Reported Outcome Measurement Information System (PROMIS) depression short form 4a, an instrument developed and validated by the National Institutes of Health.³¹ The area under the curve for detecting individuals with major depression, as diagnosed by the Patient Health Questionnaire-9, is 0.899.³² We identified participants as having depression symptoms if they had PROMIS depression scores of ≥ 8 (sensitivity 83.1%, specificity 84.3%).³²

We measured stress at Y1 using the four-item Perceived Stress Scale (PSS), an instrument with acceptable internal consistency and criterion validity that assesses the amount of stress the individual has felt in the past month.^{33 , 34} We considered participants with a PSS score ≥ ½ standard deviation [SD] above the group mean (a clinically meaningful effect size³⁵) to have high stress.

We measured coping behaviors at Y1 using a modified version of the Brief COPE Inventory, one of the most commonly used measures.³⁶ Using data from our responders, we conducted factor analysis and identified three subscales: (1) social coping (turning to others for help), (2) negative coping (self-blaming and denial), and (3) positive coping (actively dealing with the problem). We considered low scores on the social coping and positive coping subscale and high scores on the negative coping scale indicative of maladaptive coping behaviors. We defined low social coping and low positive coping behaviors as having a score ½ SD or more (a clinically meaningful effect size³⁵) below, respectively, the mean social coping score and the mean positive coping score. Similarly, we defined high negative coping as having a score ½ SD or more³⁵ above the mean negative coping score. Psychometric properties of the Brief COPE, including factor structure and internal reliability, have been published.³⁶

We measured social support at Y1 through the Medical Outcomes Study Social Support Measure.³⁷ The measure contains 19 items; higher scores indicate better social support. Details regarding the conceptual framework used to develop the construct of functional support, the multistep validation process, the convergent and discriminant validity of items, the confirmatory factor analysis, and the internal consistency reliability have been reported and are acceptable.³⁸

School-level variables.

We obtained the following medical school characteristics from public datasets: size of first-year class, in-state tuition, U.S. News and World Report (USNWR) research ranking, and median Medical College Admission Test (MCAT) score of admitted students. We categorized class size as small, medium, or large based on tertiles determined by national data for the number of matriculates.³⁹ We also categorized in-state tuition as low, medium, or high based on tertiles determined by national data.⁴⁰ We considered medical schools to be research intensive if they were ranked in the top 50 medical schools for research by USNWR.⁴¹ We dichotomized whether a student attended a medical school with a median MCAT score above or below the median for the 49 CHANGES schools.

Statistical analysis and development of the Depression PI

We used descriptive statistics to characterize the sample and explore bivariate relationships among demographic characteristics, school-related factors, and baseline depression symptoms, stress, coping behaviors, and social support. All tests were two tailed, and we defined statistical significance as a P value of less than .05.

Following the univariate analysis, we randomly divided the full dataset into discovery and replication (internal-validation) datasets. Using a random sample procedure, we allocated approximately two-thirds of the students to discovery and a third to the replication datasets, resulting in 2,455 students in the discovery and 1,288 students in the replication dataset. We performed two multivariate analyses in the discovery dataset to identify factors independently associated with depression symptoms at Y4. The first included demographic characteristics, school-related factors, as well as Y1 depression symptoms, stress, coping behaviors, and social support. The second multivariate analysis did not include baseline depression symptoms because such information may not be readily available or obtainable by medical schools. We verified the robustness of the multivariable models using bootstrapping techniques, and we included the variables present in 70% or more of the 1,000 bootstrap models in the prognostic index.

To account for differences in the magnitude of association between the individual independent factors and Y4 depression symptoms, we assigned a weighted risk score to each factor based on ranges of their corresponding odds ratios (ORs). We calculated the total Depression-PI risk score by summing the ratings of individual factors. To identify risk groups, we entered all potential risk group definitions into a univariate logistic model for high depression at Y4. The risk group definition that resulted in the smallest value of log-likelihood change and was statistically significant was chosen to define the risk groups. We calculated c statistics (or area under the receiver operating characteristic curve, c) to further evaluate discriminatory value of the prognostic index (c = 1 indicates perfect discrimination; c = 0.5 indicates equivalence to chance).^{42 , 43} As a measure of how well a model predicts the probability of the outcome, a c statistic of ≥ 0.7 is a commonly used minimum threshold for being able to discriminate at the individual level.^{42 , 43} We subsequently confirmed the ability of the prognostic index to stratify the risk of depression symptoms in the replication cohort. We developed the Depression-PI with and without baseline depression symptoms. We used SAS (version 9.4; Cary, North Carolina) for statistical analysis.

Results

Of the 4,732 medical students who completed the Y1 survey (55% of all first-year medical students at the 49 medical schools), 3,743 (79%) completed the Y4 survey. Baseline characteristics of participants have been previously reported,^20–26 and the sample is similar to the broader population of medical students who matriculated in 2010.^{44 , 45} The demographic characteristics of those who responded to both the Y1 and Y4 survey were similar to the Y1 students who did not respond to the Y4 survey with respect to sex, relationship status, and parental status. Compared with Y4 nonresponders, slightly more responders to both surveys were less than 24 years old (73.8% vs. 69.5%), white (65.8% vs. 55.6%), and U.S. born (84.8% vs. 81.7%), and fewer responders to both surveys were Hispanic (5.8% vs. 8.5%).

Overall, of the 3,743 medical students who completed both the YI and Y4 questionnaires, 1,167 (31.2%) had depression symptoms at Y4. Univariate analyses of individual and medical school factors associated with depression symptoms at Y4 are shown in Tables 1 and 2. Compared with students without depression symptoms at Y4, students with depression symptoms were more likely to be nonwhite, single, born outside the United States, from a family with a median household income below the U.S. median, and free of student loan debt. They were also more likely to have higher stress, low social support, and rely less on social and positive coping and more on negative coping behaviors at Y1. Tuition cost was the only school-related factor that differed between students with and without depression symptoms at Y4. Students attending a school with middle-tertile tuition were more likely to develop depression symptoms than those attending a school with low tuition.

Table 1

Table 2

After adjusting for demographic characteristics and school-related factors, as well as for baseline depression, stress, coping behaviors, and social support, we observed an increased risk of depression symptoms at Y4 among students who were 24 years of age or older (OR 1.34; 95% confidence interval [CI] 1.08–1.66), nonwhite (OR 1.49; 95% CI 1.21–1.84), and non-Hispanic/Latino (OR 1.83; 95% CI 1.17–2.86). Students with low social support (OR 1.55; 95% CI 1.25–1.92), high stress (OR 1.49; 95% CI 1.19–1.86), high negative coping (OR 1.60; 95% CI 1.30–1.97), and depression symptoms (OR 3.08; 95% CI 2.46–3.84) at Y1 were also at increased risk of developing depression symptoms at Y4. Students who attended a medical school with middle-tertile tuition cost were at higher risk of developing depressive symptoms than students who attended a medical school with lower tuition costs (OR 1.32; 95% CI 1.08–1.62).

When we re-ran the analysis without controlling for baseline depression symptoms, the model changed slightly. Both birth outside the United States (OR 1.36; 95% CI 1.06–1.82) and low positive coping (OR 1.24; 95% CI 1.01–1.54) became statistically significant predictors. The other predictors in the original model (i.e., age, race, ethnicity, tuition, stress, negative coping, and social support) remained significant.

Depression-PI

We next evaluated whether the variables independently associated with developing depression symptoms by Y4 could be combined to predict which students would develop depression symptoms during medical school. Using bootstrap methodologies, we verified all of the independent factors from the multivariate model that included baseline depression symptoms (i.e.., the factors that appeared in > 70% of the 1,000 bootstrapped simulations), and we assigned these factors a risk score (Table 3). We based the weighting score for each factor on the integer value of its corresponding OR (i.e., 1 point for OR 1.1–1.9; 2 points for OR 2.0–2.9; and so on). The overall Depression-PI score was the sum of the weighted risk scores of the 7 individual factors (range 0–10).

Table 3

The relationship between Depression-PI score and prevalence of Y4 depression symptoms among students in the discovery cohort is shown in Figure 1. We noted that stepwise increases in the prevalence of depression symptoms at Y4 occurred as Depression-PI score increased. We determined four risk categories according to predefined criteria (see Method, above): low risk of developing depression symptoms (score 0–1), intermediate risk of developing depression symptoms (score 2–3), high risk of developing depression symptoms (score 4–5), and very high risk of developing depression symptoms (score 6 or higher). Table 4 shows that the proposed risk categories stratify prevalence of Y4 depression symptoms from 15.4% (low risk) to 62.6% (very high risk) (P < .0001) with c = 0.71 in the discovery cohort. Compared with students in the low risk category (i.e., Depression-PI score of 1 or less), students in the intermediate risk category had 75% increased odds of developing depression symptoms by Y4 while those students in the high risk category had nearly fourfold increased odds of developing depression symptoms by Y4, and those in the very high risk category had ninefold increased odds of developing depression symptoms by Y4.

Table 4

Figure 1

Replication dataset to validate the model

The utility of the index to stratify the risk of Y4 depression symptoms developed in the discovery cohort was subsequently evaluated in the students in the replication cohort. Among students in the replication dataset, the prevalence of Y4 depression symptoms of the four risk groups was, respectively, 14.9%, 24.1%, 44.4%, and 64.2% (P < .0001; Table 4). The OR for risk associated with each incremental increase in risk category in the replication cohort was similar to that of each risk category in the discovery cohort. With respect to discrimination, c = 0.72.

Prognostic index without depression symptoms at baseline included

Results from the multivariate logistic analysis using bootstrapping methods can be found in Supplemental Digital Table 1 at http://links.lww.com/ACADMED/A593. Supplemental Digital Table 2 shows the four risk categories, distribution of scores, and prevalence of Y4 depression symptoms. The proposed risk categories stratify the prevalence of Y4 depression rates from 20.2% (low risk) to 62.9% (very high risk) with c = 0.65 in the discovery cohort, and from 17.7% (low risk) to 58.5% (very high risk) with c = 0.67 in the replication cohort.

Discussion

As mentioned, medical students with depression are at increased risk for adverse professional and personal consequences, including declines in academic performance, thoughts of dropping out of medical school, and suicide.^1–3 We found a high prevalence (31.2%; 1,167/3,743) of depression symptoms among fourth-year medical students, consistent with the rate reported in the literature.⁴ For medical students in this prospective longitudinal cohort study, demographics (e.g., age, race, ethnicity), tuition, and depression symptoms, stress, coping behaviors, and social support at baseline were independently associated with the risk of developing depression symptoms at Y4. These findings are consistent with findings from previous cross-sectional and single-institution longitudinal studies.^{1 , 13 , 14 , 16 , 17 , 19 , 27 , 46–51} Information on most of these factors is available or obtainable for all students, and each factor has independent prognostic value. We determined how these factors can be combined into a Depression-PI that stratifies risk of developing depression symptoms by Y4.

We demonstrated in the discovery dataset—and further confirmed in the replication dataset—that the Depression-PI defines four prognostic subgroups at low, intermediate, high, and very high risk, therefore providing prognostic information regarding risk of depression symptoms by Y4 of medical school. When baseline depression symptoms were included in the Depression-PI, the c statistic was 0.71 and 0.72, signifying potential prognostic utility at the individual level (as this typically requires a minimum threshold level above 0.70).^{42 , 43} We identified high risk and very high risk groups of medical students who had four- to ninefold increased odds of demonstrating depression symptoms at Y4. These two highest-risk groups represented approximately a third of the students in the prospective longitudinal cohort.

When baseline depression symptoms were not included in the Depression-PI, the c statistic is slightly below the 0.7 threshold. Although the model without baseline depression could be used to stratify risk in a group of students, it does not have utility at the individual student level.

Importantly, the Depression-PI has the potential to support medical school efforts to promote student wellness. Student wellness programming should be offered to all students. Doing so is an LCME requirement.⁹ More importantly, wellness initiatives help to fulfill the medical education community’s moral imperative to address medical student depression, especially given its high prevalence. A risk-stratified approach, however, allows delivery of core material to all students and additional support to those at greatest risk, an approach that could optimize support for students despite finite resources. Such tailoring does not undermine the need for or value of wellness programming offered to all students. For example, students in the intermediate risk group may benefit from added support, whereas students in the high risk groups may warrant allocation of further resources including individualized support, active screening, and early intervention. Implementing interventions in a way that protects confidentiality and avoids stigmatizing individuals in the higher-risk groups is and will continue to be vital. How best to engage students in a tailored primary prevention strategy, and whether doing so leads to less suffering and depression and to fewer adverse consequences, warrants study. These efforts should occur in conjunction with intentional steps to improve the work and learning environment and reduce drivers of distress. We do not recommend that the Depression-PI be used in any capacity to screen applicants or violate privacy.

To our knowledge, this is the largest longitudinal study to explore predictors of development of depression symptoms among U.S. medical students. Of the students in our cohort, those who were nonwhite and non-Hispanic/Latino were at increased odds of developing depression symptoms, even after controlling for baseline depression. The 2009–2012 National Health and Nutritional Examination Survey found non-Hispanic blacks to have higher rates of severe depression than non-Hispanic whites.⁵² Previous studies of medical students suggest that the mental health of medical students from racial and ethnic minorities may depend on whether or not their race or ethnicity has negatively affected their experience.⁵³ Similarly, other studies have found that students from racial and ethnic minorities experience more microaggressions that they attribute to their race.^54–56 Together, these studies suggest that the higher prevalence of depression symptoms among this subgroup of students is likely driven by factors within the learning environment rather than individual traits. Medical schools need to do more to improve the learning environment for nonwhite students.

Our findings—that negative coping behaviors and low social support increase the risk of subsequently developing depression symptoms—are congruent with findings from cross-sectional^46–48 and one-year longitudinal studies of medical students.^{13 , 49–51} Independent of coping behavior, we also found that high stress levels at baseline predicted subsequent depression symptoms. Stress scores have also been found to be associated with global measures of distress in cross-sectional studies,⁵⁷ and with depression^{13 , 14} and anxiety¹³ in one-year longitudinal studies of first-year medical students. In aggregate, these studies suggest that intervention studies focused on decreasing reliance on negative coping behaviors, teaching strategies to reduce stress, and promoting social support among medical students should be pursued.

Among the medical school factors we explored, only tuition was independently associated with increased risk of developing depression symptoms; that is, those who attended a medical school with middle-tertile tuition were at higher risk than students who attended a medical school with lower tuition costs. This finding persisted after adjusting for socioeconomic background. Importantly, we did not collect information on accrued undergraduate and medical school debt directly, nor on income from working while in medical school—both of which have been shown in previous studies of medical trainees to be associated with poorer mental health.^{50 , 58 , 59} Other studies suggest that grading structure,^60–62 curricular structure and learning communities,⁶³ and harassment/belittlement, as well as poor role-modeling behaviors by faculty,⁵¹ relate to medical student mental health.

Our study has a number of limitations. First, although the sample was drawn from a random stratified sample of students from 49 U.S. medical schools, we do not know how well the experience of participants represents the experience of U.S. medical students. We also detected some minor demographic differences between students who responded at both time points and students who responded only to the first survey. We did, however, have a robust response rate, and the demographic characteristics of participants were similar to those of all U.S. medical students who matriculated in the same year, suggesting that our results likely generalize to medical students overall.^{44 , 45} Second, we used the PROMIS depression short form 4a as our screening instrument for depression symptoms, and we applied a cutoff score of ≥ 8 (sensitivity 83.1%, specificity 84.3%).³² If we had used a higher cutoff score, our specificity would have improved but our sensitivity would have declined. Third, we depended on self-reported characteristics and included a limited set of individual and school characteristics. Multiple social, psychological, and biological factors contribute to depression,⁶⁴ and we did not include concomitant anxiety disorders. Fourth, we assessed depression symptoms at two time points (Y1 and Y4) and know that depression symptoms may come and go at other points during the four-year interval. Fifth, we recognize that further validation of the Depression-PI in other datasets is warranted.

Conclusions

The rate of depression symptoms in this longitudinal, national cohort of U.S. medical students at the end of Y4 was 31.2%. Being older, nonwhite, or non-Hispanic/Latino; relying on negative coping behaviors; having high stress, low social support, and depression symptoms at Y1; and being at a medical school with middle-tertile tuition were strongly associated with an increased risk of developing depression symptoms during medical school. The Depression-PI is the first prognostic model to incorporate multiple Y1 factors to stratify the risk of Y4 depression symptoms among medical students. The Depression-PI score can identify students at highest risk for developing depression symptoms and may be useful to medical schools interested in developing a tiered approach to prevention and support for medical students in or at risk for distress.

References

Supplemental Digital Content

• ACADMED_2018_08_28_DYRBYE_AcadMed-D-17-01941_SDC1.pdf; [PDF] (104 KB)