Secondary Logo

Journal Logo

Advanced Search
Research Reports

A Prognostic Index to Identify the Risk of Developing Depression Symptoms Among U.S. Medical Students Derived From a National, Four-Year Longitudinal Study

Dyrbye, Liselotte N. MD, MHPE; Wittlin, Natalie M. MS; Hardeman, Rachel R. PhD, MPH; Yeazel, Mark MD, MPH; Herrin, Jeph PhD; Dovidio, John F. PhD; Burke, Sara E. PhD; Cunningham, Brooke MD, PhD; Phelan, Sean M. PhD, MPH; Shanafelt, Tait D. MD; van Ryn, Michelle PhD, MPH

Author Information

L.N. Dyrbye is professor of medicine and medical education, Program on Physician Well-Being, Department of Medicine, Mayo Clinic, Rochester, Minnesota; ORCID: https://orcid.org/0000-0002-7820-704X.

N.M. Wittlin is PhD student, Department of Psychology, Yale University, New Haven, Connecticut; ORCID: https://orcid.org/0000-0002-0858-3576.

R.R. Hardeman is assistant professor, Division of Health Policy and Management, University of Minnesota, School of Public Health, Minneapolis, Minnesota; ORCID: https://orcid.org/0000-0003-3913-5933.

M. Yeazel is associate professor, Department of Family Medicine and Community Health, University of Minnesota Medical School, Minneapolis, Minnesota.

J. Herrin is assistant professor of medicine, Section of Cardiovascular Medicine, Yale University School of Medicine, New Haven, Connecticut; ORCID: https://orcid.org/0000-0002-3671-3622.

J.F. Dovidio is Carl Iver Hovland Professor of Psychology and professor of public health, Department of Psychology, Yale University, New Haven, Connecticut; ORCID: https://orcid.org/0000-0002-6110-8344.

S.E. Burke is assistant professor, Department of Psychology, Syracuse University, Syracuse, New York; ORCID: https://orcid.org/0000-0002-6952-924X.

B. Cunningham is assistant professor of health disparities, Department of Family Medicine and Community Health, University of Minnesota Medical School, Minneapolis, Minnesota; ORCID: https://orcid.org/0000-0002-3205-5538.

S.M. Phelan is associate professor of health services research, Division of Health Care Policy and Research, Mayo Clinic, Rochester, Minnesota; ORCID: https://orcid.org/0000-0003-2091-6297.

T.D. Shanafelt is Jeanie and Stewart Ritchie Professor of Medicine and associate dean, Stanford School of Medicine, Stanford, California; ORCID: https://orcid.org/0000-0002-7106-5202.

M. van Ryn is Grace Phelps Distinguished Professor, Oregon Health & Science University School of Nursing, Portland, Oregon; ORCID: https://orcid.org/0000-0002-4258-7319.

Editor’s Note: An Invited Commentary on this article by G.W. Gengoux and L.W. Roberts appears on pages 162–165.

Funding/Support: National Heart, Lung and Blood Institute, National Institutes of Health, award number R01-HL085631. Dr. Hardeman was further supported by the National Heart, Lung and Blood Institute through a Research Supplement to Promote Diversity in Health-Related Research 3R01HL085631-S2. Dr. Dyrbye is further supported by the Department of Medicine Program on Physician Well-Being, Mayo Clinic.

Other disclosures: None reported.

Ethical approval: University of Minnesota and the Mayo Clinic Institutional Review Boards approved the study.

Supplemental digital content for this article is available at http://links.lww.com/ACADMED/A593.

Correspondence should be addressed to Liselotte N. Dyrbye, 200 First St. SW, Rochester, MN 55905; telephone: (507) 284-2511; e-mail: [email protected]; Twitter: @dyrbye.

doi: 10.1097/ACM.0000000000002437

Abstract

Medical student depression is an important ethical concern for medical educators. Beyond the associated individual suffering it causes, depression among medical students can lead to a number of professional and personal consequences, including declines in academic performance, thoughts of dropping out of medical school, and suicide.1–3 Authors of a recent systematic review and meta-analysis estimated the prevalence of depression or depressive symptoms among medical students to be 27.2%, suggesting that depression is 2.2 to 5.2 times more common among medical students than similarly aged individuals in the general U.S. population.4 This risk occurs despite matriculating medical students entering medical school with a lower prevalence of depressive symptoms than similarly aged college graduates who pursued other careers.5

Given the high prevalence of depression symptoms,4 medical educators and others have advocated primary prevention strategies,1,6 and several screening instruments are now available.7,8 Although the Liaison Committee for Medical Education (LCME) requires medical schools to have a student wellness program available for all students,9 a targeted approach that provides additional support to the students who have the highest risk for developing depression may allow for both better support of students at greatest risk and optimal resource allocation. Previous longitudinal studies evaluating depression among U.S. medical students have been single-institution studies and have primarily focused on the first year of medical school.10–16 Few studies have followed students into year two17,18; and only two studies, both conducted over two decades ago, longitudinally assessed students at several time points across all four years.1,19 To our knowledge, no previous study has explored the relationship between the risk of developing depression symptoms during medical school and individual characteristics at the time of matriculation and medical school factors.

To address this gap in the literature, we used the longitudinal Medical Student Cognitive Habits and Growth Evaluation Study (CHANGES) database to conduct a comprehensive evaluation of individual and medical school factors associated with an increased risk of developing depression symptoms at the end of year four (Y4) of medical school. Our aim was to develop a prognostic index that stratified the risk of developing depression symptoms (Depression-PI).

Method

The design, setting, and data collection methods for CHANGES have been previously reported.20–26 Briefly, in 2011, 4,732 of 5,823 invited first-year medical students (81%) attending a stratified random sample of 49 U.S. medical schools consented to participate in the longitudinal CHANGES study and completed the year one (Y1) online questionnaire. The investigators invited these 4,732 medical students via e-mail to complete a follow-up online questionnaire at the end of Y4 of medical school. Students received an incentive for each questionnaire they completed. University of Minnesota and the Mayo Clinic institutional review boards approved the study.

Survey measures

The surveys included items assessing demographic characteristics (age, gender, race, ethnicity, relationship status, parental status [whether students have children], place of birth [whether students are U.S. born]) that are known to be associated with medical students’ mental health,2,20,27–29 as well as their socioeconomic status (parental income and debt from student loans). We did not collect information on the amount of accrued undergraduate or medical school debt, or on income from working while in school. We dichotomized age on the basis of the median age of U.S. medical students.30 We used established scales (described immediately below) to measure students’ depression symptoms, stress, coping behaviors, and social support.

Depression symptoms, stress, coping behaviors, and social support.

We assessed depression symptoms at Y1 and Y4 using the Patient-Reported Outcome Measurement Information System (PROMIS) depression short form 4a, an instrument developed and validated by the National Institutes of Health.31 The area under the curve for detecting individuals with major depression, as diagnosed by the Patient Health Questionnaire-9, is 0.899.32 We identified participants as having depression symptoms if they had PROMIS depression scores of ≥ 8 (sensitivity 83.1%, specificity 84.3%).32

We measured stress at Y1 using the four-item Perceived Stress Scale (PSS), an instrument with acceptable internal consistency and criterion validity that assesses the amount of stress the individual has felt in the past month.33,34 We considered participants with a PSS score ≥ ½ standard deviation [SD] above the group mean (a clinically meaningful effect size35) to have high stress.

We measured coping behaviors at Y1 using a modified version of the Brief COPE Inventory, one of the most commonly used measures.36 Using data from our responders, we conducted factor analysis and identified three subscales: (1) social coping (turning to others for help), (2) negative coping (self-blaming and denial), and (3) positive coping (actively dealing with the problem). We considered low scores on the social coping and positive coping subscale and high scores on the negative coping scale indicative of maladaptive coping behaviors. We defined low social coping and low positive coping behaviors as having a score ½ SD or more (a clinically meaningful effect size35) below, respectively, the mean social coping score and the mean positive coping score. Similarly, we defined high negative coping as having a score ½ SD or more35 above the mean negative coping score. Psychometric properties of the Brief COPE, including factor structure and internal reliability, have been published.36

We measured social support at Y1 through the Medical Outcomes Study Social Support Measure.37 The measure contains 19 items; higher scores indicate better social support. Details regarding the conceptual framework used to develop the construct of functional support, the multistep validation process, the convergent and discriminant validity of items, the confirmatory factor analysis, and the internal consistency reliability have been reported and are acceptable.38

School-level variables.

We obtained the following medical school characteristics from public datasets: size of first-year class, in-state tuition, U.S. News and World Report (USNWR) research ranking, and median Medical College Admission Test (MCAT) score of admitted students. We categorized class size as small, medium, or large based on tertiles determined by national data for the number of matriculates.39 We also categorized in-state tuition as low, medium, or high based on tertiles determined by national data.40 We considered medical schools to be research intensive if they were ranked in the top 50 medical schools for research by USNWR.41 We dichotomized whether a student attended a medical school with a median MCAT score above or below the median for the 49 CHANGES schools.

Statistical analysis and development of the Depression PI

We used descriptive statistics to characterize the sample and explore bivariate relationships among demographic characteristics, school-related factors, and baseline depression symptoms, stress, coping behaviors, and social support. All tests were two tailed, and we defined statistical significance as a P value of less than .05.

Following the univariate analysis, we randomly divided the full dataset into discovery and replication (internal-validation) datasets. Using a random sample procedure, we allocated approximately two-thirds of the students to discovery and a third to the replication datasets, resulting in 2,455 students in the discovery and 1,288 students in the replication dataset. We performed two multivariate analyses in the discovery dataset to identify factors independently associated with depression symptoms at Y4. The first included demographic characteristics, school-related factors, as well as Y1 depression symptoms, stress, coping behaviors, and social support. The second multivariate analysis did not include baseline depression symptoms because such information may not be readily available or obtainable by medical schools. We verified the robustness of the multivariable models using bootstrapping techniques, and we included the variables present in 70% or more of the 1,000 bootstrap models in the prognostic index.

To account for differences in the magnitude of association between the individual independent factors and Y4 depression symptoms, we assigned a weighted risk score to each factor based on ranges of their corresponding odds ratios (ORs). We calculated the total Depression-PI risk score by summing the ratings of individual factors. To identify risk groups, we entered all potential risk group definitions into a univariate logistic model for high depression at Y4. The risk group definition that resulted in the smallest value of log-likelihood change and was statistically significant was chosen to define the risk groups. We calculated c statistics (or area under the receiver operating characteristic curve, c) to further evaluate discriminatory value of the prognostic index (c = 1 indicates perfect discrimination; c = 0.5 indicates equivalence to chance).42,43 As a measure of how well a model predicts the probability of the outcome, a c statistic of ≥ 0.7 is a commonly used minimum threshold for being able to discriminate at the individual level.42,43 We subsequently confirmed the ability of the prognostic index to stratify the risk of depression symptoms in the replication cohort. We developed the Depression-PI with and without baseline depression symptoms. We used SAS (version 9.4; Cary, North Carolina) for statistical analysis.

Results

Of the 4,732 medical students who completed the Y1 survey (55% of all first-year medical students at the 49 medical schools), 3,743 (79%) completed the Y4 survey. Baseline characteristics of participants have been previously reported,20–26 and the sample is similar to the broader population of medical students who matriculated in 2010.44,45 The demographic characteristics of those who responded to both the Y1 and Y4 survey were similar to the Y1 students who did not respond to the Y4 survey with respect to sex, relationship status, and parental status. Compared with Y4 nonresponders, slightly more responders to both surveys were less than 24 years old (73.8% vs. 69.5%), white (65.8% vs. 55.6%), and U.S. born (84.8% vs. 81.7%), and fewer responders to both surveys were Hispanic (5.8% vs. 8.5%).

Overall, of the 3,743 medical students who completed both the YI and Y4 questionnaires, 1,167 (31.2%) had depression symptoms at Y4. Univariate analyses of individual and medical school factors associated with depression symptoms at Y4 are shown in Tables 1 and 2. Compared with students without depression symptoms at Y4, students with depression symptoms were more likely to be nonwhite, single, born outside the United States, from a family with a median household income below the U.S. median, and free of student loan debt. They were also more likely to have higher stress, low social support, and rely less on social and positive coping and more on negative coping behaviors at Y1. Tuition cost was the only school-related factor that differed between students with and without depression symptoms at Y4. Students attending a school with middle-tertile tuition were more likely to develop depression symptoms than those attending a school with low tuition.

Table 1
Table 1:
Baseline Personal Characteristics Associated With Depression Symptoms at End of Y4 Among 3,743 U.S. Medical Students Who Completed Surveys in 2010 and 2014
Table 2
Table 2:
School-Related Factors Associated With Depression Symptoms at End of Y4 Among 3,743 U.S. Medical Students Who Completed Surveys in 2010 and 2014

After adjusting for demographic characteristics and school-related factors, as well as for baseline depression, stress, coping behaviors, and social support, we observed an increased risk of depression symptoms at Y4 among students who were 24 years of age or older (OR 1.34; 95% confidence interval [CI] 1.08–1.66), nonwhite (OR 1.49; 95% CI 1.21–1.84), and non-Hispanic/Latino (OR 1.83; 95% CI 1.17–2.86). Students with low social support (OR 1.55; 95% CI 1.25–1.92), high stress (OR 1.49; 95% CI 1.19–1.86), high negative coping (OR 1.60; 95% CI 1.30–1.97), and depression symptoms (OR 3.08; 95% CI 2.46–3.84) at Y1 were also at increased risk of developing depression symptoms at Y4. Students who attended a medical school with middle-tertile tuition cost were at higher risk of developing depressive symptoms than students who attended a medical school with lower tuition costs (OR 1.32; 95% CI 1.08–1.62).

When we re-ran the analysis without controlling for baseline depression symptoms, the model changed slightly. Both birth outside the United States (OR 1.36; 95% CI 1.06–1.82) and low positive coping (OR 1.24; 95% CI 1.01–1.54) became statistically significant predictors. The other predictors in the original model (i.e., age, race, ethnicity, tuition, stress, negative coping, and social support) remained significant.

Depression-PI

We next evaluated whether the variables independently associated with developing depression symptoms by Y4 could be combined to predict which students would develop depression symptoms during medical school. Using bootstrap methodologies, we verified all of the independent factors from the multivariate model that included baseline depression symptoms (i.e.., the factors that appeared in > 70% of the 1,000 bootstrapped simulations), and we assigned these factors a risk score (Table 3). We based the weighting score for each factor on the integer value of its corresponding OR (i.e., 1 point for OR 1.1–1.9; 2 points for OR 2.0–2.9; and so on). The overall Depression-PI score was the sum of the weighted risk scores of the 7 individual factors (range 0–10).

Table 3
Table 3:
Factors Independently Associated With End of Y4 (2014) Depression Symptoms on Multivariate Logistic Analysis Using Bootstrap Methods and Risk Scores Based on 2,455 U.S. Medical Students in the Discovery Cohort

The relationship between Depression-PI score and prevalence of Y4 depression symptoms among students in the discovery cohort is shown in Figure 1. We noted that stepwise increases in the prevalence of depression symptoms at Y4 occurred as Depression-PI score increased. We determined four risk categories according to predefined criteria (see Method, above): low risk of developing depression symptoms (score 0–1), intermediate risk of developing depression symptoms (score 2–3), high risk of developing depression symptoms (score 4–5), and very high risk of developing depression symptoms (score 6 or higher). Table 4 shows that the proposed risk categories stratify prevalence of Y4 depression symptoms from 15.4% (low risk) to 62.6% (very high risk) (P < .0001) with c = 0.71 in the discovery cohort. Compared with students in the low risk category (i.e., Depression-PI score of 1 or less), students in the intermediate risk category had 75% increased odds of developing depression symptoms by Y4 while those students in the high risk category had nearly fourfold increased odds of developing depression symptoms by Y4, and those in the very high risk category had ninefold increased odds of developing depression symptoms by Y4.

Table 4
Table 4:
Prevalence of End of Y4 (2014) Depression by Baseline Depression Prognostic Index Score in the Discovery and Replication Cohorta
Figure 1
Figure 1:
The prevalence of depression symptoms at the end of year four increases as the Depression Prognostic Index score established at baseline increases among 2,455 U.S. medical students in the discovery cohort, 2010–2014.

Replication dataset to validate the model

The utility of the index to stratify the risk of Y4 depression symptoms developed in the discovery cohort was subsequently evaluated in the students in the replication cohort. Among students in the replication dataset, the prevalence of Y4 depression symptoms of the four risk groups was, respectively, 14.9%, 24.1%, 44.4%, and 64.2% (P < .0001; Table 4). The OR for risk associated with each incremental increase in risk category in the replication cohort was similar to that of each risk category in the discovery cohort. With respect to discrimination, c = 0.72.

Prognostic index without depression symptoms at baseline included

Results from the multivariate logistic analysis using bootstrapping methods can be found in Supplemental Digital Table 1 at http://links.lww.com/ACADMED/A593. Supplemental Digital Table 2 shows the four risk categories, distribution of scores, and prevalence of Y4 depression symptoms. The proposed risk categories stratify the prevalence of Y4 depression rates from 20.2% (low risk) to 62.9% (very high risk) with c = 0.65 in the discovery cohort, and from 17.7% (low risk) to 58.5% (very high risk) with c = 0.67 in the replication cohort.

Discussion

As mentioned, medical students with depression are at increased risk for adverse professional and personal consequences, including declines in academic performance, thoughts of dropping out of medical school, and suicide.1–3 We found a high prevalence (31.2%; 1,167/3,743) of depression symptoms among fourth-year medical students, consistent with the rate reported in the literature.4 For medical students in this prospective longitudinal cohort study, demographics (e.g., age, race, ethnicity), tuition, and depression symptoms, stress, coping behaviors, and social support at baseline were independently associated with the risk of developing depression symptoms at Y4. These findings are consistent with findings from previous cross-sectional and single-institution longitudinal studies.1,13,14,16,17,19,27,46–51 Information on most of these factors is available or obtainable for all students, and each factor has independent prognostic value. We determined how these factors can be combined into a Depression-PI that stratifies risk of developing depression symptoms by Y4.

We demonstrated in the discovery dataset—and further confirmed in the replication dataset—that the Depression-PI defines four prognostic subgroups at low, intermediate, high, and very high risk, therefore providing prognostic information regarding risk of depression symptoms by Y4 of medical school. When baseline depression symptoms were included in the Depression-PI, the c statistic was 0.71 and 0.72, signifying potential prognostic utility at the individual level (as this typically requires a minimum threshold level above 0.70).42,43 We identified high risk and very high risk groups of medical students who had four- to ninefold increased odds of demonstrating depression symptoms at Y4. These two highest-risk groups represented approximately a third of the students in the prospective longitudinal cohort.

When baseline depression symptoms were not included in the Depression-PI, the c statistic is slightly below the 0.7 threshold. Although the model without baseline depression could be used to stratify risk in a group of students, it does not have utility at the individual student level.

Importantly, the Depression-PI has the potential to support medical school efforts to promote student wellness. Student wellness programming should be offered to all students. Doing so is an LCME requirement.9 More importantly, wellness initiatives help to fulfill the medical education community’s moral imperative to address medical student depression, especially given its high prevalence. A risk-stratified approach, however, allows delivery of core material to all students and additional support to those at greatest risk, an approach that could optimize support for students despite finite resources. Such tailoring does not undermine the need for or value of wellness programming offered to all students. For example, students in the intermediate risk group may benefit from added support, whereas students in the high risk groups may warrant allocation of further resources including individualized support, active screening, and early intervention. Implementing interventions in a way that protects confidentiality and avoids stigmatizing individuals in the higher-risk groups is and will continue to be vital. How best to engage students in a tailored primary prevention strategy, and whether doing so leads to less suffering and depression and to fewer adverse consequences, warrants study. These efforts should occur in conjunction with intentional steps to improve the work and learning environment and reduce drivers of distress. We do not recommend that the Depression-PI be used in any capacity to screen applicants or violate privacy.

To our knowledge, this is the largest longitudinal study to explore predictors of development of depression symptoms among U.S. medical students. Of the students in our cohort, those who were nonwhite and non-Hispanic/Latino were at increased odds of developing depression symptoms, even after controlling for baseline depression. The 2009–2012 National Health and Nutritional Examination Survey found non-Hispanic blacks to have higher rates of severe depression than non-Hispanic whites.52 Previous studies of medical students suggest that the mental health of medical students from racial and ethnic minorities may depend on whether or not their race or ethnicity has negatively affected their experience.53 Similarly, other studies have found that students from racial and ethnic minorities experience more microaggressions that they attribute to their race.54–56 Together, these studies suggest that the higher prevalence of depression symptoms among this subgroup of students is likely driven by factors within the learning environment rather than individual traits. Medical schools need to do more to improve the learning environment for nonwhite students.

Our findings—that negative coping behaviors and low social support increase the risk of subsequently developing depression symptoms—are congruent with findings from cross-sectional46–48 and one-year longitudinal studies of medical students.13,49–51 Independent of coping behavior, we also found that high stress levels at baseline predicted subsequent depression symptoms. Stress scores have also been found to be associated with global measures of distress in cross-sectional studies,57 and with depression13,14 and anxiety13 in one-year longitudinal studies of first-year medical students. In aggregate, these studies suggest that intervention studies focused on decreasing reliance on negative coping behaviors, teaching strategies to reduce stress, and promoting social support among medical students should be pursued.

Among the medical school factors we explored, only tuition was independently associated with increased risk of developing depression symptoms; that is, those who attended a medical school with middle-tertile tuition were at higher risk than students who attended a medical school with lower tuition costs. This finding persisted after adjusting for socioeconomic background. Importantly, we did not collect information on accrued undergraduate and medical school debt directly, nor on income from working while in medical school—both of which have been shown in previous studies of medical trainees to be associated with poorer mental health.50,58,59 Other studies suggest that grading structure,60–62 curricular structure and learning communities,63 and harassment/belittlement, as well as poor role-modeling behaviors by faculty,51 relate to medical student mental health.

Our study has a number of limitations. First, although the sample was drawn from a random stratified sample of students from 49 U.S. medical schools, we do not know how well the experience of participants represents the experience of U.S. medical students. We also detected some minor demographic differences between students who responded at both time points and students who responded only to the first survey. We did, however, have a robust response rate, and the demographic characteristics of participants were similar to those of all U.S. medical students who matriculated in the same year, suggesting that our results likely generalize to medical students overall.44,45 Second, we used the PROMIS depression short form 4a as our screening instrument for depression symptoms, and we applied a cutoff score of ≥ 8 (sensitivity 83.1%, specificity 84.3%).32 If we had used a higher cutoff score, our specificity would have improved but our sensitivity would have declined. Third, we depended on self-reported characteristics and included a limited set of individual and school characteristics. Multiple social, psychological, and biological factors contribute to depression,64 and we did not include concomitant anxiety disorders. Fourth, we assessed depression symptoms at two time points (Y1 and Y4) and know that depression symptoms may come and go at other points during the four-year interval. Fifth, we recognize that further validation of the Depression-PI in other datasets is warranted.

Conclusions

The rate of depression symptoms in this longitudinal, national cohort of U.S. medical students at the end of Y4 was 31.2%. Being older, nonwhite, or non-Hispanic/Latino; relying on negative coping behaviors; having high stress, low social support, and depression symptoms at Y1; and being at a medical school with middle-tertile tuition were strongly associated with an increased risk of developing depression symptoms during medical school. The Depression-PI is the first prognostic model to incorporate multiple Y1 factors to stratify the risk of Y4 depression symptoms among medical students. The Depression-PI score can identify students at highest risk for developing depression symptoms and may be useful to medical schools interested in developing a tiered approach to prevention and support for medical students in or at risk for distress.

References

1. Clark DC, Zeldow PB. Vicissitudes of depressed mood during four years of medical school. JAMA. 1988;260:2521–2528.
2. Dyrbye LN, Thomas MR, Power DV, et al. Burnout and serious thoughts of dropping out of medical school: A multi-institutional study. Acad Med. 2010;85:94–102.
3. Dyrbye LN, Thomas MR, Massie FS, et al. Burnout and suicidal ideation among U.S. medical students. Ann Intern Med. 2008;149:334–341.
4. Rotenstein LS, Ramos MA, Torre M, et al. Prevalence of depression, depressive symptoms, and suicidal ideation among medical students: A systematic review and meta-analysis. JAMA. 2016;316:2214–2236.
5. Brazeau CM, Shanafelt T, Durning SJ, et al. Distress among matriculating medical students relative to the general population. Acad Med. 2014;89:1520–1525.
6. Goebert D, Thompson D, Takeshita J, et al. Depressive symptoms in medical students and residents: A multischool study. Acad Med. 2009;84:236–241.
7. Dyrbye LN, Schwartz A, Downing SM, Szydlo DW, Sloan JA, Shanafelt TD. Efficacy of a brief screening tool to identify medical students in distress. Acad Med. 2011;86:907–914.
8. Moutier C, Norcross W, Jong P, et al. The suicide prevention and depression awareness program at the University of California, San Diego School of Medicine. Acad Med. 2012;87:320–326.
9. Liasion Committee on Medical Education; Association of American Medical Colleges; American Medical Association. Standards, publications, and notification forms. http://lcme.org/publications. Updated March 2018. Accessed August 7, 2018.
10. Camp DL, Hollingsworth MA, Zaccaro DJ, Cariaga-Lo LD, Richards BF. Does a problem-based learning curriculum affect depression in medical students? Acad Med. 1994;69(10 suppl):S25–S27.
11. Vitaliano PP, Maiuro RD, Russo J, Mitchell ES, Carr JE, Van Citters RL. A biopsychosocial model of medical student distress. J Behav Med. 1988;11:311–331.
12. Ball S, Bax A. Self-care in medical education: Effectiveness of health-habits interventions for first-year medical students. Acad Med. 2002;77:911–917.
13. Vitaliano PP, Maiuro RD, Russo J, Mitchell ES. Medical student distress. A longitudinal study. J Nerv Ment Dis. 1989;177:70–76.
14. Richman JA, Flaherty JA. Gender differences in medical student distress: Contributions of prior socialization and current role-related stress. Soc Sci Med. 1990;30:777–787.
15. Pyskoty CE, Richman JA, Flaherty JA. Psychosocial assets and mental health of minority medical students. Acad Med. 1990;65:581–585.
16. Lloyd C, Gartrell NK. Sex differences in medical student mental health. Am J Psychiatry. 1981;138:1346–1351.
17. Zoccolillo M, Murphy GE, Wetzel RD. Depression among medical students. J Affect Disord. 1986;11:91–96.
18. Levine RE, Litwins SD, Frye AW. An evaluation of depressed mood in two classes of medical students. Acad Psychiatry. 2006;30:235–237.
19. Rosal MC, Ockene IS, Ockene JK, Barrett SV, Ma Y, Hebert JR. A longitudinal study of students’ depression at one medical school. Acad Med. 1997;72:542–546.
20. Hardeman RR, Burgess D, Phelan S, Yeazel M, Nelson D, van Ryn M. Medical student socio-demographic characteristics and attitudes toward patient centered care: Do race, socioeconomic status and gender matter? A report from the Medical Student CHANGES study. Patient Educ Couns. 2015;98:350–355.
21. Hardeman RR, Przedworski JM, Burke SE, et al. Mental well-being in first year medical students: A comparison by race and gender: A report from the Medical Student CHANGE study. J Racial Ethn Health Disparities. 2015;2:403–413.
22. Phelan SM, Burgess DJ, Puhl R, et al. The adverse effect of weight stigma on the well-being of medical students with overweight or obesity: Findings from a national survey. J Gen Intern Med. 2015;30:1251–1258.
23. Phelan SM, Dovidio JF, Puhl RM, et al. Implicit and explicit weight bias in a national sample of 4,732 medical students: The medical student CHANGES study. Obesity (Silver Spring). 2014;22:1201–1208.
24. Phelan SM, Puhl RM, Burke SE, et al. The mixed impact of medical school on medical students’ implicit and explicit weight bias. Med Educ. 2015;49:983–992.
25. van Ryn M, Hardeman R, Phelan SM, et al. Medical school experiences associated with change in implicit racial bias among 3547 students: A Medical Student CHANGES study report. J Gen Intern Med. 2015;30:1748–1756.
26. van Ryn M, Hardeman RR, Phelan SM, et al. Psychosocial predictors of attitudes toward physician empathy in clinical encounters among 4732 1st year medical students: A report from the CHANGES study. Patient Educ Couns. 2014;96:367–375.
27. Dyrbye LN, Thomas MR, Shanafelt TD. Systematic review of depression, anxiety, and other indicators of psychological distress among U.S. and Canadian medical students. Acad Med. 2006;81:354–373.
28. Dyrbye LN, Harper W, Moutier C, et al. A multi-institutional study exploring the impact of positive mental health on medical students’ professionalism in an era of high burnout. Acad Med. 2012;87:1024–1031.
29. Dyrbye L, Shanafelt T. A narrative review on burnout experienced by medical students and residents. Med Educ. 2016;50:132–149.
30. Association of American Medical Colleges. Age of applicants to US medical schools at anticipated matriculation by sex and race/ethnicity, 2014–2015 through 2017–2018. https://www.aamc.org/download/321468/data/factstablea6.pdf. Published 2017. Accessed January 3, 2019.
31. Pilkonis PA, Choi SW, Reise SP, Stover AM, Riley WT, Cella D; PROMIS Cooperative Group. Item banks for measuring emotional distress from the Patient-Reported Outcomes Measurement Information System (PROMIS®): Depression, anxiety, and anger. Assessment. 2011;18:263–283.
32. Kroenke K, Yu Z, Wu J, Kean J, Monahan PO. Operating characteristics of PROMIS four-item depression and anxiety scales in primary care patients with chronic pain. Pain Med. 2014;15:1892–1901.
33. Cohen S, Williamson GM. Oskamp S, Spacapan S. Perceived stress in a probability sample of the United States. In: The Social Psychology of Health. 1988:Newbury Park, CA: Sage; 31–67.
34. Cohen S, Kamarck T, Mermelstein R. A global measure of perceived stress. J Health Soc Behav. 1983;24:385–396.
35. Norman GR, Sloan JA, Wyrwich KW. The truly remarkable universality of half a standard deviation: Confirmation through another look. Expert Rev Pharmacoecon Outcomes Res. 2004;4:581–585.
36. Carver CS. You want to measure coping but your protocol’s too long: Consider the brief COPE. Int J Behav Med. 1997;4:92–100.
37. Hays RD, Sherbourne CD, Mazel RM. User’s Manual for the Medical Outcomes Study (MOS) Core Measures of Health-Related Quality of Life. 1995. Santa Monica, CA: RAND; https://www.rand.org/content/dam/rand/pubs/monograph_reports/2008/MR162.pdf. Accessed August 7, 2018.
38. Sherbourne CD, Stewart AL. The MOS social support survey. Soc Sci Med. 1991;32:705–714.
39. Association of American Medical Colleges. U.S. medical school applications and matriculants by school, state of legal residence, and sex. https://www.aamc.org/download/321442/data/factstablea1.pdf. Published 2016. Accessed April 24, 2017. [No longer available.]
40. Association of American Medical Colleges. U.S. Medical Schols Tuition and Student Fees Report, 2012–2013 through 2016–2017. https://www.aamc.org/data/tuitionandstudentfees. Published 2017. Accessed August 7, 2018.
41. U.S. News and World Report. Best medical schools: Research. https://www.usnews.com/best-graduate-schools/top-medical-schools/research-rankings. Published 2017. Accessed April 27, 2017. [No longer available.]
42. Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: Issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15:361–387.
43. Hosmer DW, Lemeshow S. Applied Logistic Regression. 2000.2nd ed. New York, NY: John Wiley & Sons, Inc..
44. Przedworski JM, Dovidio JF, Hardeman RR, et al. A comparison of the mental health and well-being of sexual minority and heterosexual first-year medical students: A report from the Medical Student CHANGE study. Acad Med. 2015;90:652–659.
45. Association of American Medical Colleges. Diversity in medical education: facts and figures 2012. https://members.aamc.org/eweb/upload/Diversity%20in%20Medical%20Education%20Facts%20and%20Figures%202012.pdf. Published Fall 2012. Accessed August 7, 2018.
46. Mosley TH Jr, Perrin SG, Neral SM, Dubbert PM, Grothues CA, Pinto BM. Stress, coping, and well-being among third-year medical students. Acad Med. 1994;69:765–767.
47. Vitaliano PP, Maiuro RD, Russo J, Becker J. Raw versus relative scores in the assessment of coping strategies. J Behav Med. 1987;10:1–18.
48. Vitaliano PP, Russo J, Carr JE, Heerwagen JH. Medical school pressures and their relationship to anxiety. J Nerv Ment Dis. 1984;172:730–736.
49. Vitaliano PP, Maiuro RD, Mitchell E, Russo J. Perceived stress in medical school: Resistors, persistors, adaptors and maladaptors. Soc Sci Med. 1989;28:1321–1329.
50. Dyrbye LN, Power DV, Massie FS, et al. Factors associated with resilience to and recovery from burnout: A prospective, multi-institutional study of US medical students. Med Educ. 2010;44:1016–1026.
51. Haglund ME, aan het Rot M, Cooper NS, et al. Resilience in the third year of medical school: A prospective study of the associations between stressful events occurring during clinical rotations and student well-being. Acad Med. 2009;84:258–268.
52. Centers for Disease Control and Prevention. Depression in the U.S. household population, 2009–2012. https://www.cdc.gov/nchs/data/databriefs/db172.htm. Published 2014. Accessed August 7, 2018.
53. Dyrbye LN, Thomas MR, Eacker A, et al. Race, ethnicity, and medical student well-being in the United States. Arch Intern Med. 2007;167:2103–2109.
54. Montenegro RE. A piece of my mind. My name is not “interpreter.” JAMA. 2016;315:2071–2072.
55. Ulloa JG, Talamantes E, Moreno G. Microaggressions during medical training. JAMA. 2016;316:1113–1114.
56. Beagan BL. “Is this worth getting into a big fuss over?” Everyday racism in medical school. Med Educ. 2003;37:852–860.
57. Notman MT, Salt P, Nadelson CC. Stress and adaptation in medical students: Who is most vulnerable? Compr Psychiatry. 1984;25:355–366.
58. West CP, Shanafelt TD, Kolars JC. Quality of life, burnout, educational debt, and medical knowledge among internal medicine residents. JAMA. 2011;306:952–960.
59. Jackson ER, Shanafelt TD, Hasan O, Satele DV, Dyrbye LN. Burnout and alcohol abuse/dependence among U.S. medical students. Acad Med. 2016;91:1251–1256.
60. Reed DA, Shanafelt TD, Satele DW, et al. Relationship of pass/fail grading and curriculum structure with well-being among preclinical medical students: A multi-institutional study. Acad Med. 2011;86:1367–1373.
61. Rohe DE, Barrier PA, Clark MM, Cook DA, Vickers KS, Decker PA. The benefits of pass–fail grading on stress, mood, and group cohesion in medical students. Mayo Clin Proc. 2006;81:1443–1448.
62. Bloodgood RA, Short JG, Jackson JM, Martindale JR. A change to pass/fail grading in the first two years at one medical school results in improved psychological well-being. Acad Med. 2009;84:655–662.
63. Slavin SJ, Schindler DL, Chibnall JT. Medical student mental health 3.0: Improving student wellness through curricular changes. Acad Med. 2014;89:573–577.
64. World Health Organization. Depression. http://www.who.int/mediacentre/factsheets/fs369/en. Updated March 22, 2018. Accessed August 7, 2018.

Supplemental Digital Content

Copyright © 2018 by the Association of American Medical Colleges