Secondary Logo

Journal Logo

Curriculum and Assessment

Scholarly Concentration Program Development: A Generalizable, Data-Driven Approach

Burk-Rafel, Jesse MRes; Mullan, Patricia B. PhD; Wagenschutz, Heather MA, MBA; Pulst-Korenberg, Alexandra MD, MBA; Skye, Eric MD; Davis, Matthew M. MD, MAPP

Author Information
doi: 10.1097/ACM.0000000000001362


Scholarly concentration (SC) programs—also known as scholarly projects, pathways, tracks, or pursuits—are designed to provide medical students learning opportunities beyond their standardized core curricula, facilitating development of specialized expertise in an area of scholarship, critical thinking, and methods of investigation.1,2 Unlike most dual-degree programs, scholarly concentrations do not prolong time to graduation. SC programs are increasingly featured and prevalent at U.S. medical schools, as evidenced by a 2010 Academic Medicine theme issue3 and publications describing the trend.4,5

Despite broadening adoption in medical schools, research describing SC programs is limited. Several publications describe well-established programs (e.g., at Yale, Duke, and Stanford)6–8 or focus on components (e.g., content, mentorship, administration, evaluation) of fully scaled programs.5,9–11 Some authors indicate a process for creating new scholarly concentrations incorporating “interest to students.”8 However, the literature to date has not detailed a replicable systematic approach for planning, creating, and scaling up SC programs, including understanding the utility of eliciting student preferences or determining the “ideal” number of concentrations to offer.

Our SC development approach was informed by the framework of Kern and Thomas,12 which advocated curricula be responsive to population-based needs and local stakeholders through a systematic methodology involving general and targeted needs assessments. As our institutional stakeholders sought to develop a new SC program, three key questions arose: Nationally, what are the predominant SC content domains and program missions? At an institutional level, how can student preferences for proposed concentrations and content be determined? Can student preferences inform operational decisions, such as the ideal number of concentrations and expected capacity requirements? In addressing these questions, we demonstrate the feasibility and yield of a data-driven, student-focused, and generalizable approach to creating SC programs.


Systematic approach

Our systematic approach to SC program development involved general and targeted needs assessments (Figure 1), which shaped subsequent institutional actions.

Figure 1
Figure 1:
Systematic approach to scholarly concentration program development. (1) General needs assessment via a national review of SC programs. Targeted needs assessment through (2) a committee process, (3) student preference elicitation, and (4) statistical analysis. (5) Needs assessments inform creation and scale-up of SC program. (6) Iteration ensures program alignment with student interest and institutional strengths.

National review of programs

In April 2014, we analyzed scholarly concentration offerings at U.S. medical schools ranked among the top 25 for research or primary care by U.S. News & World Report (n = 43 institutions).13 Search methods included (1) an Internet search using relevant terms (“scholarly,” “concentration,” “project,” “track,” “pathway,” “thesis,” AND “{school name}”); (2) literature search using similar terms; and (3) manual exploration of institutional Web sites. We defined SC programs as elective or required longitudinal programs (minimum one-month duration) focusing on scholarship and mentorship that do not lengthen time to graduation. Honorific degrees and isolated electives were excluded. We recorded and thematically analyzed program titles, concentrations, and mission statements in the identified SC programs.

Collaborative planning with stakeholders

An SC planning committee of students, faculty, and other key stakeholders met monthly to formulate proposed “Pathways” (our institution’s term for scholarly concentrations) and “Topics” (potential co-curricular content). The committee discussed established SC programs and brainstormed Pathways and Topics not elsewhere identified.

Student preferences survey

The committee sought to determine which Pathways our students most desired and the minimum number of Pathway offerings needed to place the majority of students in their top preferences (presuming only one Pathway per student). Students on the committee developed, pilot tested, and distributed a student preferences survey. All medical students enrolled at the study institution (n = 781) were invited to participate in a voluntary, anonymous survey distributed electronically in August 2014. The University of Michigan medical institutional review board approved the study. At that time, our institution offered two elective, fully deployed Pathways: “Global Health & Disparities” and “Bioethics.” The survey, available by request from the authors, elicited students’ demographic information and preferences for Pathways and Topics, which were presented on separate survey pages with items in random order to mitigate ordering effects. Students were asked to rank Pathway options from 1 (highest) to 10 (lowest) and were permitted to tie rankings or leave Pathways unranked. Topic interest was rated on a scale of neutral, somewhat, or very interested, with the percentage of students “very interested” serving as the outcome measure.

Three nonoverlapping student subgroups were identified a priori for analysis: (1) “preclinical” (first- or second-year students, excluding Medical Scientist Training Program [MSTP] MD/PhD students); (2) “clinical” (third- or fourth-year students, excluding MSTP); and (3) MSTP (all years). Subgroups were assessed for differences using chi-square testing, with subsequent two-sample z tests for independent proportions at the P < .05 level with a Bonferroni correction for multiple comparisons. Confidence intervals for proportions were calculated using Wilson’s14 method. Cronbach alpha and McDonald omega were calculated in R v3.2.2 (Vienna, Austria) using the “psych” package,15 with bootstrapped 95% confidence intervals.

Exploratory factor analysis

We used exploratory factor analysis (EFA) (SPSS v23, Chicago, Illinois) to examine Topic relationships as expressed through students’ levels of interest on the three-point scale, following best EFA practices suggested by Wetzel16 in Academic Medicine. We used principal axis factoring, as the normality assumption necessary for maximum likelihood was not met,17 and an oblique rotation (direct oblimin with delta = 0, Kaiser normalization), as we could not assume that interests were unrelated.18 The Cattell scree test and parallel analysis were used to determine the number of factors to retain.19,20 Items with rotated factor loadings greater than 0.32 (10% overlapping variance) were included in factors for naming, as recommended by Tabachnick and Fidell.21

Capacity optimization algorithm

Our optimization algorithm, designed by the authors in Microsoft Excel (Microsoft Corporation, Redmond, Washington), maximized the number of students receiving their first or second Pathway choice for a fixed number of Pathway offerings, with versus without constraints on the number of students who could enroll in any given Pathway.


National review of programs

Our national review of 43 top-ranked U.S. medical schools identified SC programs at 32 institutions (74%). Analysis of these 32 SC-offering institutions found a total of 199 scholarly concentrations (mean concentrations per program: 6.2, mode: 5, range: 1–16) (Supplemental Digital Appendix 1,, among which we identified 10 cross-cutting content domains through thematic analysis (Table 1).

Table 1
Table 1:
Scholarly Concentration Content Domains in 32 Medical Schools Offering Scholarly Concentration Programs Among a National Sample of 43 U.S. Medical Schools, April 2014

Examination of SC program mission statements revealed an emphasis on training students to pose and answer scholarly questions. Multiple programs sought to develop habits of self-directed lifelong learning, scholarship, and enthusiasm for medicine. Fostering students’ intellectual curiosity, independent creativity, leadership, and passion for discovery was also frequently emphasized. Moreover, programs sought to create opportunities for mentorship, peer education, and career development.

Pathways and Topics emerge

In parallel to our review of SC programs nationally, our planning committee—incorporating additional student voices and content experts—drafted scholarly concentrations intended to appeal to students while aligning with institutional expertise and resources. These committee-generated concentrations overlapped significantly with the national content domains in Table 1 and were reconciled to generate 10 proposed “Pathways”: Global Health & Disparities; Scientific Discovery & Research; Medical Decision Making; Health Policy; Entrepreneurship & Innovation; Scholarship of Learning & Teaching; Medical Humanities; Quality Improvement & Patient Safety; Bioethics; and Health Informatics. Concurrently, the committee and content experts identified 30 proposed “Topics,” representing content areas that could be delivered through the proposed Pathways.

Eliciting student preferences

The overall response rate to our student-driven, institution-wide survey was 60% (n = 468/781). Almost all first-year medical students participated (97%, 171/176). Participation rates of preclinical students (85%, 272/322) were higher than clinical (39%, 138/351) and MSTP students (54%, 58/108).


In pilot testing, students noted that our ranking approach—permitting tied rankings and unranked Pathways—allowed them to express their preferences fully. Pilot testers also emphasized that receiving a Pathway assignment below their second choice would be unsatisfactory. Thus, we dichotomized the ranking data to first/second choice versus not first/second choice (Table 2). Scientific Discovery was the only Pathway with significant differences across student subgroups, with MSTP students rating it a top choice more frequently (70.7%, 95% confidence interval: 58.0%–80.8%) compared with preclinical (34.9%, 95% CI: 29.5%–40.8%, P < .001) and clinical students (31.2%, 95% CI: 24.0%–39.3%, P < .001).

Table 2
Table 2:
Medical Students Ranking a Pathway (i.e., Scholarly Concentration) as a First or Second Choice,a University of Michigan Medical School, August 2014


Student interest in Topics varied. For example, 44% of students selected “Clinical/Outcomes Research” and 38% selected “Global Health,” versus only 8% selecting “Market Research” (Table 3, “All Students”). Preclinical, clinical, and MSTP student Topic preferences differed: MSTP students indicated greater interest in “Translational Research” (MSTP 71% vs. preclinical 31% and clinical 25%, both P < .001), “Basic Science Research” (69% vs. 11% and 9%, both P < .001), and “Bioinformatics” (47% vs. 7% and 9%, both P < .001). MSTP students demonstrated less interest in “Clinical/Outcomes Research” (28%) compared with preclinical (46%, P = .01) and clinical students (46%, P = .02), but these comparisons did not reach statistical significance after Bonferroni correction. Preclinical and clinical students did not differ significantly in their Topic preferences. Of note, first-year students who had published scientific articles prior to medical school indicated lower interest in 25 of 30 Topics compared with nonpublished colleagues, while those who had conducted full-time health-related work prior to medical school indicated greater interest in 20 of 30 Topics (data not shown).

Table 3
Table 3:
Students “Very Interested” in Topics, Among All Students and Students Ranking a Pathway as Their First or Second Choice, With Maximum Topic Endorsement Rates in Boldface, University of Michigan Medical School, August 2014

Prior to performing EFA, we found that students’ stated Topic preferences had good-to-excellent internal consistency: Cronbach α was 0.88 (95% CI: 0.78–0.91), and McDonald ω was 0.93 (95% CI: 0.92–0.95). Our overall respondent sample size (n = 468), participant-to-item ratio (16:1), and Kaiser–Meyer–Olkin measure of sampling adequacy (0.87) indicated that EFA was appropriate.17,22 Variables were correlated (Bartlett test of sphericity, P < .001), and the determinant of the R matrix (1.3 × 10–5) indicated some multicollinearity without warranting removal of correlated variables.23

The EFA rotation converged after 17 iterations. The Cattell scree test and parallel analysis both indicated that eight factors were appropriate, producing large loadings and few cross-loadings (Supplemental Digital Appendix 2, As an example, the Topics “Health Policy,” “Health Economics,” and “Leadership/Hospital Administration” loaded cleanly on one factor, which we named “Health Policy, Economics, & Leadership.” The eight extracted factors captured 63% of the variance in Topic interest ratings.

Topic–Pathway relationships.

Among students ranking a Pathway as their first or second choice, the percentage indicating they were “very interested” in a Topic varied systematically across Pathways (Table 3). For example, 67% of students rating Global Health & Disparities as a top-choice Pathway also indicated strong interest in the “Service/Community Development” Topic, compared with only 8% among students interested in the Scientific Discovery Pathway.

Capacity optimization algorithm

We modeled the relationship between the number of potential Pathway offerings and student placement into a desired Pathway using first-year medical students’ responses (n = 171), who most closely represented our target Pathway “selectors.” With two Pathways, 27% of students could not be placed into their first or second choice; with six Pathways, only 5% (9/171) faced nonpreferred Pathway choices (Figure 2). In contrast, increasing from six to eight Pathways increased the proportion of students placed into their first choice from 84% to 92%, but did not substantially reduce the proportion of students who did not receive either their first or second preference. Notably, many of these unplaced students indicated no interest in any of the proposed Pathways.

Figure 2
Figure 2:
Relationship between number of Pathways (i.e., scholarly concentrations) offered and student placement into first-choice, second-choice, or below-second-choice Pathway, based on optimization model of Pathway selection using first-year medical student preference data (n = 171).

Given that Pathways may not have elastic capacity because of faculty or institutional constraints, we also conducted an analysis in which Pathway enrollment was capped to ensure equal enrollment across the offered Pathways (data not shown). This constrained model had deleterious effects on preservation of students’ strong preferences (i.e., students were placed in their lower-rated Pathways because their top-choice Pathways had reached capacity).


In this study, we have analyzed SC offerings nationally and developed an optimization algorithm that incorporates student preferences and institutional faculty constraints to inform program development.

Our review confirmed that SC programs are highly prevalent at leading U.S. medical schools (74% in our sample)—but it also revealed remarkable consistency in SC content, with several domains (global/public health and clinical/translational research) nearly ubiquitous at the sampled institutions. These findings may reflect growing learner interest, areas of deficiency in core curricula, or a founder effect, with new programs taking cues from more established peers. The national review also influenced our local SC program development efforts, enhancing our resolve to collaborate with students regarding the focus and content of our program.

In our student-led survey of Pathway preferences, we found that students’ interests varied substantially. Notably, the Medical Decision Making and the Entrepreneurship & Innovation Pathways were rated highly by our students. These Pathways were not identified in our national sample and may be promising for SC development at other institutions. Likewise, interest in Topics ranged widely among students. Associations between Pathway and Topic preferences were largely intuitive (e.g., students ranking the Global Health & Disparities Pathway highly had greater interest in the “Service/Community Development” Topic). However, Topic interest was not confined to silos: every Pathway attracted students with diverse Topic interests, and some relationships were less intuitive (e.g., students ranking the Scientific Discovery Pathway highly had less interest in “Service/Community Development”). In response to these findings, our burgeoning SC program targeted high-demand Pathways for rapid scale-up while preparing for diverse interests within each Pathway.

MD/PhD (MSTP) students systematically differed from their MD-only peers. MSTP students were significantly more likely to identify Scientific Discovery as a high-preference Pathway, and endorsed greater interest in basic science research, translational research, and bioinformatics, and less interest in clinical/outcomes research. These findings suggest that SC programs attracting MSTP students must be mindful of their specific research-intensive career goals. Additionally, student interest in Topics appeared to be higher among those who had conducted health-focused work prior to entering medical school and lower among those who had published scientific articles, suggesting two important subgroups for investigation. Further exploration of demographic factors may provide insight into which students would be well served by specific SC program offerings, informing admissions efforts and counseling of new students.

We also demonstrated the utility of EFA for identifying latent variables describing Topic interest covariance. Although most associations were intuitively plausible, some Topics comprising factors were unexpected. For example, “Law and Medicine” loaded more heavily on the “Humanism in Medicine” factor than on “Health Policy, Economics, & Leadership.” Notably, “Clinical/Outcomes Research” did not load on any one factor because of its broad appeal. These results helped our planning committee understand potential Pathway curricular content that might be developed and deployed as a unit for all students, rather than limited to select Pathways.

Most useful to our institution in developing an SC program has been our development of an algorithm incorporating capacity optimization. Student engagement and enthusiasm for their selected scholarly concentration is vital to the success of any SC program. Through modeling, our planning committee determined that at least six Pathways would be needed for a required program if we wished to offer the vast majority of students (≥ 95%) either their first or second choice. Offering more than six Pathways increased the proportion of students placed into their first choice, but had little effect in placing a small group of students who indicated no interest in any of the offered Pathways. This study prepared us to identify and counsel these students.

Of note, Pathway capacity was not constrained in optimal models. In contrast, models with constrained Pathway enrollment indicated low preference satisfaction for many students. This finding reframed our subsequent planning committee discussions and led us to design an implementation model that could adapt to varying year-to-year student preferences while anticipating Pathway size differences.


Our national needs assessment was restricted to a targeted cohort of medical schools ranked in the “top 25” by a third-party publication. Additionally, our Internet-based analysis of program offerings was only as accurate as the materials institutions placed online and our ability to locate those materials, as we did not confirm SC offerings with the sampled institutions. Moreover, our preference elicitation was conducted at a single institution, and while 97% of first-year students responded, our overall response rate (60%) was more moderate, raising the possibility of nonresponse bias. Nonetheless, we emphasize that our focus was developing a generalizable approach rather than ascertaining externally valid estimates.

Our ranking process also had potential limitations. Students were asked to rank 10 Pathways; it is possible, however, that student ranking preferences may vary depending on the number of Pathways offered. Furthermore, respondents relied on brief written descriptions of Pathway offerings. Faculty-led information sessions and other informational materials might influence student understanding and preferences, leading to different rankings. Nevertheless, this process could be repeated after information sessions to obtain more accurate estimates.


Our study informs efforts to provide captivating SC programs to medical students. After describing the national landscape of SC programs in top-ranked U.S. medical schools, we demonstrated how institutions can elicit and incorporate their learners’ preferences to develop SC programs reflecting students’ passions.

Importantly, this work was student led and learner focused. Despite lack of monetary or other incentives, our student body was responsive to a “student-to-student” survey. By engaging students in the process, we facilitated buy-in. In turn, disseminating the learner preference findings contributed to the interest and morale of our “community of practice” among students, faculty, and administrators. Such transparency promoted trust that our institution was listening carefully to student expectations for Pathway offerings.

The Kern and Thomas framework guided a rigorous needs assessment that influenced key decisions in scaling up our SC program. By providing national data and empiric learner preferences to working groups, time-consuming guesswork was greatly reduced. Ultimately, this work informed operational concerns, including determining the optimal number of Pathways, targeting administrative resources, and developing content. Our approach is feasible and provides prompt results: the needs assessment took under six months to complete and shape SC institutional planning. Additionally, capacity optimization offers a method that should enable other institutions to determine which Pathways to scale up and the minimum number of Pathways necessary to ensure that students receive their top choices. In our institution, the student preference survey, EFA, and optimization methods informed the decision to increase our SC offerings in 2015 from two to six Pathways, with a further increase to eight Pathways in 2016.

This evidence-based, learner-focused approach can be adapted by other medical schools seeking to create (or enhance) an SC program that reflects both their learners’ interests and their institution’s capacity constraints. The approach is applicable beyond SC program development to other professional education programs.

Acknowledgments: The authors wish to thank the many Pathways of Excellence planning committee members, who volunteered their time and provided invaluable discussion, generating Pathways and Topics. The authors also thank the hundreds of students who responded to the voluntary and nonincentivized survey. Finally, they thank Whitney Townsend for help selecting keywords and Brent Stansfield for helpful discussion of exploratory factor analysis.


1. Parsonnet J, Gruppuso PA, Kanter SL, Boninger M. Required vs. elective research and in-depth scholarship programs in the medical student curriculum. Acad Med. 2010;85:405408.
2. Bierer SB, Chen HC. How to measure success: The impact of scholarly concentrations on students—A literature review. Acad Med. 2010;85:438452.
3. Boninger M. Scholarly concentrations in the medical student curriculum. Acad Med. 2010;85:403404.
4. Schor NF, Troen P, Kanter SL, Levine AS. The Scholarly Project Initiative: Introducing scholarship in medicine through a longitudinal, mentored curricular program. Acad Med. 2005;80:824831.
5. Green EP, Borkan JM, Pross SH, et al. Encouraging scholarship: Medical school programs to promote student inquiry beyond the traditional medical curriculum. Acad Med. 2010;85:409418.
6. Ebbert A Jr. A retrospective evaluation of research in the medical curriculum. J Med Educ. 1960;35:637643.
7. Cook S, Grochowski CO, Atherton A, et al. Developing physician leaders for over 50 years: The Duke medical student research experience in the US and Singapore. Med Sci Educ. 2011;21:5358.
8. Laskowitz DT, Drucker RP, Parsonnet J, Cross PC, Gesundheit N. Engaging students in dedicated research and scholarship during medical school: The long-term experiences at Duke and Stanford. Acad Med. 2010;85:419428.
9. Boninger M, Troen P, Green E, et al. Implementation of a longitudinal mentored scholarly project: An approach at two medical schools. Acad Med. 2010;85:429437.
10. Gotterer GS, O’Day D, Miller BM. The Emphasis Program: A scholarly concentrations program at Vanderbilt University School of Medicine. Acad Med. 2010;85:17171724.
11. Williams BC, Mullan PB, Haig AJ, et al. Developing a professional pathway in health equity to facilitate curricular transformation at the University of Michigan Medical School. Acad Med. 2014;89:11531156.
12. Kern DE, Thomas PA. Pangaro L. “What do leaders need to know about curriculum planning?” Leadership Careers in Medical Education. 2010:Philadelphia, PA: ACP Press; 101127.
13. U.S. News & World Report. Best graduate schools. 2015. Accessed July 12, 2016.
14. Wilson EB. Probable inference, the law of succession, and statistical inference. J Am Stat Assoc. 1927;22:209212.
15. Revelle W. Procedures for psychological, psychometric, and personality research. 2015. Accessed July 12, 2016.
16. Wetzel AP. Factor analysis methods and validity evidence: A review of instrument development across the medical education continuum. Acad Med. 2012;87:10601069.
17. Fabrigar LR, Wegener DT, MacCallum RC, Strahan EJ. Evaluating the use of exploratory factor analysis in psychological research. Psychol Methods. 1999;4:272299.
18. Costello AB, Osborne JW. Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Pract Assess Res Eval. 2005;10(7). Accessed July 7, 2016.
19. Horn JL. A rationale and test for the number of factors in factor analysis. Psychometrika. 1965;30:179185.
20. O’Connor BP. SPSS and SAS programs for determining the number of components using parallel analysis and velicer’s MAP test. Behav Res Methods Instrum Comput. 2000;32:396402.
21. Tabachnick BG, Fidell LS. Using Multivariate Statistics. 2001.Boston, MA: Allyn and Bacon.
22. Williams B, Brown T, Onsman A. Exploratory factor analysis: A five-step guide for novices. Australas J Paramedicine. 2010;8(3):113.
23. Field A. Discovering Statistics Using SPSS for Windows. 2000.London, UK: Sage.

Supplemental Digital Content

Copyright © 2016 by the Association of American Medical Colleges