Critical to the success of the program is orientation of students as well as mentors. Students first learn about the SP when they apply to UPSOM such that all matriculants are aware that it is a requirement for graduation. Early in their first year, students attend two mandatory information sessions that occur as part of the preparatory course work. These sessions clarify the goals, procedures, and timelines of the SP, provide students with advice on identifying appropriate mentors and projects, and demonstrate how to use the Zone.
A number of ways exist for students to learn of topic areas for their SP. A “research-opportunities fair” is held in the fall of each year for first-year students and highlights potential SP and SRP projects. The fair invites representatives from the AOCs and from departments with research programs who are interested in attracting students. Multiple lunchtime sessions featuring different departments and National Institutes of Health-funded summer programs provide another resource for first-year students exploring SP options. These lunch programs are usually dually focused on SRP and SP opportunities. Further, the Zone provides a list of all current and former projects and the associated mentors. In addition, the Zone lists mentors who have volunteered to take on students, even if they are not currently working with a student. On occasion, a specific mentor will inform the SP directors about a project, and this, too, will be posted on the site.
Students often find mentors without using these resources. This usually occurs on the basis of interest area. For example, a student interested in pediatrics may e-mail or call the pediatrics department and meet with the pediatrics research director. Another primary way students identify mentors is by approaching a lecturer whose class they have enjoyed and asking if he or she would possibly be willing to serve as a mentor while they work on their SP. Students also hear of good mentors or projects from other students and approach these mentors directly. Students may change mentors or projects if they have received their director's approval.
Faculty are oriented to the SP in a variety of ways. Annual town hall meetings in the fall semester permit the SP directors to update mentors—current and potential—on the progress of the program and allow the faculty to provide feedback. The entire faculty also receives mailings describing the SP initiative and addressing frequently asked questions (FAQs). These FAQs cover what is expected of students during clinical months and how the elective research months work. Once a mentor becomes linked to a student, he or she receives additional information regarding the workings of the Zone and how it is used to provide approval and foster communication.
The preparatory course work for the SP consists of a curricular block titled Scientific Reasoning and Medicine, comprising two courses. A one-semester course—Medical Decision Making—focuses on basic experimental design and statistical methods commonly used in medical studies. This is followed by a two-semester course—Methods and Logic in Medicine—that, employing small-group formats, (1) focuses on reading, discussing, and critiquing primary literature in medicine, and (2) allows students to present ideas and plans for their own SPs and receive feedback from classmates and faculty facilitators.
Mentors and mentoring
The primary instructor for the SP is the mentor. This arrangement is very similar to that of many PhD or master programs. UPSOM expects the mentor to have regular contact with the student over the course of the project. Mentors must sign off on the proposal and final report, and they must evaluate their students twice during the course of the SP. In addition, mentors receive e-mail copies of their students' quarterly reports for review. UPSOM's expectation, communicated to the mentors during orientation, is that mentors will review all phases of the project from ethical consideration to data collection, analysis, and interpretation. UPSOM does not restrict the students' selection of mentors, but the school does strongly encourage students to select mentors with ongoing research in their area of interests. Students are free to select mentors from other institutions. However, in these cases they are required to have a secondary, local mentor who is a member of the UPSOM faculty.
The custom-designed Zone provides tools to address the educational and administrative needs of the SP initiative. Features of this Web site include
1. The SP public site, http://zone.medschool.pitt.edu/sites/programs/scholarlyproject/default.aspx, which contains general information about the SP that may be of interest to, for example, potential medical school applicants.
2. The SP program site, a password-protected area that provides SP information including special announcements, general information on procedures, and answers to FAQs. The site provides a list of potential faculty mentors and their academic interests, links to current student projects, and ethical status information for projects involving human and animal research and quality improvement studies. The SP program site also acts as a storehouse of documents critical to the SP, including the instructions for submitting the SP proposal and final report, the criteria by which faculty and directors judge the proposal and final report, and policies that govern grading and satisfactory process.A section of this site, accessible only to the SP Executive Committee, contains supervisory documentation such as the minutes of director meetings and SP Executive Committee policies. In addition, this section of the site documents SP work processes (e.g., research elective approval, changing mentors, notifying students of late reports) with a series of flow charts, which supervisory personnel can reference.
3. Student collaborative learning Web sites (Figure 1; described below), which students own and manage. Students use these sites to share documents, upload progress reports, and communicate with their mentors and SP directors.
Student collaborative learning Web sites.
These are the portfolio-like student project Web sites (developed using business collaboration software, SharePoint [versions 2003 and 2007], Microsoft, Redmond, Washington), and collectively they include the following features:
1. Password-protected access to the Web site based on role. Thus, the student designates a mentor who is invited to the Web site. If needed, secondary mentors and outside collaborators, even other students, receive access. In addition, the five SP directors receive access for oversight purposes.
2. A home page where students declare their project and invite a mentor. This page prominently displays the project's approval status and regulatory/compliance status, such as institutional review board (IRB) approval (Figure 1).
3. A custom discussion board for dialogue among site members related to all aspects of the SP.
4. Custom tracking of the proposal, electives, progress reports, and the final report based on a predetermined timeline.
5. An automated e-mail alert system for students and faculty that includes links to any new element of the student's Web site, such as a new discussion post or the submission of a quarterly report.
The quarterly reports.
These are Web-based forms on the student's project Web site that ask five questions:
1. Has your mentor changed since the last quarterly report?
2. Has your project title changed since the last quarterly report?
3. Has there been activity related to your SP during the past three months?
4. Has work with your mentor resulted in publications, presentations, or awards?
5. Do you have questions for your mentor or the SP directors?
Items 1 to 3 help identify students who may be having problems with their SP. Item 3 includes a text box where students are asked to describe their SP-related activities during the previous three months. Items 4 and 5 also have a text box for the student to fill in if the answer is “yes.” In addition, there are two text boxes all students are asked to complete: one asking what SP-related activities they have planned for the next three months, and another asking them to respond to questions posed by reviewers in response to their initial proposal or subsequent quarterly reports.
Automatic tracking and queries.
The five SP directors are primarily responsible for tracking the approximately 560 students working on their SP at any one time. Because this is a mandatory, graded activity, assessing student progress and determining whether students are experiencing problems related to their SPs are vital. Two primary mechanisms help ensure that the SP directors remain aware of SP activity. Three times a week, each director receives an automated e-mail that lists any activity occurring on the student SP Web sites. This e-mail contains a brief summary of the specific activity in addition to direct links for quickly accessing the items. Examples of items reported in the e-mail are notifications when students post or edit a document or when they submit a quarterly report, messages posted by mentors or students related to ongoing online discussions, and changes in projects' ethical status. These reminders help ensure that no activity on the Zone occurs without the knowledge of the directors. This proves especially helpful if a student posts a question or responds to a query raised by his or her mentor or director.
The second method to help track student progress is an administrative management/reporting tool available only to the directors and support staff. The tool allows the directors to query the data of all students or just those for whom they are responsible for tracking. The interface provides the directors with a number of predefined and user-entered search parameter fields, which may be used in combination to track all aspects of progress associated with the SP (Figure 2). For example, the directors can quickly identify delinquencies or patterns such as which students lack mentor approval or are having difficulty with IRB or other ethical approvals. Once a query runs and the list of students meeting the selected criteria is generated, it is possible to execute different administrative actions for individual students or groups of students, such as sending student and/or mentor e-mails, visiting the student's SP Web site directly, or updating confidential administrative notes used solely by the directors.
The mentor provides the first level of evaluation of the student's work. Each student is also assigned an SP director who follows his or her progress. That director provides a second level of approval for the proposal and final report and ensures that the student and mentor are meeting appropriate milestones. If the student is completing an SP under an AOC, the AOC director also follows the student's progress and is part of the team that approves the proposal and final project. The SP directors are responsible for ensuring that all students meet acceptable levels of scholarship and thus work to maintain uniformity in the SP requirement.
Students receive interim grades of either satisfactory or unsatisfactory at three points during the longitudinal process. The first grade is given after the student submits both his or her proposal and the first quarterly report (which is due late in the second year). To receive a satisfactory grade at this point, the student must have an approved proposal and have worked with the mentor since submitting the proposal. The SP directors assign the second and third interim grades at the middle of the third year and the beginning of the fourth year, respectively. Whereas students may have periods of inactivity, UPSOM requires that they interact with their mentors at minimum every six months and provide some evidence that they have completed work on the project. This level of activity is required for a satisfactory grade. Students are able to remediate unsatisfactory interim grades.
The final (noninterim) grade for the SP is based on the student's final report which consists of a tangible product such as a journal article, educational instrument (e.g., a medical sign language dictionary or a brochure for teaching children about diabetes), or other item such as a medically related novella. The final report also includes a student-produced, two-page summary that addresses the significance of the project, the approach the student took to accomplish project goals, the student's level of independence, and the project's degree of originality. The SP directors assign a satisfactory or unsatisfactory final report grade in the spring of the fourth year. The specific areas the mentor and directors evaluate when they assess proposals and final reports partially align with the areas students address in their final two-page summary: project significance, student approach to the project, student independence, project originality, and student consideration of project limitations, alternative approaches, future directions, and ethical status. Directors and mentors may also use any information on the Zone to evaluate students' final reports.
SP initiative evaluation
Currently, UPSOM is evaluating the SP initiative in a number of ways. As part of the final report, the SOM collects information on student publications, presentations, and awards. In addition, students self-evaluate how their SPs contributed to their analytical skills. Finally, UPSOM has IRB approval to analyze all aspects of the information stored on the Zone to assess the initiative. Future evaluations will include following and documenting graduates' success in obtaining grant funding.
The most significant challenge UPSOM has faced while implementing the SP initiative has been orienting the students and mentors to the SP requirement. As described above, the requirements related to the SP are very different from other medical school courses. Mentors and students need to learn what the SOM expects of them and how to continue SP work during the third and four years, often in the absence of dedicated time. Another major challenge relates to ethical status (e.g., human participants protection). SP directors have found that students do not always receive appropriate advice from their mentors regarding the need for ethical review of their projects. The SP directors often have had to educate both parties, which has delayed the start of some students' projects. As expected, the difficulties in ensuring that all students receive appropriate mentoring are considerable. In the beginning, the absence of the IT infrastructure cost the directors significant time. The IT support has improved over time.
Implementation at the AMS
The SC program steering committee, which is chaired by a member of the faculty and comprises the SC manager (E.G.), the associate dean for medical education (P.G), and the chair of the Medical Curriculum Committee (J.B.), directs the program at AMS. Each concentration designates a director and up to two codirectors. Meetings of the concentration directors and the steering committee occur each semester. The SC program manager, who is also responsible for the summer assistantship program and recruitment of faculty mentors, provides administrative support. The SC program manager has an administrative assistant who assists with all aspects of the program. Both of these positions are currently funded from the AMS operating budget.
As is the case at UPSOM, orientation for students to the SC program begins before students enroll in the medical school. Thus, the provision of information, a critical part of the orientation process, takes place prior to medical school. Although students receive some information during first-year orientation, the most important information session occurs during the first semester in November. This session has evolved into an “informational fair” that allows first-year students to meet with all concentration directors and with select second- and third-year students involved in each concentration. At the informational fair, the program manager addresses issues such as the program basics, a suggested four-year timeline, the application process, and funding issues. A secondary event occurs in January for students interested in summer research, whether through or outside a concentration. During this January event, students who want to be involved in the SC program may have faculty review their proposals and funding applications. AMS also encourages concentration directors to hold discussions, which are open to any interested students, throughout the year to address issues (e.g., caring for an aging population, approaches to community advocacy, and the neuroscience of mindfulness) within their concentration as they relate to medicine.
As an elective program, AMS requires no specific courses; however, students do participate in activities related to their SC areas. These SC areas evolved out of faculty interest and student initiative. The Aging concentration, for example, grew out of faculty interest. It is one aspect of a broader SOM initiative to integrate geriatric and aging content into the medical curriculum and to provide students with an array of relevant activities (e.g., field trips to nursing homes and establishing longitudinal relationships with elderly patients). As part of this initiative, the concentration has an established infrastructure of dedicated faculty and staff that does not necessarily exist within other concentrations. The Physician as Communicator concentration evolved from a desire on the part of students to focus on writing and other means of communication; it grew from what originally began as a writing group. The Physician as Communicator curriculum now consists of a preclinical elective, open to all medical students, in which students share and critique one another's writing and participate in seminars and discussions about different forms of communication. The students participating in this concentration take a leadership role in the elective course, assisting the faculty director with the course logistics.
Mentors and mentoring
As is the case at UPSOM, each student identifies a primary advisor for his or her SP. This person undertakes a mentoring role that is analogous to that of a PhD student's primary advisor. A mentor's initial involvement occurs when the student is preparing his or her application to a concentration and/or, when applicable, to a summer assistantship. Mentors are required to contribute to the periodic progress reports that students make to the central administration of the program. The experience at AMS thus far has indicated, not surprisingly, that the mentor's role is critical to the student's progress and ultimate success in the program. Similar to at UPSOM, mentors at AMS assume primary responsibility for the student's conduct of the project, including ethical aspects, human or animal investigation approval, and all aspects of data collection and management.
AMS developed an SC Web site as a means by which to inform students and faculty about concentration areas and their curricula. Additionally, AMS made space available on its internal curriculum site, My Course. My Course allows concentration area faculty to post materials and other information for their concentration area participants. However, while these aspects of IT infrastructure are in place, the relatively small size of the program has allowed most administrative communication to occur effectively through committee meetings (i.e., at the aforementioned SC steering committee meetings and meetings of concentration area directors) and e-mail. To effectively view student progress, the SC program is currently looking into a tool similar to the Zone.
During the second year of the program, the program manager and SC steering committee developed a monitoring plan to ensure that students and their mentors are on track to achieve the goals and requirements of the program. Students complete periodic progress reports and, after reviewing them with their mentors, submit them to the program manager via e-mail. Concentration directors are responsible for ensuring that students prepare accurate and complete progress reports.
All students participating in the SC program complete a total of three progress reports—at the end of the summer between Years 1 and 2, in the spring of Year 2, and in the spring of Year 3. These progress reports require students to summarize their project work, to report on their accomplishments (including presentations, publications, awards, and grants), to outline their preparation for their final scholarly product, and to provide feedback on the program generally. The presentation of their scholarly product in Year 4 is considered the fourth and final progress report.
At the time of this writing, the SC program at AMS is in its fourth year. Thus, the first class to take part in the program has just reached the fourth year of medical school. The SC program manager and the steering committee have discussed the planned process for evaluating each student's SP and the characteristics of a successful project at length. Because of the multiplicity of topics and disciplines, they decided that primary responsibility for evaluating the SPs would rest with mentors and concentration directors. However, if requested, the steering committee could potentially provide assistance with evaluation. In addition, the program manager and the steering committee will assume the ultimate responsibility of certifying students as having completed a concentration. A further critical aspect of the evaluation of student progress at the start of Year 4 is the inclusion of relevant information in the student's medical student performance evaluation (“Dean's Letter”). Each student's mentor, the relevant concentration director, and the steering committee will jointly undertake compiling and composing this information.
AMS recognizes that evaluation of student work by various faculty may result, especially in the first few years of the program, in differing standards for students in different concentration areas. However, the SC program steering committee has facilitated multiple discussions with the concentration area directors around product evaluation during regular oversight meetings. Additionally, the SC program Web site includes a page on the dimensions of scholarship to help directors with the evaluation process, and AMS has developed a rubric using a modified version of these dimensions. Directors may choose to use this rubric in their evaluation of student work. Finally, AMS encourages directors to monitor student progress to ensure product quality before the actual submission of the scholarly work. In academic year 2009–2010, the first cohort of students will be submitting their work for evaluation; the SC program faculty look forward to refining the review process as the program matures.
SC program evaluation
At this early stage of its development, evaluation of the program has consisted largely of individual feedback from both students and mentors, as well as review during steering committee and concentration director meetings; however, future evaluations will include tracking students' publications, presentations, and grants, and comparing SC participants and nonparticipants on a variety of dimensions (e.g., career choice, residency matching results, future in academia).
Challenges in implementing the SC program at AMS, which are clearly part of the growing pains of this new program, are similar to some of those at UPSOM. Many of the most significant challenges have stemmed from the great variability between SC areas, making central regulation and administration difficult. Not only have faculty from different specialties and colleges (e.g., engineering, humanities, and medicine) been involved, but each area has developed its own sense of requirements. Most of the SC areas have faced difficulties in fostering ongoing communication with, and attendance at SC events by, second- and third-year medical students. A few of the SC areas have had leadership challenges as concentration directors either moved to other institutions or took on responsibilities that precluded their active participation in the SC program. In response to this, AMS now requires all SC areas to have more than one leader and, if possible, for smaller SC areas to consider merging. Finally, managing students who request admission into the SC program after their first year is challenging; given the prescribed requirements of this elective program, making up missed elements is difficult for students.
In this article, we report the implementation process for a scholarly activity at two schools, one with a required SP and the other with an optional SC. The goals for each school are similar, stressing critical thinking, self-directed learning, and mentorship, although the two schools' supervisory structures and reliance on IT differ. Further, the SC at AMS emphasizes the development of nontraditional scholarly activity, whereas the UPSOM's SP builds on and broadens traditional approaches while also allowing nontraditional approaches.
An important aspect in implementing the programs at each school is educating both students and mentors. Both institutions have observed that, even with multiple orientation sessions, educational efforts pay off only when the mentors and students start to work on projects together. The orientation sessions help in a general fashion, but answers to specific questions require additional attention once actual work begins. The ongoing interaction with faculty has been an important element in the success of the SP/SC effort. Although UPSOM provides an incentive for contributing to educational activities for mentors, mentors at both programs must contribute significant, uncompensated time to the students. Equally important has been the education of students as to what is meant by “scholarly” and “longitudinal.” The quarterly report process at UPSOM is proving to be critical in helping students understand what is required over time.
A critical aspect of a required SP is the culture of the school. Whereas over 60% of UPSOM students participated in some form of scholarly activity before the official start of the SP program, making an SP a required part of the curriculum has still been a significant change. In both programs, students may be expected to multitask by working on their scholarly activity, at least to some extent, even during clinical rotations. This reality also requires a slightly new mindset for clinical clerkship directors who, for example, may not expect a student to be reading anything outside of the required literature or patient-specific materials during the month. Interestingly, the SP at UPSOM was a voluntary research enrichment program for three years before it became a mandatory curricular element. Despite this, the transition to a mandatory SP required a major change both in students' expectations for doing independent research and in their perceptions of research in general. Even in the few years that the SP has been a curricular requirement, UPSOM has seen a change in the culture of the school overall and, in particular, a greater commitment by faculty to this program. At AMS, the addition of the SC program has affected culture by encouraging mentored and funded scholarship in areas beyond traditional biomedical investigation. Through support of student activities and a greater number of sponsored talks outside the curriculum, the program has emphasized the commitment of the school to rigorous scholarship in a broad array of areas related to medicine and patient care.
A significant portion of this article focuses on IT-related support and tracking needed for the required SP program at UPSOM. A critical ingredient for success is dedicated support personnel familiar with the entire curriculum. IT solutions, although efficient, still require human backup so that specific questions and problems can be addressed. A major consideration when planning or implementing an SC program is resource availability. Funding for the SP at UPSOM and SC at AMS is very different. UPSOM supports the program using internal funds; these funds represent a considerable investment. At AMS, donors support specific SCs. Such an approach to funding is likely not viable if the scholarly activity is required.
Notably, the implementation strategies for both programs have constituted an evolutionary process. The information systems and policies did not exist from the beginning. Constant revision has been the norm and will likely continue, although at a slower pace, into the future. Given this constant evolutionary process, continued communication via frequent meetings among all parties involved is essential. These meetings can foster a close relationship between groups such as IT and those designing the process. At each institution, implementation continues to require a team effort. In the end, the team effort and evolutionary processes have resulted in programs that benefit students. Both schools are proud of their programs and feel they can be models for other institutions.
The authors extend their thanks to Suzann Beardsley and the students and faculty who, through their participation during the early years of both programs, provided the impetus for their development. They would particularly like to show appreciation for faculty members, too many to list here, who give their time to mentor students.
The scholarly concentration program at Warren Alpert Medical School of Brown University has been supported by a generous grant from the Donald W. Reynolds Foundation to develop curriculum on aging and through support from Ms. Jill Braufman who has been instrumental in the development of curriculum, service, and research projects in the area of women's reproductive health, freedom, and rights.
1Kanter SL, Wimmers PF, Levine AS. In-depth learning: One school's initiatives to foster integration of ethics, values, and the human dimensions of medicine. Acad Med. 2007;82:405–409.
2Schor NF, Troen P, Kanter SL, Levine AS. The Scholarly Project Initiative: Introducing scholarship in medicine through a longitudinal, mentored curricular program. Acad Med. 2005;80:824–831.
3Parsonnet J, Gruppuso PA, Kanter SL, Boninger M. Required vs. elective research and in-depth scholarship programs in the medical student curriculum. Acad Med. 2010;85:405–408.
4Green EP, Borkan JM, Pross SH, et al. Encouraging scholarship: Medical school programs to promote student inquiry beyond the traditional medical curriculum. Acad Med. 2010;85:409–418.
© 2010 Association of American Medical Colleges
5Bierer SB, Chen HC. How to measure success: The impact of scholarly concentrations on students—A literature review. Acad Med. 2010;85:438–452.