Schor, Nina Felice MD, PhD; Troen, Philip MD; Kanter, Steven L. MD; Levine, Arthur S. MD
As technological advances extend our scientific armamentarium, as information becomes increasingly and readily available not only to professionals but to laypeople as well, and as imperatives to reduce medical errors and standardize treatment protocols drive the movement to computerize and codify therapeutic and diagnostic algorithms, it becomes more and more important that medical education foster analytical thinking, individualization of medical decision making, and critical evaluative skills. Medical schools around the country have explored and developed various didactic mechanisms by which to accomplish this. In the latter years of the 20th century, many such schools, nationally and internationally, embraced problem-based learning (PBL) as a way to enhance the analytical thinking, collaborative and collegial interaction, information retrieval capabilities, and communication skills of medical students.1 PBL sessions typically involve the presentation to students of a clinical case; the students themselves then generate hypotheses, answerable questions whose answers would perhaps help test the hypotheses, and, ultimately, literature-based “research” that would answer the questions. The challenges of using PBL for this purpose include the tendency of “traditional” clinician–educators to lecture, or at least respond to questions with declarative answers, and the tendency of students to believe implicitly everything that is in print, including that unpoliced body of information that lives in the ever-widening lanes of the information superhighway.2,3
Even before PBL became popular in U.S. medical education, a few institutions around the country instituted a requirement for students to undertake a scholarly project of some sort and to submit for faculty evaluation a final product (usually a written and/or oral report) that reflects the original thinking and results engendered in the scholarly project. Encouraging students to perform and mentoring them through the development and presentation of a scholarly body of work in some measure serves the purpose of fostering independent and analytical thought and communication skills, yet most institutions have not systematized either the preparation of students for this enterprise or the preparation and monitoring of mentors in its execution. Eleven of the 19 institutions we identified with scholarly project initiatives ascertainable on their current Web sites encourage but do not require performance of a scholarly project for graduation from medical school (see Table 1). Only Vanderbilt University School of Medicine currently requires students to take a course that reviews broadly such topics as study design, analysis and critical evaluation of biomedical or sociomedical literature, and preparation and presentation of scholarly information for professional and public consumption. Even the Vanderbilt program implements these principles in performance of research as an elective activity either during the summer between students' first and second years or during a year off from medical school.
In order to address this lacuna, the University of Pittsburgh School of Medicine developed a novel approach to introducing scholarly work in medicine, which was implemented with the entering class in July 2004. We describe herein the Scholarly Project Initiative, a component of the Scientific Reasoning and Medicine curricular sequence. Critical steps toward the conceptualization and development of the Scholarly Project Initiative included implementation of a pilot program, the Longitudinal Enrichment Program for Biomedical Research, beginning in July 2001, and development of the didactic components of the Scientific Reasoning and Medicine block, including the Introduction to Medical Decision Making course (first given in July 2003 and in revised form in July 2004) and the Methods and Logic in Medicine course (first given in July 2004). We therefore describe these curricular elements as well. The Scholarly Project Initiative is a template for a comprehensive and longitudinal approach to fostering scholarly pursuits and original, analytical thought in medical students.
The Pilot Program
In the academic year 2001–02, largely as a pilot for the planned integration of the Scholarly Project Initiative into the curriculum for all University of Pittsburgh School of Medicine students, the school offered entering students an elective program for performance and presentation of a research project and involvement in research-related activities spanning the four years of medical school. In that year and in each of the two years that followed, a total of 25 students elected to enroll in the Longitudinal Enrichment Program in Biomedical Research. The elective Longitudinal Enrichment Program was closed for new enrollment in June 2004, in light of the requirement that all students entering the School of Medicine in July 2004 participate in the Scholarly Project Initiative. However, the program will continue to exist through June 2007, when all those who enrolled in it will have completed the program. Students enrolled in this program were required to
▪ participate in the first- and second-year weekly journal club and participate at least once in the Medical Student Summer Research Retreat and the Midsummer Research Symposium;
▪ participate in quarterly Student Research Mentored Focus Group meetings;
▪ participate in at least two “Survival Skills” workshops (e.g., grantsmanship, ethics of authorship, academic job hunting) over four years;
▪ conduct a scholarly project deemed sufficiently meritorious and substantive by the Program Executive Committee and conducted under the guidance of a faculty mentor; and
▪ deliver a poster and/or oral presentation at the University of Pittsburgh Medical Student Research Day.
The first- and second-year journal club, which is mandatory for students enrolled in the Longitudinal Enrichment Program, was attended by between 20% and 40% of each year's entire class. The journal club paper for each week is chosen by a faculty member who also teaches that week's required course, and the topic for discussion uniformly dovetails with the topics presented in class. This feature encourages students to think of the journal club as true enrichment rather than as an added burden.
The Medical Student Summer Research Retreat is a day-long meeting attended by all participants in the University of Pittsburgh Medical Student Summer Research Program and by all students enrolled in the pilot program. Each year for the past 20 years, approximately 40 University of Pittsburgh School of Medicine students have spent eight weeks engaged in mentored clinical, basic science, or outcomes research under the auspices of the Medical Student Summer Research Program. The retreat starts with a breakfast, and includes an outside keynote speaker, generally a physician who has made research a major part of his or her career, and a presentation by the associate dean for medical student research about the Longitudinal Enrichment Program and other venues and programs through which University of Pittsburgh School of Medicine students can incorporate research in a longitudinal fashion into their medical school activities. It concludes with a luncheon during which students meet in small groups with individual faculty members from the Program Executive Committee of the Longitudinal Enrichment Program. During the pilot program (i.e., in the years prior to implementation of the Scholarly Project Initiative), approximately 40 students attended the retreat each year. Outside speakers have included the chief of pediatric hematology–oncology at the University of North Carolina School of Medicine, the director of the Neonatal Brain Disorders Center at the University of California, San Francisco, and the medical director of the Kennedy-Krieger Institute of Johns Hopkins University.
The Midsummer Research Symposium is a half-day-long meeting where students attend and participate in a lecture delivered by the dean of the School of Medicine, who addresses the future of biomedical research both globally and at the University of Pittsburgh. Students also attend a more informal presentation and demonstration that deals with preparation of poster presentations, and participate in a small-group discussion, facilitated by members of the Program Executive Committee, of a series of cases that raise issues related to the ethics of authorship in biomedical research.
Student Research Mentored Focus Groups are quarterly informal meetings attended by members of the Program Executive Committee and students enrolled in the Longitudinal Enrichment Program. Students submit to the committee a brief, structured quarterly progress report prior to this meeting, and the topic for discussion at each meeting often grows out of issues raised by the students themselves in these reports. Specific topics discussed have included options for taking a year off from medical school to pursue research and programs and opportunities for doing so, the approach to the student–mentor pair that is not “working” as anticipated, preparation of project proposals for submission to the University of Pittsburgh Institutional Review Board or the Animal Care and Use Committee and the “wait” for the response, and issues involved with taking a voluntary program and making it a required component of the curriculum.
The “Survival Skills” sessions, from which the Longitudinal Enrichment Program students choose two, are part of a nationally renowned and emulated series of day-long, Saturday seminars and workshops on specific topics relevant to individuals who want to pursue a career of which biomedical research is a major component. The workshop series, initiated in 1985, has been featured in many national forums and is best described on the Web site developed by its founders and mentors.4
The University of Pittsburgh Medical Student Research Day presentation, a late fall event, involves poster presentations by some 40-odd students involved in research either over the summer or during the preceding academic year. Students intending to present at the event submit an abstract to the Program Executive Committee (regardless of whether they are part of the Longitudinal Enrichment Program), and these abstracts are each assigned two or three reviewers, each of whom give it a score (like those used by the National Institutes of Health [NIH]) based upon previously agreed-upon criteria. On the basis of these scores, four abstracts are chosen for oral presentation on Research Day and awarded a certificate and a cash prize (the Mirsky Award), and an additional ten are chosen to receive a certificate of merit.
Table 2 illustrates the topical breadth of the research projects undertaken by students in the Longitudinal Enrichment Program. It is too early to determine the impact of this program upon the careers of these students; however, the program has already produced a substantial number of Sarnoff Endowment, NIH-Cloisters, and Howard Hughes Program participants. Clearly, some of the success of the pilot program in this regard lies in its voluntary nature and its consequent selection bias. However, it remains to be seen whether the Scholarly Project Initiative will have the same degree of success. In fact, we expect that the broader way in which the Scholarly Project Initiative is defined relative to the Longitudinal Enrichment Program in Biomedical Research will allow it to achieve success across the broad spectrum of interests represented by the medical school class as a whole.
The New Curriculum
Analytical thought and rational decision making are the hallmarks of modern medical practice. The Scientific Reasoning and Medicine component of the new University of Pittsburgh School of Medicine curriculum grows out of the conviction that the abilities to read and critically evaluate medical literature and to think through a patient's medical issues from first principles are essential components of undergraduate medical education in the current era. This is true regardless of whether the student intends to pursue a career in academic medicine, be it based in research, teaching, or clinical service, or in private clinical practice.5,6 Furthermore, the design, performance, and presentation of an independent project and the critical evaluation of the projects of others are outstanding ways to develop and crystallize analytical thinking skills and the tools for rational decision making.6 As the Scholarly Project Initiative is conceived, students at the University of Pittsburgh School of Medicine will be required to read and critically evaluate scientific and medical literature, present a research proposal and get direct feedback from faculty and other students on that proposal, perform a research project under direct mentorship from an expert in that field, present the research project in both informal and formal collegial venues, and discuss, trouble-shoot, and critique the research and presentations of their colleagues in their medical school class. The aims of this component of the new curriculum are therefore to
▪ foster analytical thinking skills and the development of tools for rational decision making in future physicians;
▪ provide role models, mentorship, and guidance for students regarding careers that integrate research, teaching, and clinical service;
▪ present research and biomedically relevant scholarly pursuits to students as endeavors that often, but not always, involve collegial interaction;
▪ enhance the medical school culture of self-directed and peer-group-fostered learning; and
▪ enhance the oral and written communication skills of graduating medical students.
Five critical components of the new University of Pittsburgh School of Medicine curriculum will both prepare all students in a uniform way for undertaking a scholarly project and emphasize for them the generalizability and applicability to all aspects of medicine of the skills acquired and honed during performance of a scholarly project. These components are:
▪ First- and second-year courses
▪ Weekly journal club and discussion session
▪ Performance of a scholarly project (years 2–4) under the guidance of a faculty mentor
▪ Third- or fourth-year seminar
▪ Presentation of a written scholarly product
We discuss each of these components in detail below.
First- and second-year courses.
All University of Pittsburgh School of Medicine first-year students will take a course that presents and addresses biostatistics, study design, research and clinical ethics, and evidence-based medicine. These sessions will be reinforced and rendered practical by both the journal club and the student's performance of a scholarly project.
The course material specifically designed to prepare students for choice, design, execution, and presentation of a scholarly project will consist of a ten-week course entitled “Introduction to Medical Decision Making” that meets one afternoon per week. This course will introduce students to the elements of deductive reasoning, statistical analysis and study design, and critical analysis of the literature.
During the second semester of the first year and the first semester of the second year, all students will take the “Methods and Logic in Medicine” course. This course includes lectures, along with small group and journal club sessions described below, that introduce students to evidence-based medicine and clinical research study design and interpretation, as well as careers in academic medicine that integrate research and clinical service.
Weekly journal club and discussion session.
All University of Pittsburgh School of Medicine first- and second-year students will participate in weekly small-group (ten or 12 students) sessions that will rotate through four types of discussion, which we present below. Because no examination will be given for the small-group sessions, students will be graded in this component of the curriculum on the basis of attendance (e.g., no more than three unexcused absences allowed per semester) and participation (as judged by the faculty small-group facilitator) in a manner analogous to the Integrated Case Studies course we have previously described.1,7
Approximately one week ahead of each session, students will be given a research journal article whose content is directly related to the course material presented that month in class lectures. Each session will be facilitated by a faculty member, but the discussion will be led each time by a different student from the small group chosen at the session itself, so that all of the students come prepared to lead the discussion. Students are to concentrate on discussing not primarily the conclusions of the study, but rather the methods used, their appropriateness for answering the question posed, and the degree to which the conclusions as stated are direct outgrowths of the data presented.
Faculty member presentation.
Each small group will host a different faculty member, who will present a brief and informal lecture on his or her specific involvement in scholarly pursuits and the role it has played in shaping his or her career. For some, this may involve a description of laboratory or clinical research, but for others, it may involve informatics system development or preparing a meta-analysis in a particular clinical area. An effort will be made to configure the groups and faculty member assignments such that, over the course of the year, each group is exposed to an extremely wide variety of faculty scholarly pursuits and career styles. Students will be given ample opportunity to ask questions that will help them choose projects and interest areas and understand the virtues and drawbacks of each approach presented.
Each small group will host a different third- or fourth-year student, who will present a brief and informal lecture on his or her specific involvement in scholarly pursuits, with particular emphasis on choosing a mentor, project, and/or Area of Concentration,8 obtaining funding and institutional review board and/or Institutional Animal Care and Use Committee approval, and negotiating roadblocks or pitfalls that were encountered along the way. Students will be given ample opportunity to ask questions that will help them choose projects and interest areas and understand the virtues and drawbacks of each approach presented.
Practical aspects of scholarly pursuits.
Approximately one week ahead of these sessions, students will be given reading material relevant to the topic at hand. Each session will be facilitated by a faculty member, but the discussion will be led each time by a different student from the small group who will seek out additional material and faculty input that enriches and supplements that material. Examples of discussion topics for these sessions include choosing a mentor, designing a clinical trial, writing a paper, obtaining grant funding, and structure and function of the extramural NIH program.
Performance of a scholarly project.
All students must complete a project that is deemed sufficiently meritorious and substantive by the Scholarly Project Initiative Executive Committee—which includes faculty members conversant with clinical research, basic research, epidemiology and statistics, and principles of evidence-based medicine—or by an analogous committee whose members represent approved University of Pittsburgh School of Medicine Areas of Concentration, medical school degree-granting programs (e.g., MD-PhD, MD-MPH), or the Clinical Scientist Training Program.9,10 For medical students, this project begins with submission of an abstract by December of the second year of medical school and continues through submission of a final paper in March of the fourth year. For most students, the initiation of a scholarly project will take place in the context of an existing Area of Concentration, or as part of the Medical Student Summer Research Program or the Clinical Scientist Training Program. For this initiative to be successful, the number, variety, and rigor of available Areas of Concentration must increase, and the information provided to students about each of these areas must be comprehensive, available, and readily visible prematriculation through the early years of medical school. Another subset of students will perform their scholarly projects through a concomitant non-MD degree-granting program (e.g., PhD, MPH, masters of clinical research, MBA). In each of these instances, the mentor choice, the project, its execution, and its ultimate presentation will be overseen and judged by the executive committee of the individual Area of Concentration, members of the Clinical Scientist Training Program, or the degree-granting program involved. The Medical Student Summer Research Program provides another opportunity for students to develop a relationship with a mentor and fulfill the scholarly project requirements. For the small number of remaining students who are part of neither an Area of Concentration nor a degree-granting program, the mentor choice, the project, its execution, and its ultimate presentation will be overseen and judged by a faculty committee (a Study Section of sorts) convened specifically for this purpose.
The goals of the Scholarly Project Initiative intentionally support a conception of the scholarly project that is broad-based and inclusive. The following are examples of types of projects that, given sufficient quality of the work undertaken, will be deemed acceptable for fulfillment of this requirement:
▪ Conducting a mentored clinical research project. For this project type, there is no “minimum” data generation or technique acquisition requirement. For example, the design, implementation, and analysis of a clinical trial using appropriate methodology would fulfill the requirement. Alternatively, a student may analyze existing data collected by her or his mentor. Evaluation will be based upon demonstration of an understanding of the principles of research design and performance in general and of the subject of the student's project in particular.
▪ Generating a clinical case report and an associated comprehensive and critical review of the biomedical literature. For this project type, the expectation is that the student will generate a scholarly review of what is known about a particular clinical situation illustrated by the patient(s) presented. This review must include a critical appraisal and synthesis of the implications of the literature assembled.
▪ Performing a meta-analysis of previously performed studies aimed at answering an important clinical question. This will involve assembling existing literature, critically reviewing it, and using the assembled data to create a composite analysis of relevance to a particular question.
▪ Designing an instrument for clinical research, care, and/or teaching (e.g., an interactive Web site for patients; a questionnaire for a particular study) and writing a research grant proposal or institutional review board application aimed at beta-testing this instrument, including its reliability, validity, and generalizability. The instrument does not have to have been tested; however, a credible plan for testing it must be put forth.
▪ Critically evaluating a group of related, published primary research articles and writing a research grant, institutional review board application, or Institutional Animal Care and Use Committee application aimed at delineating and experimentally approaching the next level of question (i.e., the questions that follow from the information generated in those published research articles). For this type of project, the student must ask him- or herself, “From what I learned from these papers, what new questions or issues are raised, and how might I go about solving them?”
▪ Designing and executing a clinical study using survey methodology.
▪ Designing and conducting a health services research study aimed at identifying critical issues in and/or improving delivery of health care services to the community at large or to particular subpopulations therein.
▪ Completing a basic research project in molecular, cell, structural, developmental, or computational biology; in neuroscience, immunology, pharmacology, physiology; or in other basic scientific disciplines.
These project types are not exhaustive. The committees will expect to review requests from students for projects that do not fit any of these descriptions, and all will be reviewed on an individual basis. New categories of projects will be added to the list of examples above as they arise, or may even promote the establishment of new Areas of Concentration.
Research and other scholarly pursuits increasingly involve collaborations and collegial interactions. For this reason, it will be permissible for students to perform a scholarly project that is a small-group (maximum of five students) effort, provided a written statement is handed in both with the initial proposal and with the completed report detailing briefly the role played and contribution made by each participating student.
Third- or fourth-year seminar.
Students will complete of at least one course (four weeks in length) that includes sessions on scientific writing and research presentation. This may be one of the school's existing Integrated Life Sciences courses.
Presentation of a written scholarly product.
Students will give a poster and/or oral presentation, based on their scholarly projects, at the University of Pittsburgh Medical Student Research Day. Ideally, this will be a peer-reviewed, publishable report.
Additional components of the Scholarly Project Initiative
On a quarterly basis, each student will be encouraged to generate a brief progress report up to a page in length. This document will include “prompts” with several lines for response. The prompts will include such questions as:
▪ What is the project title (if known)?
▪ Who is the project mentor (if known)?
▪ What progress have you made towards formulating, executing, and/or preparing for reporting your project?
▪ What problems have you encountered?
▪ What assistance from the Scholarly Project Initiative Executive Committee (or other appropriate committee, i.e., Clinical Scientist Training Program Committee) might be helpful at this point?
▪ What are your aims and plans for the next six months?
The Scholarly Project Initiative Executive Committee or appropriate Area of Concentration or degree-granting body will review all of these progress reports, and, where indicated, will contact each student and/or mentor to encourage progress toward fulfillment of the requirement and to provide career and additional project mentoring for students. A “final” report will be due no later than three months prior to the each student's expected date of graduation. This will allow those students whose projects do not yet fulfill criteria for completion of the scholarly project requirement to receive detailed critiques and concrete expectations in enough time to complete their projects and have them evaluated by the committee one month prior to the expected date of graduation. Throughout the four-year evolution of each scholarly project, the review of each student's abstract proposal, progress reports, and final product will be facilitated by maintenance of all documents in an interactive “e-portfolio” that automatically alerts an individualized list of reviewers and participants to the submission of new material or modification of existing documents.
Summer research symposia.
Each year, just before the start of the mentored Medical Student Summer Research Program, the Medical Student Summer Research Retreat is held to introduce students to the program and to promote a sense of community as they embark upon their mentored projects. In the past, this retreat begins with an informal breakfast followed by two talks: one by a member of the Program Executive Committee of the Longitudinaly Enrichment Program in Biomedical Research, and one by an outside speaker who served as a role model for aspiring physicians who wish to make research a significant component of their careers. Following the talks, the students were divided into small groups to discuss their planned projects with a member of the Program Executive Committee.
As part of the Scholarly Project Initiative, all University of Pittsburgh first-year medical students will be encouraged to attend an analogous one-day retreat and its associated Midsummer Research Symposium, but this will not be mandatory.
Mentoring the mentors: team building.
As the Scholarly Project Initiative Executive Committee has begun informing both students and faculty about the goals and details of the Scholarly Project Initiative, it has become clear that there is a critical need for mentoring of faculty, particularly those who have not previously served as mentors for medical students. Faculty must engage in the process not only of mentoring their own student(s) but also in learning from and guiding one another as they do so. Faculty need to feel they have an important hand in the shaping and refinement of the Scholarly Project Initiative and the development of new mentors and mentoring strategies at the University of Pittsburgh. To this end, we sent letters to 268 faculty members of the University of Pittsburgh, both inside and outside of the School of Medicine, who either had a track record of mentoring students through independent projects or were junior faculty members engaged in notable research, asking if they would be willing to have their names listed on the school of medicine student Web site as possible mentors for students completing a scholarly project. The reaction to this letter was initially mixed; however, after we held three educational sessions for those faculty who agreed, and advertised and conducted a similar session for all faculty interested in learning about the Scholarly Project Initiative, well over 300 faculty members, many not on the original contact list, responded affirmatively. Faculty, students, Area of Concentration directors, and members of the Scholarly Project Initiative Executive Committee have reviewed and discussed a list of generic criteria by which to judge the quality of student-authored abstracts and final papers. Faculty have begun talking about serving as a mentor for the Scholarly Project Initiative as being a member of an elite “club,” rather than as yet another duty or rite of passage. A system is in development for monitoring the dialogue between each student and his or her mentor, having mentors meet semiannually with one another to discuss issues that have arisen, and having a member of the Scholarly Project Initiative Executive Committee or other senior-level mentor serve as a mentor to any faculty member experiencing significant difficulty with the student mentoring process. Another element of the system will be an electronic database in which to maintain and track outcomes relative to students' pursuit of academic careers, pursuit of fellowships after residency, and receipt of research grants by graduates of this program.
The Scholarly Project Initiative differs significantly from other medical school programs that include a research or independent study component in three major ways. First, to our knowledge no other medical school introduces students to scholarship in medicine with the thoroughness of the University of Pittsburgh's program. Only schools of medicine at the University of Pennsylvania, Vanderbilt University, and Stanford University include required coursework in their curricula designed to foster analytical thinking and an understanding of research design and evaluation (see Table 1). The medical school courses at Stanford University and the University of Pennsylvania focus on clinical research; the course at Vanderbilt University focuses on biomedical research. In contrast, the Scientific Reasoning and Medicine curriculum at the University of Pittsburgh School of Medicine leads students through a sequence of course-based learning (theoretical knowledge), examples (passively acquired practical knowledge), and creative implementation (actively acquired practical knowledge) in preparation for their scholarly projects. This initiative allows staged acquisition of the skills necessary for successful conduct of scholarly work, whether students' projects involve clinical, basic, or population-based research or a nontraditional scholarly endeavor.
Second, of the 11 other schools that require submission of a written product for research-project credit, only seven make this written product a graduation requirement. None of these include didactic sessions or course-based practice in scientific writing in their required curricula. This is a key component of the Scientific Reasoning and Medicine curriculum and Scholarly Project Initiative.
Third, none of the other programs that afford students the opportunity to perform a scholarly project have formal mentoring and team-building programs for faculty mentors. This is a critical element of the Scholarly Project Initiative that will ensure the quality of the student–mentor interaction and engage, encourage, and reward faculty mentors, as well as make the Scholarly Project Initiative a career-building experience for students and mentors alike.
The authors gratefully acknowledge the collegial support and helpful discussions offered in the course of the development of the Scholarly Project Initiative by Steven Graham, MD, PhD, and the Scholarly Project Initiative Executive Committee: Amber Barnato, MD, Michael Boninger, MD, Joan Harvey, MD, Allen Humphrey, PhD, Janine E. Janosky, PhD, Wishwa Kapoor, MD, Stephen L. Phillips, PhD, Margaret V. Ragni, MD, and Stephanie Studenski, MD. The authors also thank Ms. Yvonne Harlow and Ms. Suzann Beardsley for expert administrative support.