Over the past two decades, medical schools throughout the United States have devoted an increasing amount of curriculum time and resources to training medical students in ambulatory settings where they are supervised by physician–preceptors.1–4 Assessment of medical education in the ambulatory setting has proliferated since the 1990s, particularly as this education affects choice of a residency program.1–9 Systematic reviews of educational programs in the ambulatory setting revealed that ambulatory clinics provided a more active learning environment than did the inpatient setting and that students were more satisfied with curricular experiences in rural settings compared with those in metropolitan settings. It was also noted that cross-institutional studies with common educational goals were not the norm, and that evaluative studies with rigorous research designs were lacking.1,10 There was no conclusive or definitive evidence that ambulatory experiences including those of the Generalist Initiative and Title VII were consistently or significantly associated with graduates’ primary care career choices.11–13 In general, the following variables were associated with choosing a career in primary care: female gender, growing up in a rural town, and participation in a rural ambulatory-medicine program.9,13–15 Requiring students to participate in mandatory primary care preceptorship experiences or overemphasizing the benefits of a generalist education at the expense of other preclinical curricula activities appeared to produce negative attitudes toward primary care among medical students and faculty.16
Since the late 1960s, most states have made specific efforts to support family medicine as a way of training primary care physicians for communities.10 However, there are few studies of the effect of statewide programmatic efforts on specialty choices. Three evaluation projects originating at the state level have been undertaken to investigate medical career choices based on a student’s experience during his or her primary care ambulatory rotations. The reported outcomes of these initiatives on primary care career choices provide some limited evidence of a preference for practicing generalist specialties. Most of the medical schools in Pennsylvania began to track their graduates’ career choices using a longitudinal database that was analyzed to determine the impact of the Generalist Initiative in the choice of a primary care residency. The results of this statewide initiative have not been published, but the program emphasized early primary care curricular experiences as an effective method of increasing the number of generalists in the medical community and expanding people’s access to medical care.17 Alternatively, reports from primary care physicians in Kentucky, who were students before the introduction of ambulatory primary care clerkships, revealed that they preferred practicing medicine in rural settings and made early career decisions to enter the family practice field. More than 50% of Kentucky’s medical school graduates eventually pursued careers as generalists.18 Statewide data from family physicians in Colorado who had worked at a decentralized health-education center revealed that they practiced primary care in rural Colorado 15% more frequently than those who had not worked at the center.19
In 1979, the state of Texas responded to its need for more primary care physicians by establishing a coordinated, statewide, interinstitutional ambulatory preceptorship training program. The broad goal of the Texas Statewide Family Practice Preceptorship Program (TSFPPP) was to offer ambulatory training in the offices of family physicians for as many medical students as possible to increase the number of students who choose careers in family practice or primary care. More specifically, the preceptorship was directed at providing preclinical and clinical students with primary care exposure that included observing physicians and interacting directly with patients. Through the TSFPPP, students are exposed to the full range of inpatient and outpatient family practice experiences, the difficulties and rewards of maintaining a family medicine practice, and the relationships between family physicians and other health care providers and the community. Funds from the Texas Higher Education Coordinating Board were awarded to the TSFPPP to support a central state administrative office, provide stipends to defray the students’ expenses, and support primary care faculty development activities.20,21 TSFPPP programs were made available to preclinical and clinical medical students from eight Texas medical schools: The University of Texas Medical School at Houston, The University of Texas Southwestern, The University of Texas Medical School at San Antonio, The University of Texas Medical Branch in Galveston, Baylor College of Medicine, Texas Tech University, Texas A&M University, and North Texas College of Osteopathic Medicine. Each school had an institution-supported TSFPPP staff member who helped coordinate the program for enrollees from that school. All preceptors served voluntarily and were unpaid.
In 1986, a family practice clerkship in the third year of medical school of at least four weeks became mandatory for all medical students in Texas. The TSFPPP continued to offer students from all eight schools voluntary four- and eight-week experiences in preclinical and clinical ambulatory family practice. The voluntary preceptorship and required family practice clerkships at each school shared the common goal of encouraging careers in family practice in Texas. In 1999, the Texas Higher Education Coordinating Board, which funds the state’s higher education curricula, committed funds for the first longitudinal evaluation of the TSFPPP. In this paper, we describe the evaluation questions that were asked, the methods by which the interinstitutional evaluation was accomplished, the results of the evaluation, and our conclusions.
Generating Evaluation Questions
At the annual meeting of the TSFPPP in 2000, program users (administrators from each school, student members of the Family Medicine Student Association, and representatives of the coordinating board that funds the TSFPPP and of the Texas Academy of Family Physicians) were encouraged to discuss and identify important aspects about the TSFPPP program to evaluate its efficacy in meeting defined goals. Preceptors also were informed about the evaluation process. After reviewing the program goals and the evaluation protocols administered by all eight schools in the program, the program users approved four questions to be used in evaluating the TSFPPP:
- ▪ Question 1: Is there a relationship between participating in the TSFPPP and selecting family practice as a career choice?
- ▪ Question 2: Is participation in the TSFPPP preceptorship associated with a primary care career choice?
- ▪ Question 3: Does TSFPPP participation and the gender of the student have an association with the choice of each specialty—family practice, internal medicine, pediatrics, and obstetrics/gynecology—accredited as primary care in Texas?
- ▪ Question 4: Has the TSFPPP increased the number of family practice physician–preceptors who participate in the training of preclinical and clinical medical students?
All study data were collected and entered into the electronic preceptor database, which is located in and supported by the TSFPPP’s office at The University of Texas Health Science Center at Houston. Each participating institution submitted information that included the student’s name, gender, year of graduation, residency program selected at graduation, and location of their residency program. When specialty-selection data were not available in an electronic format, the TSFPPP preceptorship staff obtained the institutions’ graduation programs and entered the data into an electronic file. A unique identifying number was given to each graduating student, and the first data file was developed from these students’ graduation information. A second data file was created that contained the student’s name, year of participation in the TSFPPP, whether he or she participated in a preclinical or clinical rotation, and the preceptor’s name. This file served as a link between the database containing information about the student’s year of graduation and specialty choices and the database containing preceptor information.
Population and Data Analysis
The population consisted of 10,081 (96.0%) of the 10,506 students who graduated from eight Texas medical schools from 1992–2000 and for whom there were complete data regarding their choice of primary care specialty. Of the 10,081 students, 63.0% were men and 37.0% were women, and 5,898 (58.5%) chose one of four specialties that the Texas Statewide Health care Coordinating Council considers as primary care. Specifically, 19% (1,892) of the students chose family practice, 21.8% (2,199) chose general internal medicine, 6.5% (661) chose obstetrics/gynecology, and 11.4% (1,146) chose pediatrics. However, 41.5% (4,183) of the 10,081 students selected non–primary care specialties. A fifth primary care category in Texas, general practice, was not available as a residency choice during the period studied.
From 1992–2000, an average of 724 preceptors were available to teach in the preclinical or clinical TSFPPP. Of these, an average of 377 preceptors each year actually taught medical students. The distribution of family preceptors across counties correlated (r = .62) with the distribution of primary care physicians in the state.21 The mean number of students accepted into the TSFPPP each year was 1.2 per preceptor. Preceptors reported an average tenure of 11.9 years in their communities. Of the available preceptors, 83.0% were men, 17.0% were women, 20.6% practiced in rural counties, and 79.4% practiced in metropolitan communities.
Descriptive statistics (means and standard deviations) were calculated for continuous variables, whereas frequencies and percentages were computed for categorical variables. The Pearson chi-square test was used to compare the homogeneity of frequency of choice of specialty and participation in the TSFPPP. Logistic regression was used to identify the factors that were associated with the students’ choice of a specialty. All analyses were done using SAS software (Version 8.0, SAS Institute Inc., Cary, North Carolina).
Question 1: Is There a Relationship between Participation in the TSFPPP and Selecting Family Practice as a Career Choice?
All 10,081 students with complete data who graduated in 1992–2000 from one of Texas’ eight medical schools were included in the analysis. All graduates had completed a third-year family medicine clerkship. We compared the proportion of students who participated in the voluntary TSFPPP and chose family practice residencies with those who did not participate and chose family practice residencies. Of the students who participated in the TSFPPP, 27.9% (703 of 2,517) chose family practice residencies. Of the students who did not participate, only 15.7% (1,189 of 7,564) chose family practice. This is shown graphically in Figure 1. Students were classified into three groups: preclinical, clinical, and both, regardless of the number of times they participated in each preceptorship. Each year over the nine-year period, there was an increased tendency for students to choose a family practice residency if they participated in the preclinical (odds ratio [OR], 1.62; 95% confidence interval [CI], 1.41–1.87), clinical (OR, 2.31; 95% CI, 1.99–2.68), or both TSFPPP rotations (OR, 4.98; 95% CI, 3.75–6.68), compared to the control group of students who did not participate in the TSFPPP but who chose a family practice residency (see Figure 2).
To gain more specific information about students’ reactions to the TSFPPP, we summarized the ratings of the preceptorships by students who participated from 1995–2000 and who responded to an identical questionnaire at the close of the educational program. A five-point Likert scale, from strongly disagree to strongly agree, measured 25 items that described perceptions of the preceptorship. Responses of agree and strongly agree were considered positive perceptions of the TSFPPP. We also analyzed 972 written comments from the students about their experiences during the same years. From the questionnaires, we found that over 90% of the students had positive perceptions of the program. Written comments reflected those positive perceptions.
Question 2: Is Participation in the TSFPPP Preceptorship Associated with a Primary Care Career Choice?
In the nine graduating classes from 1992–2000, there were 1,667 (66.2% of 2,517) students who chose a primary care specialty after participating in TSFPPP activities. At the same time, there were 4,231 students (55.9% of 7,564 graduates) who chose one of the primary care specialties without participating in any TSFPPP rotation. The percentages of students with TSFPPP experiences who chose one of the four primary care specialties were: 27.9% family practice, 21.8% internal medicine, 9.3% pediatrics, and 7.2% obstetrics/gynecology. In comparison, 15.7% of nonparticipants chose family practice, 21.8% chose internal medicine, 12.0% chose pediatrics, and 6.4% chose obstetrics/gynecology. Thus, the major TSFPPP contribution to increased primary care residency choices was due to the increased choices of family practice residencies.
Question 3: Does TSFPPP Participation and the Gender of the Student Have an Association with the Choice of Each Specialty—Family Practice, Internal Medicine, Pediatrics, and Obstetrics/Gynecology, Accredited as Primary Care in Texas?
We found significant differences between men and women with respect to choice of a primary care specialty. In general, men were less likely to choose a primary care specialty than were women (OR, 0.55; 95% CI, 0.50–0.60). Specifically, men were less likely to choose pediatrics (OR, 0.46; 95% CI, 0.40–0.52), and obstetrics/gynecology (OR, 0.44; 95% CI, 0.38–0.52) as their residency. There were no significant differences with regard to gender in the choice of family practice or general internal medicine residencies.
Question 4: Has the TSFPPP Increased the Number of Family Practice Physician–Preceptors Who Participated in the Training of Preclinical and Clinical Medical Students?
Although preceptors volunteer their professional services, the number of preceptors who registered with the TSFPPP to supervise medical students increased steadily to 779 between 1991, when the TSFPPP began to collect data about preceptors, and 2000. There was a 44.0% increase in preceptors willing to precept medical students over the nine-year period. An increase was reflected for both the number of preceptors willing to supervise preclinical students (34.0% increase) and the number of preceptors willing to supervise clinical students (47.9% increase) from 1991–2000 (see Figure 3). Students were placed with 724 of these registered physicians.
To our knowledge, this is the most comprehensive evaluation of a statewide preceptorship program designed to improve the number of family practice residency choices. The process of evaluating the program, supported by the Texas Higher Education Coordinating Board since 1979, provided an opportunity for the eight schools’ representatives to identify specific evaluation questions related to shared TSFP PP goals. Our study also resulted in a new, centralized statewide specialty choice database with a large number of students. This database contains information about preceptors and their practices as well as information about students, their TSFPPP rotations, and their residency choices. The online preceptor database (〈www.familypracticepreceptorship.org〉) is accessible and useful to both students interested in TSFPPP curriculum offerings and to TSFPPP physician members who join a large online ambulatory-based teaching network. In a few short months, the database became a very useful resource that is used, among many purposes, for the recruitment of students and preceptors into the program. We believe that we have developed a mode for evaluating medical education programs that includes a cooperative decision-making process about the evaluations and an Internet-data collection process. The successes in analyzing the outcomes of this interinstitutional program depended equally on the achievement of strong support from the Texas Higher Education Coordinating Board and the involvement of all eight medical schools in Texas.
As a result of our retrospective program evaluation, we can now begin to answer the four questions posed. Stated simply, there is a significant association between participation in the TSFPPP and the selection of a family practice residency. The greater the level of participation, the more likely the student is to choose a family practice residency (Question 1). Multiple factors influence the very complex decision process involved in a student’s choice of a residency and it is difficult to identify all of them.22 Like Fryer and his colleagues,20 we were unable to obtain complete information about each student from the participating medical schools and we were unable to gather students’ opinions about primary care before or immediately after matriculation, before choosing a preceptorship, or at other times during medical school. We also were unable to assign students randomly to groups that participated or did not participate in the TSFPPP. We cannot conclude from our study that the preceptorship experience is an independent factor in family practice career choices; however, the positive associations demonstrated in our study, the students’ positive perceptions about their preceptorships, and their written comments are consistent with the TSFPPP being contributory. It is possible to conclude that the 27.9% of students who chose family practice after doing the TSFPPP were provided with clinical experiences that helped them cement their initial interest in family practice as their career. It is also possible that the 15.7% of students who chose family practice as their career but did not do a preceptorship actually were more predisposed to family practice. They may have felt that they didn’t need to do a preceptorship to solidify that career choice.
If the students who participated in the TSFPPP had selected family practice residency at the same rate as had those who did not participate, over 300 fewer students would have chosen family practice careers. With funding just over $300,000 per year, TSFPPP’s investment of less than $9,000 per additional student selecting family practice may be considered cost effective. In our study, there was an association between participating in the TSFPPP rotations and choosing a primary care residency (Question 2). Choices of family practice residencies were the main reason for this association. Thus, the TSFPPP was mainly associated with the family practice profession. This finding supports earlier studies that concluded that grouping primary care specialties for analysis can obscure differences in career choice and lead to an overestimation of physicians who will choose family practice. Choosing family practice as compared to choosing other primary care specialties is unlikely to lead to subspecialization and may contribute more directly to the primary care physician pool.21–23
Conclusions from the analyses of Questions 2 and 3 were that the likelihood of choosing a primary care specialty following TSFPPP participation, as compared to nonparticipation, varies by specialty and gender. Our results comparing men’s and women’s preferences for primary care residencies were consistent with an earlier comprehensive study except for our finding of no difference between the choice of family practice by men and women.13 It will be interesting to see if this tendency remains as more women enter medical school in Texas. The finding that the choice of family practice was the major contributor to primary care residency choices supports the interpretation that the TSFPPP is mainly associated with family practice.
Specifically, responses to Question 4 suggested that recruitment of unpaid volunteer preceptors in Texas continues to expand despite the increasing time demands that have been placed on family physicians. Busy preceptors continue to teach because it is gratifying and the TSFPPP has had great success in recruiting willing physicians.24–27
Nationally, as well as in Texas, the debate continues as to whether the supply of generalist or primary care physicians, particularly those working in metropolitan areas, is now adequate.28 Still, Texas remains at the lower benchmark of what is considered minimum nationally. Indeed, while the Texas population has grown substantially since 1999, its supply of 57 to 66 physicians per 100,000 population has slipped below the national benchmark of 60 to 80 physicians per 100,000 population. There is a particular need along the Texas–Mexico border for physicians.29 Adding to this concern is the cycle of decline in match rates for family practice residency programs.30,31 Thus, it is particularly important that we continue to nurture the preceptorship program as a means of recruiting Texas’s graduating students to family practice residencies and responding to the health care needs of the American population. It is also important that we continue to evaluate the TSFPPP and seek improvements for achieving its primary goal of producing family physicians for our state. Only by taking these two actions will we understand the TSFPPP’s outcomes precisely and have the ability to take more purposeful steps toward fulfilling the educational needs of our medical students and the primary care workforce needs of the state of Texas.
Project support and funding was provided by the Texas Higher Education Coordinating Board.
1.Kurth RJ, Irigoyen MM, Schmidt HJ. Structuring student learning in the primary care setting: where is the evidence? J Eval Clin Pract. 2001;7(3):325–33.
2.Biddle B, Siska K, Erney S. A promising approach to teaching primary care in the ambulatory care setting. Acad Med. 1992;67(7):457.
3.Fincher RM, Lewis LA, Jackson TW. Why students choose a primary care or nonprimary care career. The Specialty Choice Study Group. Am J Med. 1994;87(5):407–9.
4.Irby DM. Teaching and learning in the ambulatory care setting: a thematic review of the literature. Acad Med. 1995;70(10):898–931.
5.Retchin SM, Boling PA, Nettleman MD, Mick SS. Marketplace reforms and primary care career decisions. Acad Med. 2001;76(4):316–23.
6.Herold AH, Woodard LJ, Pamies RJ, et al. Influence of longitudinal primary care training on medical students’ specialty choices. Acad Med. 1993;68(4):281–4.
7.Zachary TE, Smith-Barbaro P. Value of a family medicine preceptorship/clerkship to students, preceptors, and communities: observations from a 25-year old program. Fam Med. 2001;33(7):500–1.
8.Wartman S, Davis A. Wilson M, Kahn N, Sherwood R, Norwalk A. Curricular change: recommendations from a national perspective. Acad Med. 2001;76(4 suppl):S140–S145.
9.Kassebaum DG, Szenas PL, Schuchert MK. Determinants of the generalist career intentions of the 1995 graduating medical students. Acad Med. 1996;71(2):198–209.
10.Ogrinc G, Mutha S, Irby DM. Evidence for longitudinal ambulatory care rotations: a review of the literature. Acad Med. 2002;77(7):688–93.
11.Cohen JJ, Whitcomb ME. Are the recommendations of the AAMC’s task force on the generalist physician still valid? Acad Med. 1997;72(1):13–6.
12.Rabinowitz HK, Diamond JJ, Markham FW, Paynter NP. Critical factors for designing programs to increase the supply and retention of rural primary care physicians. JAMA. 2001;286(9):1041–8.
13.Bland CJ, Meurer LN, Maldonado G. Determinants of primary care specialty choice: a non-statistical meta-analysis of the literature. Acad Med. 1995;70(7):620–41.
14.Gazewood JD, Owen J, Rollins LK. Effect of generalist preceptor specialty in a third-year clerkship on career choice. Fam Med. 2002;34(9):673–7.
15.Levy BT, Hartz A, Merchant ML, Schroeder BT. Quality of a family medicine preceptorship is significantly associated with matching into family practice. Fam Med. 2001;33(9):683–90.
16.Steele D, Steyer T, Norwalk A. What did we learn about student and faculty “backlash” to the Interdisciplinary Generalist Curriculum Project? Acad Med. 2001;76(4 suppl):S61–S67.
17.Rabinowitz HK, Veloski JJ, Aber RC, et al. A statewide system to track medical students’ careers: the Pennsylvania model. Acad Med. 1999;74(1 suppl):S112–S118.
18.Blue AV, Donnelly MB, Harrell-Parr P, Murphy-Spencer A, Rubeck RF, Jarecky RK. Developing generalists for Kentucky. J Ky Med Assoc. 1996;94(10):439–45.
19.Fryer GE, Stine C, Krugman RD, Miyoshi TJ. Geographic benefit from decentralized medical education: student and preceptor practice patterns. J Rural Health. 1994;10(3):193–8.
20.Texas Statewide Health Coordinating Council. Texas Health Care Workforce update 〈http://www.texasshcc.org/4shp.pdf
〉. Accessed 12 August 2003. In: 2003–2004 Texas State Health Plan Update. Austin, Texas: Texas Statewide Health Coordinating Council, 2002:75.
22.Pugno PA, McPherson DS, Schmittling GT, Kahn NBJr. Results of the 2002 National Resident Matching Program: family practice. Fam Med. 2002;34(8):584–91.
23.Rabinowitz HK, Xu G, Veloski JJ, et al. Choice of first-year residency position and long-term generalist career choices. JAMA. 2000;284(9):1081–2.
24.Ricer RE, Van Horne A, Filak AT. Costs of preceptors’ time spent teaching during a third-year family medicine outpatient rotation. Acad Med. 1997;72(6):547–51.
25.Vinson DC, Paden C, Devera-Sales A, Marshall B, Waters EC. Teaching medical students in community-based practices: a national survey of generalist physicians. J Fam Pract. 1997;45(6):487–94.
26.Grayson MS, Klein M, Lugo J, Visintainer P. Benefits and costs to community-based physicians teaching primary care medical students. J Gen Intern Med. 1998;13(7):485–8.
27.Usatine RP, Tremoulet PT, Irby D. Time-efficient preceptors in ambulatory care settings. Acad Med. 2000;75(6):639–42.
28.Cooper RA, Getzen TE, McKee HJ, Laud P. Economic and demographic trends signal an impending physician shortage. Health Aff (Millwood). 2000;21(1):140–54.
29.Statewide Health Coordinating Council. 2001–2002 Texas State Health Plan update: ensuring a quality health care workforce for Texas. First biennial update of the 1999–2004 Texas State Health Plan 〈http://www.tdh.state.tx.us/stateplan01/default.htm
〉. Accessed August 12, 2003. Austin, Texas: Texas Department of Health, 2003.
30.McPherson DS, Schmittling GT, Pugno PA, Kahn NB Jr. Entry of US medical school graduates into family practice residences: 2001–2002 and 3-year summary. Fam Med. 2002;34(8):575–83.
© 2004 Association of American Medical Colleges
31.Green LA, Fryer GE Jr. Family practice in the United States: position and prospects. Acad Med. 2002;77(8):781–9.