Carney, Patricia A. PhD; Eliassen, M Scottie MS; Pipas, Catherine F. MD; Genereaux, Stephen H. MD; Nierenberg, David W. MD
Nationwide, about 94% of medical schools use community preceptors as clinical teachers, especially in ambulatory settings.1 This shift has occurred for two reasons: (1) inpatient capacity has decreased and care delivered in outpatient settings has increased,2 and (2) community settings offer certain educational experiences that often cannot be taught in clinics in academic health centers.3 The bulk of medical care, approximately 95%, is now delivered in busy ambulatory settings,4 but teaching in ambulatory settings has its challenges.
Competition for patients by multiple levels of aspiring physician–learners complicates the educational process. Diverse educational settings are now commonplace in both undergraduate and graduate ambulatory education; however, the teaching associated with patient care in ambulatory settings can vary in content and quality.1 In addition, conditions seen and management of those conditions may differ based on the clinical diagnostic and treatment resources available. The roles and responsibilities of physicians in community settings, which are important elements of medical practice, are also likely to differ from those in academic settings. It may be difficult to apply consistent curricular objectives in different educational settings. This challenge can be mitigated by frequent and reciprocal communication between teaching sites and the academic institution, but achieving this is also difficult.
Understanding how various educational settings compare could help clerkship directors, preceptors, and learners plan how best to achieve educational goals in each setting and may identify important areas for faculty development. In addition, as underscored in a recent editorial on ambulatory care education,5 conducting research in ambulatory settings is of utmost importance because it will allow us to both understand and improve education in this challenging learning environment. Using our handheld computer documentation system, we conducted an observational study designed to evaluate three ambulatory teaching settings with different levels of affiliation to Dartmouth Medical School at Dartmouth—Hitchcock Medical Center: (1) academic medical center-based clinics (AMCs); (2) affiliated residency teaching sites (ARTs), sites where residents are taught but which are not located at the academic medical center; and (3) community-based practices (CBPs). Our goals were two-fold: first, to compare the educational content in clinical experiences with patient symptoms, disease states, and other clinical activities, such as patient counseling, and common procedures performed; and second, to compare the educational process that occurs between teachers and learners in these diverse settings.
The development and implementation of our handheld computer documentation system (ClinEdDoc) has been described elsewhere.6,7 Briefly, the system has two parts. First, using the common problem tally section, students record, on a patient-by-patient basis, the symptoms, medical conditions, and clinical activities (counseling and procedures) they see or perform in each encounter with all patients seen. Second, students use the teaching and learning documentation section to record information about the educational process that occurs between students and preceptors, including patient demographic information, level of student independence in taking a history or conducting a physical examination, amount of feedback students receive about clinical skill development, and presence of teaching about patient disease, diagnosis, and management. The teaching and learning documentation section is completed for each patient seen on one day per week. Once a week, students synchronize their handheld computers on campus, and data are cleaned and imported into a relational database that stores all clinical encounters recorded. As previously reported,8 the system has established reliability with an overall κ of 0.68 for interobserver agreement of variables included in the system.
All encounters included in the analysis were classified by setting (AMCs, ARTs, and CBPs). In general, faculty at the first two sites have regular academic appointments of instructor or higher in a clinical department of the medical school, and most are involved in both undergraduate and graduate medical education. Community-based preceptors are not based at or affiliated with our AMC and are not involved in the training of residents. These preceptors usually, but not always, have appointments as adjunct faculty members of a clinical department of the medical school (community and family medicine [FM]). The majority of CBPs in this study were within a 60-mile radius of the medical school. Using these categories, we created an indicator variable in the analysis file that identified each teaching site as an AMC, ART, or CBP for purposes of this study.
Our analysis included encounters documented by third-year medical students during their eight-week FM clerkship, which primarily involves care delivered in ambulatory settings. Clinical placements are determined by the clerkship director using a ranking of students’ top three choices and a listing of their past clerkship experiences. Preceptors practicing in AMCs and ARTs take students throughout the year. CBP preceptors take one to two students per year and indicate to the clerkship director their preferred months for teaching and months they are not available. CBP preceptors are least likely to have students during the summer months because most of these sites are small and cannot accommodate students owing to vacation schedules. Students in this clinical rotation spend four full days per week working with their preceptor(s) in the care of patients. On the fifth day, students return to campus and receive didactic curriculum on both clinical content (e.g., managing chronic illnesses such as diabetes, hypertension, asthma, and depression) and ambulatory setting content (e.g., office practice and evidence-based medicine).
Data were based on encounters that occurred between August 28, 1997 and June 19, 2002. Encounters were excluded if they were collected from military training sites (very few) or from distant community sites (Bethel, Alaska or Tuba City, Arizona), because these Indian Health Service settings are very different from the local CBPs of interest in this analysis. Data from the common problem tally section were calculated as the mean number of occurrences per student per week. To assist with interpretation of data, we additionally calculated a ratio of educational activities (per student per week) in AMCs (as the reference group) versus the ratios in ARTs and CBPs. This calculation provides the percentage over or under what occurred in AMCs by educational setting for each variable, with 1.00 indicating no differences between that site and the reference site. Data on the educational process was categorical and reflected the proportion of total encounters within each category.
Descriptive statistics were used to characterize the students, preceptors, and patients seen in encounters. Univariate analysis included chi-square tests to assess differences in categorical variables and analysis of variance to assess differences in continuous data among the three settings. We anticipated patient age would likely influence clinical educational content and processes,10 so we conducted multivariate analysis of variance with patient age as a covariate in the model whenever possible. Patient age is not collected on the common-problem tally section, and there is no link to the teaching and learning documentation section, which does collect patient age. Thus, we were not able to account for patient age in the analysis of common problems. All tests were two-tailed with an α value set at 0.05 to determine statistical significance.
A total of 9,378 encounters fit the inclusion criteria for this analysis. Of these, 2,782 (29.7%) occurred in AMCs, 1,413 (15.1%) occurred in ARTs, and the remaining 5,183 (55.3%) occurred in CBPs. Table 1 compares characteristics of the students contributing data and the preceptors and patients with whom they worked while on their rotations. A total of 164 students provided data on educational experiences; the mean number of encounters per student was 57.2. Of the 164 students, 36 (22.0%) provided data on educational experiences at AMCs (mean number of encounters per student = 63.9), 51 (31.1%) provided data obtained at ARTs (mean = 27.6), and 77 (47%) provided data from CBPs (mean = 65.8). The number of encounters contributed per student differed significantly by training site (p < .001).
Students contributing data worked with 26 AMCs preceptors (mean number of encounters per preceptor = 88.4). The number of preceptors teaching students in ARTs was more than double this (68) and, subsequently, the mean number of encounters per preceptor was much lower (20.7 per student). In CBPs, students contributed data on 42 preceptors (mean number of encounters per preceptor = 120.7; p < .01).
The gender distribution was similar in AMCs, but it differed in the other two settings. Male preceptors dominated in CBPs and, to a lesser degree, in AMCs, and female preceptors dominated in ARTs. Patients seen in all settings were more likely to be female. Lastly, the proportion of visits where a student saw the same patient more than once was lowest in AMCs compared with ARTs and CBPs. The ages of patients seen in the three settings also differed statistically, when calculated as a continuous as well as a categorical variable (see Table 1).
Students saw a total of 56,965 symptoms and conditions. Of this total, 21,798 (38.3%) common symptoms were compared by educational setting (see Table 2). Statistical differences were observed among the three settings for seven common symptoms (36.8%): back pain, cough, dyspnea, ENT (ears, nose, throat) problems, fatigue, knee pain, and vomiting. With one exception (vomiting), all of these were least likely to occur in ARTs. Back pain, cough, fatigue, and knee pain were most likely to occur in AMCs, and dyspnea and ENT symptoms were most likely to occur in CBPs.
Students’ saw a total of 35,167 conditions, which were compared by educational setting (see Table 3). Statistical differences were noted in 15 (41%), with all of these occurring least often in ARTs. Six of these 15 conditions (40.0%) occurred most often in AMCs: asthma/allergies, depression, dermatologic conditions, developmental disorders/delay, hypertension, and musculoskeletal disorders. Eight (53%) occurred most often in CBPs: angina/coronary artery disease (CAD), congestive heart failure, type 2 diabetes, genitourinary conditions, hyperlipidemia, lower respiratory disorders, and otitis media. Endocrine disorders occurred at similar rates in both AMCs and CBPs, but were so much less likely to occur in ARTs that this item achieved statistical significance in our analysis.
Table 4 compares the counseling skills and procedures students reported performing in the three educational settings. Counseling skills were performed most often in AMCs, with two (exercise and weight control) being statistically higher in this setting. Pelvic examinations were performed similarly across the settings, but the category containing other procedural skills (e.g., flexible sigmoidoscopy, suturing, skin biopsy) were less often performed at AMCs and ARTs compared with CBPs.
Table 5 compares educational process variables among the three settings. Students were more likely to be involved in health maintenance and acute care visits in AMCs and ARTs compared with CBPs, although they were more likely involved with chronic conditions in CBPs compared with AMCs and ARTs. Of nine components of the physical examination that the students performed, statistical differences in performance were noted in six (67%). Abdominal, pulmonary, HEENT, gynecologic, and neurologic examinations were most often performed in ARTs and cardiovascular examinations were most often performed in CBPs.
The type of teaching and learning that occurred in history taking and physical examinations is also compared in Table 5. Students were more likely to observe their preceptors conduct histories and physical examinations or do them together in AMCs, and they were more likely to perform them unobserved in ARTs. No differences were observed among settings in the amount of feedback students reported receiving on clinical skills, though students did report receiving more teaching about disease in AMCs versus ARTs and CBPs and more teaching about patient management in ARTs versus AMCs and CBPs
Our observational study has revealed several important findings that can be framed using the structure of the learning environment described by Bowen and Irby9 in their recent article on the quality and costs of education in ambulatory settings. This structure refers to the general learning environment, educational culture, characteristics of teachers, and learning resources.
We found important similarities as well as differences in the general learning environment. For the majority of patient symptoms (> 63%) and conditions (> 59%) seen by third-year medical students in any of the settings, the frequency of their occurrence did not vary significantly. This suggests that curricular goals for patient workups, symptom analyses, and review of systems are likely to be consistent for these symptoms and conditions across teaching sites.
Students in CBPs did report seeing more cardiac problems, such as dyspnea, angina, arrhythmias, coronary artery disease, congestive heart failure, and hyperlipidemia, as well as many of the more chronic illnesses and conditions, such as type 2 diabetes and genitourinary problems (benign prostatic hypertrophy, impotence, and urinary tract infections). This is likely because students’ encounters in CBPs involved older patients to a much greater extent than in AMCs or ARTs, and older patients are more likely to have these chronic conditions. We could not determine whether the case mix of patients in CBPs differs from the mix in other sites, or whether preceptors in these settings assigned older patients to students. Perhaps students who did not see many of these conditions during their inpatient medicine rotation would prefer to select a setting where these are more likely to be seen. Students in CBPs also reported seeing more ears, nose, throat (ENT) problems than did students in the other settings. This is likely related to the specific diagnostic conditions that students reported seeing. Otitis media and lower respiratory disorders were more frequently seen in CBPs than in the other two settings.
Conversely, students placed in AMCs reported seeing more patients with symptoms or conditions that might be due to the younger age range of patients (e.g., asthma or allergies, depression, dermatologic conditions, developmental disorders). Another reason for this difference could be the referral practices undertaken at AMCs versus CBPs. An educational implication of these findings is that students interested in primary care careers and students who were not exposed to many of these disorders on their other primary care rotations might want to be placed in CBTs or AMCs for their FM rotations to take advantage of the greater volume of common primary care problems.
We also found important differences in the educational cultures of AMCs and CBPs. Students in AMCs reported more experiences counseling patients about exercise and weight control. Perhaps health values or priorities in academic sites are somewhat different than those in community sites. Maybe this finding is due to differences in case mix of patients; however, it is difficult to imagine, with the chronic conditions that occur more often in CBPs, that there is a lesser degree of obesity and sedentary lifestyle in the CBPs’ patient panel.
Differences inpatient volume or the level of competing demands may also contribute to the difference in counseling performed between the two settings. It may be that counseling is more often done by a nurse or other practice member rather than by the physician in CBP and, therefore, students are not exposed to this experience. Or it may be that patients in need of these services in CBPs are more often referred to other community organizations or agencies, which again would prevent students from seeing this clinical activity role modeled in this setting. In either case, students could take a more active role in counseling as relevant opportunities arise, which would make them a more integral and beneficial member of the CBP.
Another difference that could be attributed to the educational culture in these settings is in procedures performed. These were most often done by students in CBPs rather than in sites where other physician–learners were also in training. This difference could be due to several factors. Students in AMCs and ARTs are more likely to be in competition with residents, and residents function at a higher learner level and may more aggressively seek out or be in greater need of procedural experiences than medical students. The difference may also be due to the relative ease of referral for various procedures within AMCs, whereas in CBPs, the primary care provider more often performs procedures as a matter of course. It is likely that the two settings differ culturally in ways that we did not capture in our documentation system, and the learning environment structures of these settings need further study.
In considering the characteristics of teachers, we found that the ratio of male preceptors to females is much higher in CBPs than in AMCs or ARTs. We know from our previous work10 that gender does make a difference in ambulatory-based teaching and learning: same-gender dyads appear to teach and learn more interactively than opposite-gender dyads.
In addition, we found that the number of preceptors involved in student encounters was much higher in multilearner training sites, especially ARTs compared with CBPs. In general, academic medical centers and hospitals and clinics affiliated with residency training tend to be larger and have more teaching faculty available, including residents. We know from site visits that students are more often scheduled to work with different preceptors on a half- or whole-day basis at AMCs, whereas CBPs tend to have four or fewer preceptors who work with a given student during an entire eight-week rotation.
These different approaches to teaching may offer important educational value. Because more preceptors are available, students may move from preceptor to preceptor based on scheduling and to see interesting cases, trading off continuity in the teaching and learning relationship and continuity visits with patients. Our finding that repeat patient visits were most likely in CBPs further supports this. This may be due to a greater likelihood that students in CBPs work with fewer preceptors, as mentioned previously, so when a follow-up visit is scheduled the student is more likely to be working with the same preceptor and see the patient again. In CBPs, students may also see more patients initially on hospital rounds and then later in the office, which may be less likely to occur at academic centers where housestaff cover hospitalized patients.
Several educational research studies indicate that the majority of interactions between preceptors and learners take between three and five minutes,11–14 an amazingly short period. We found no differences in the amount of feedback provided to students by training setting, which indicates that such interactions likely occur similarly in all settings under study.
We did find, however, differences in teaching about patient management across the three settings, with more occurring in multilearner training sites (ARTs and AMCs). This was surprising because patients with chronic conditions were seen more frequently in CBPs. It would seem that the opportunity for teaching about disease management could be greater in CBPs, unless this teaching involved discussions of referral, which would perhaps occur more often in AMCs. An additional reason for this finding may be that teachers in ARTs and AMCs are more opportunistic or more direct teachers than those in CBPs, which may indicate a need for faculty development and certainly indicates a need for further research.
In any case, our data indicate that the educational relationships between students and preceptors are different in the study settings, as illustrated by students working more independently in ARTs and CBPs. Perhaps preceptors who are more distantly related to the medical school become comfortable with the students more quickly because of the types of interactions they share.
An interesting finding from our study, possibly related to the availability of educational resources or time pressures, is what appears to be strikingly different numbers of patients seen by students in the three settings. Students in ARTs recorded seeing less than half the number of patients that students in the AMCs and CBPs saw. This difference may reflect differing levels of enthusiasm on the part of students in documenting encounters or perhaps the higher volume of patients seen in academic and community settings. Perhaps the competition for patients in ARTs is much higher. Or, students at these training sites may be spending time in the library or at regular educational seminars that are likely more available at academic-affiliated sites than CBPs. The tradeoff here is between individual patient exposure versus case or literature presentations or self-study. Regardless, the critical educational implication is that students should undertake primary care learning experiences in both settings to ensure a range of exposures, and more important, they should take more responsibility for their educational needs than they currently do.
Few studies have addressed comparisons of what and how students learn in academic versus community ambulatory training sites. We found two such studies.15,16 Our findings are similar to those of Greer et al.15 in that students saw a higher mean number of patients and did more procedures at community sites than did students in residency training sites. The study conducted by Leone-Perkins et al.16 assessed students’ preferences for community versus academic sites and found that preceptors’ teaching behaviors were favorable at both sites, as rated by students; however, students in community sites had higher ratings of their own clinical performances. Evaluations were subjective rather than objective, which must be considered when interpreting these findings. We did not specifically look at student satisfaction in our analysis, which is another area for future research.
There are several important strengths of our study. First, the clinical encounters included in the analysis represent teaching and learning that occurs at multiple sites in two of the three settings under study. Bowen and Irby9 pointed out the lack of studies representing multiple sites as a problem in their recent review paper on ambulatory education. Unfortunately, we have only one FM teaching site at our AMC, which limits the generalizability of our findings. Another strength of our study is that our documentation system has undergone rigorous testing to determine its accuracy, and our analysis dataset is very robust, with over 9,000 encounters collected over a five-year period.
Our study is not without limitations, however, and these include the fact that our data are based on students’ self-reports, which may represent the opportunities they engage in or chose to document rather than the full range of opportunities that exist for learning. Another limitation is that students were systematically rather than randomly assigned their clinical placements, which indicates our findings may be affected by selection bias. In addition, our data are representative of one medical school located in rural northern New England and generalizability may be limited. Certainly, more collaborative research is needed to completely address this important topic.
Another potential limitation is that only FM encounters were included in this analysis. This discipline had the highest volume of data in our master database, and it is one discipline that focuses on ambulatory education and covers the span of adult, obstetrical, and child care that is also involved in the other common ambulatory disciplines, providing experiences that are especially relevant to undergraduate medical education. Over 90% (113 of 125) of medical schools have required clerkships in FM, so the relevance of this paper to those programs is significant.17 From an institutional perspective, undergraduate medical education must identify and understand more specifically the sources of valuable clinical experiences. FM can both complement other ambulatory clinical experiences and provide the basis for a strong foundation.
Lastly, although statistical differences exist in our dataset, which we have discussed in detail, the educational significance of these differences for teachers and learners is unclear. More research should be done to determine disease-state exposure or procedural experience thresholds relevant to developing clinical competency. The only article we found addressing this issue18 was a descriptive study that found that students experience different disease states by training site for three major diseases, including essential hypertension, diabetes mellitus and upper respiratory infections. Clearly, clinical teaching and learning must undergo additional study to understand the complex process of becoming a physician.
In conclusion, our findings suggest areas where academic and community settings complement each other as learning sites for an ambulatory clerkship in FM and areas where common curricular expectations can be achieved. Medical educators should take these differences and similarities into account when developing, implementing, or revising clerkship programs. They should also be considered, with students’ interests and previous clinical experiences, before assigning students to a teaching site for an ambulatory clinical clerkship experience.
This work was supported by two Family Medicine Predoctoral Training grants (5D15 PE80040 and 6D16 HP00011) and the Office of Community-Based Education and Research at Dartmouth Medical School.