Saultz, John W. MD; O'Neill, Peggy; Gill, James M. MD, MPH; Biagioli, Frances E. MD; Blanchard, Shawn MD; O'Malley, Jean P. MPH; Brown, David; Rogers, John C. MD, MPH, MEd; Carney, Patricia A. PhD
Dr. Saultz is professor and chair, Department of Family Medicine, Oregon Health & Science University, Portland, Oregon.
Ms. O'Neill is community outreach coordinator, Department of Family Medicine, Oregon Health & Science University, Portland, Oregon.
Dr. Gill is president, Delaware Valley Outcomes Research, Newark, Delaware, and associate professor of family and community medicine, Thomas Jefferson University, Philadelphia, Pennsylvania.
Dr. Biagioli is associate professor of family medicine, Oregon Health & Science University, Portland, Oregon.
Dr. Blanchard is assistant professor of family medicine, Oregon Health & Science University, Portland, Oregon.
Ms. O'Malley is research associate, Biostatistics and Design Program, Oregon Clinical and Translational Research Institute, Oregon Health & Science University, Portland, Oregon.
Mr. Brown is senior research applications developer, Department of Family Medicine, Oregon Health & Science University, Portland, Oregon.
Dr. Rogers is professor of family and community medicine, Baylor College of Medicine, Houston, Texas.
Dr. Carney is professor of family medicine and public health and preventive medicine, Oregon Health & Science University, Portland, Oregon.
Please see the end of this article for information about the authors.
Correspondence should be addressed to Dr. Saultz, Department of Family Medicine, Oregon Health & Science University, 3181 SW Sam Jackson Park Rd., Mail Code: FM, Portland, OR 97239; telephone: (503) 494-6602; fax: (503) 494-4496; e-mail: email@example.com.
A national debate is now under way regarding how to reform the U.S. health care system. Central to this debate is the need to transform both the financing of health care and the way in which it is delivered. Nowhere is the need for delivery system reform more evident than in ambulatory primary care practice. Based in part on work by Starfield and others regarding primary care's favorable population health effects and lower health care costs,1–6 the concept of patient-centered medical homes (PCMH) has emerged as a keystone in the reform process. Early work suggests that traditional practices will require major changes in order to become medical homes,7 and the impact of this change process on community-based medical education is unknown.
A PCMH is a transformed model of delivering basic health services to populations that brings together four key concepts: the well-established benefits of primary care, an improved model to care for chronic illnesses, an increased focus on the patient as consumer, and an increased use of communication and information technology.7–14 Specific features of this model include (1) an ongoing relationship between each patient and a personal physician trained to provide first contact and continuous and comprehensive care, (2) care provided by physician-led teams, (3) care teams that provide or arrange care through all stages of life, (4) care coordinated across all elements of the health care system and community, (5) care facilitated by disease registries and information technology, (6) enhanced access available through expanded hours and advanced-access scheduling, (7) quality and safety ensured through the use of continuous quality improvement, evidence-based medicine, and clinical decision-support tools with active patient participation and feedback, and (8) a payment system reformed to recognize the added value provided to PCMH patients.12 A national demonstration project has already occurred, showing how community-based primary care practices might adopt this model, and others are planning similar projects.7 Early results from this demonstration project suggest that most existing primary care practices require major changes before they can actualize these features.
What has been missing from the debate about health care reform is a discussion of the implications of such reform on undergraduate and graduate medical education. While important work is either under way or beginning on the redesign of residency education in primary care, including family medicine (FM),15,16 general pediatrics,17 and general internal medicine,18 there is a paucity of educational research assessing the PCMH features that medical students in the third-year core clerkship curriculum in U.S. medical schools are exposed to and how this exposure may vary among medical schools. Undoubtedly, medical students should learn to use information technology and other features of practice that are likely to become part of residency training, and graduating residents need to move smoothly into contemporary clinical practice. Without knowing the status of evolving new elements of practice, medical educators can only speculate about what they must teach, how they can teach it, and what will best prepare graduating physicians for clinical work.
We conducted a cross-sectional study to examine the extent to which ambulatory FM clerkship sites are implementing features of the PCMH.
To identify participating medical schools, we posted a message calling for volunteers on a listserv for FM predoctoral clerkship directors. To be eligible for the study at the medical school level, we required participating schools to have both a well-established third-year FM clerkship mandatory for all students and a curriculum requiring these students to round at ambulatory clinical practices as part of the clerkship. In addition, we sought schools that were representative of the nation in terms of geography and size. The Society of Teachers of Family Medicine (STFM) and the STFM Research Committee assisted in recruiting institutions. Through the recruitment process, we identified 13 medical schools that indicated initial interest. An FM predoctoral faculty leader at each of these 13 schools received a letter detailing the study procedures. That recipient completed a form to provide information typically collected about all active clerkship sites (e.g., number of students, methods of patient payments; further described below). After receiving this letter and form, 12 of the 13 schools (92%) agreed to participate in the study. From the completed forms, we were able to ascertain the number of eligible clinic sites involved in clerkship training at each school.
We asked the FM predoctoral faculty leader at each participating school who had completed the form to identify a local study coordinator who would administer and collect study measures. We sent this coordinator a packet of practice surveys and asked him or her to provide these to each clerkship site that the medical school had identified as meeting eligibility criteria. We also asked the coordinator to collect the completed documents. To be eligible for the study, a clerkship site must have hosted at least one clerkship student per year for each of the previous three academic years (2005–2006, 2006–2007, 2007–2008). The participating medical school FM department received $700 for completing the project. The institutional review board at Oregon Health & Science University approved the study.
Survey instrument development and testing
We developed two paper-based study instruments. We designed the first, a predoctoral leader survey completed by an FM predoctoral faculty leader at each participating school, to identify the clerkship sites affiliated with each institution including (1) the total number of practice sites eligible to complete the PCMH survey (described below) and (2) the total number of practice sites that returned the completed survey. We developed and tested this predoctoral leader survey through cognitive interviews with predoctoral faculty members at a nonparticipating school, and we revised the instrument on the basis of their feedback regarding the clarity of questions and the accuracy of information collected.
The second instrument was the PCMH Practice Survey, which we designed to be self-administered by the lead physician at each eligible clerkship site. The PCMH Practice Survey had three parts. The first part described features of the practice as a learning setting (e.g., status as a Federally Qualified Health Center [FQHC], type of practice, and whether residents are also trained at the site); the second part assessed electronic and nonelectronic aspects of the PCMH (e.g., electronic health records [EHRs], asynchronous electronic communication with patients and other clinicians, group visits, advanced access scheduling, and use of disease registries); and the third part assessed preceptor attitudes about electronic record systems and their impact on medical education. We adapted this instrument from previous studies of community FM practices7 and FM residencies,16 and we extensively pilot tested it at three FM clerkship practice sites not involved in the current study. In these early testing phases, we used cognitive interview techniques to ensure that respondents understood the questions as intended by the survey design. The possible responses for the presence of each component of the PCMH were 1 = absent (no plans to implement in the future), 2 = planning (active planning under way but not implemented), 3 = present but not mature (revisions planned), and 4 = mature (no planned changes in the near future).
Data collection for the study began in August 2008 and was completed in November of the same year. The designated FM predoctoral leader at each participating medical school completed the predoctoral leader survey. The local study coordinator at each medical school distributed the PCMH Practice Survey to each school's eligible clerkship sites, collected the completed surveys, and returned them to the project team at Oregon Health & Science University. Study staff worked with contacts at each medical school to encourage data collection and the return of completed instruments, and staff contacted individual practice sites regarding incomplete surveys or suspected data errors.
Data management and statistical analyses
Two staff members independently entered the information from the submitted questionnaires, and a third staff member reviewed and, if necessary, corrected the entries using source documents if agreement between the two data entry staff members was incongruent. We ran frequencies to identify outliers, and we called study sites to either correct or confirm the accuracy of data that might have been incorrect. At least two study staff members reviewed and initialed all resulting changes to source documents.
We created analytic files to examine characteristics of the practices and the status of PCMH features, and we used descriptive statistics (SPSS Inc., Chicago, Illinois) to characterize the practices and their overall adoption of features of the PCMH. We used general linear mixed models to account for covariance due to nesting of clinics according to medical school. For purposes of this analysis, we collapsed absent and planning into one category called “absent” and present and mature into a second category called “present.” We examined electronic and nonelectronic features of the PCMH separately. For another analysis, we created composite variables that were a sum of all PCMH features (electronic and nonelectronic, separately). We used the Pearson correlation coefficient to assess the relationships between the composite scores and patient volume, and we additionally explored relationships between the composite scores and practice setting features, such as being an integrated health system or an FQHC. We set alpha at 0.05 or less to assess for statistical differences, and all tests were two tailed.
Nine* of the 12 medical schools (75%) that agreed to participate completed a majority of the study elements. Three schools withdrew from the study because of difficulty obtaining completed surveys from their clerkship sites. Overall, participating schools captured data on 104 (44%) of their eligible FM clerkship sites, with a range of 18% to 100% (Table 1). Table 2 lists descriptive information about these practices. The practices were primarily community-based, single-specialty clinics (n = 48; 46%), and more than half (n = 55; 53%) were part of an integrated health system. Twenty percent (n = 21) were FQHCs. The vast majority of the practices assign a personal physician to each patient (>95%), and the clinics had substantial differences in the number of patients seen annually (mean = 25,436; standard deviation [SD] = 35,497; range 1,000–216,000).
Status of all electronic features of the PCMH aggregated for all medical schools (Figure 1) indicates that 58% of clinics (n = 60) have EHRs in place, and of these, nearly 37% (n = 22) are mature. Other electronic elements are less likely to be mature. Electronic orders (e.g., labs, X-ray) integrated with EHRs and preventive registries are most likely to be in the planning stages, whereas conducting practice-based research using the EHR and having a functional quality telephone monitoring system are features that these practices were most likely not to have active plans for implementing. Status of nonelectronic features of the PCMH aggregated for all medical schools (Figure 2) indicates that adequate free parking and convenient access to public transportation are the most present and mature aspects of the PCMH, and group visits and clinical pharmacy support are the most likely to be absent features.
Our general linear mixed-model analyses are presented in Tables 3 to 5. Of the 14 electronic features assessed (Table 3), none had statistically significant differences according to medical school, even though up to 10-fold variation in implementation of other PCMH features existed. Three PCMH electronic features reached 50% implementation when averaged according to medical school: (1) presence of an EHR, which ranged from 33% to 100%, (2) availability of fully secured remote access (range 33%–100%), and (3) electronic scheduling system integrated into the EHR (22%–100%).
Implementation of nonelectronic features of the PCMH (Table 4) indicates that of the 12 features assessed, three (25%) were found to vary significantly among schools: (1) the availability of a credible, reliable patient satisfaction survey (range 20%–100%; P = .02), (2) the presence of clinical pharmacy support (range 0%–100%; P = .05), and (3) adequate free parking (range 7%–100%, P = .007). No significant difference existed in the other nonelectronic elements of the PCMH. By medical school, more than half of the clerkship sites had implemented 7 of the 12 non-electronic features assessed (58%) including (1) expanded hours (range 0%–100%), (2) credible, reliable patient satisfaction surveys (range 20%–100%), (3) integrated behavioral health care (range 13%–100%), (4) adequate physical space (range 60%–100%), (5) adequate free parking (range 7%–100%), (6) access to convenient public transportation (range 56%–100%), and (7) overall status of practice as patient-centered versus physician-centered (range 33%–100%).
Table 5 outlines our findings related to physicians' attitudes about how new technology in clinical practice influences medical education. Approximately 86% (n = 51) of clinics that have EHRs allow medical students to access their EHRs, whereas 14% (n = 8) do not. We found wide variation among the physician attitudes regarding both the effect of new technologies on the quality of medical education and the larger role students are playing in the patient-care team as a result of new technology—although this latter variation was not significantly different among the medical schools.
We examined several factors that we thought might be associated with high versus low PCMH-component implementation to determine the best analytic approach using our composite scores. These included being an FQHC or integrated health system. We found no associations with type of practice setting for either of these variables. We also examined the relationship between patient volume and electronic features of the PCMH to test the hypothesis that high patient volume might correlate with more electronic and nonelectronic features of the PCMH (using the composite scores). The correlation coefficient between these two variables was 0.48 (P = .66) for electronic features and −0.04 (P = .71) for nonelectronic features, indicating no significant relationship between patient volume and features of the PCMH.
Discussion and Conclusions
Health care reform is likely to generate a period of rapid change in how medicine is practiced throughout the United States, and these changes are already having a significant impact on the day-to-day work of physicians.7,14,16,19,20 Educating medical students while in this transitional mode raises questions about curricular objectives and how best to prepare students for careers in a new model of practice.
To our knowledge, our study is the first to use a national sample to examine the degree to which medical students in the United States are experiencing elements of the PCMH in their required curriculum as well as the extent to which student exposure to PCMH features varies by medical school. Because third-year internal medicine and pediatric clerkships are often based primarily in hospital settings, FM clerkships are the most likely place in the curriculum for students to first experience these new care models in the ambulatory setting. Although we found up to 10-fold differences from school to school in our analysis of clerkship training sites, only three areas reached statistically significant variation, and all three were in nonelectronic features. We found that over 50% of clerkship placement sites have EHRs, which differs from the findings of Linder and colleagues,21 who found that in 2003 and 2004 only 18% of the patients seen by physicians had EHRs as recorded on the Ambulatory Medical Care Survey. Our findings suggest that implementation of these features may be rapidly expanding or that medical school faculty may be selectively assigning students to practices with more of these features. Although other reports have shown wide variation among clinical practices in how and when elements of this new model of care arise,7,19 these did not focus on the teaching that occurs in these settings, which is the specific contribution to the literature that our study makes.
We explored several possible sources of variation including whether the practices were part of integrated health systems or FQHCs or whether high patient volume was associated with more versus fewer PCMH features. We found no such association, which leaves us to speculate other possible explanations. Some regions of the country may be implementing the PCMH model more readily than others. Or size and other features of a practice simply may not affect implementation of innovations likely to improve patient care. Also, medical school clerkship faculty members might pay varying degrees of attention to the presence of PCMH features when they select training sites.
We were surprised to learn how much variability there is regarding preceptor attitudes about the effects of practice transformation and information technology on medical education, even though these variables did not reach statistical significance when we analyzed them by medical school. Preceptors seemed to reach no overall consensus on whether emerging technologies in the care of patients will help or hinder student education. In three of the nine schools, over 65% of preceptors either agreed or strongly agreed that new technology would improve medical education, while at another six schools more than half of respondents disagreed or strongly disagreed that new technology would improve the quality of medical education. Previous work suggests that there is considerable disagreement on this point among medical students, residents, and medical educators.22–25 Peled and colleagues22 speculate that use of EHRs bypasses the need for trainees to synthesize clinical information because the EHRs do so much of this for them. They also note that EHRs can be a significant distraction for learners as they focus too heavily on the computer and not enough on patients. However, the results of another study23 showed that preceptors gave more and better feedback to third-year medical students on progress notes when the students entered the notes into an EHR.
The preceptors' responses to our survey questions regarding the influence of electronic technology on medical education may be due to factors identified in the studies cited above,22–25 or they may be due to the timing of our survey compared with when EHR use started in each practice. We conducted our survey when many practices were just beginning to use EHRs (only 37% described their EHR as mature). Transforming a practice from paper to electronic is stressful, especially at the beginning,19 and physicians may initially perceive that EHRs create barriers to patient care and complicate time management. Physicians working during such times of transition may have a hard time seeing the benefits of the EHR for patient care, let alone fathoming the benefits to medical education. This initial lack of enthusiasm is understandable in light of published research questioning whether EHRs improve ambulatory care21 or reduce cost26 at this point in their evolution. Students working in practices with newly implemented EHRs often experience an environment of chaotic change, and this is likely to affect their education in unpredictable ways. Stressed physicians are likely to be less effective teachers.
Preceptors also may not know how to best use new technology when teaching. A lack of faculty development in the area of using electronic media for medical education may leave these preceptors at a loss when confronted with the daily demands of both patients and students. Preceptors need instruction not only in using the technology clinically but also in using it educationally.
If students do not have access to the EHR, it is hard to imagine how they could review patients' history, identify and interpret lab results, write notes, or investigate consultant reports, all essential tasks for clinical learning. In 14% of the EHR-equipped practices in our study, students did not have access to the record system. Using EHRs in education requires a change in the systems of teaching, and the infrastructure for this education needs support. Excellent medical education using an EHR requires that students have a place to use the computer, a computer to use, a password to get into the system, adequate practice with the EHR system, permission to enter orders and write notes in the chart, and an understanding of the power of the EHR in quality chronic disease and population management.
We found more variability in the adoption of nonelectronic than electronic features of the medical home. These included the availability of a credible, reliable patient satisfaction survey, the support of a clinical pharmacy, and the presence of adequate free parking. Reasons for the variation we found for these items may include practice location (metropolitan versus suburban or rural settings) and the status (i.e., as an FQHC) of the health systems to which some of these practices belong. If patients are to embrace the PCMH, then this new model of care must be more immediately consumer-accessible and more user-friendly. Some have argued that such patient-centeredness is the most essential concept in the PCMH.8,20 Many of the attributes of patient-centeredness are key components of professionalism for students, and practicing these attributes will help students learn how to be available to patients while also managing appropriate boundaries between personal and professional responsibilities. It is not clear how much attention medical school faculty are paying to the principles of the PCMH when choosing practices in which to place clerkship students, but differences in this emphasis may also explain some of the variation we found from school to school.
The strength of our study is that we were able to collect detailed information about features of the PCMH from over 100 clinics that are active ambulatory care teaching sites for third-year medical students. Our study also had some limitations. One is that only nine medical schools were represented, and although these nine represent a national sampling, our results are not necessarily generalizable to all sites where medical student education occurs. In addition, though we attempted to control who completed the survey at each site, we are not confident that the lead physician was always the one who performed this task, so response bias may have influenced our findings. We chose to focus our analysis on established clerkship training sites with a clear history of taking students over time. While this choice allowed us to survey those sites that take the most students, we did not include new or part-time sites, and this exclusion might have influenced our results.
We surveyed preceptors of the practices and not the students because our primary interest was in how these teaching practices were changing, and we sought to compare our results with similar analyses in previous studies at the residency and community practice levels.7,16 An obvious follow-up approach might be to measure student experiences and attitudes before and after completing required clerkships in these practices.
In conclusion, we found that the process of transforming primary care to the PCMH is already well under way in a national sample of FM clerkship sites. Modest variability exists in specific features of the PCMH, and considerable variation exists in preceptor attitudes about the impact of new technologies on ambulatory medical education. Managing this variability will be a major challenge to FM clerkship and medical school curricular leaders for the next several years.
The authors wish to thank the departments of family medicine at the nine participating medical schools (Albert Einstein College of Medicine; Jefferson Medical College; New Jersey Medical School; Medical University of South Carolina; Southern Illinois University School of Medicine; Eastern Virginia Medical School; University of North Dakota School of Medicine & Health Sciences; University of Texas, San Antonio School of Medicine; and University of Washington School of Medicine) for their assistance in practice identification and data collection.
The authors also wish to thank Ms. LeNeva Spires for her editorial assistance with the final version of the manuscript.
The Society of Teachers of Family Medicine and the Research Program at Oregon Health & Science University's Department of Family Medicine supported this study. The Oregon Clinical and Translational Research Institute (OCTRI), grant number UL1-RR024140 from the National Center for Research Resources (NCRR), a component of the National Institutes of Health (NIH) and NIH Roadmap for Methods Research, provided statistical expertise.
The institutional review board at Oregon Health & Science University approved the study.
1Wagner EH, Austin BT, Davis C, Hindmarsh M, Schaefer J, Bonomi A. Improving chronic illness care: Translating evidence into action. Health Aff (Millwood). 2001;20:64–78.
2Starfield B, Shi L, Macinko J. Contributions of primary care to health systems and health. Milbank Q. 2005;83:457–502.
3Starfield B. Why More Primary Care: Better Outcomes, Lower Costs, Greater Equity. Presented at: The Commonwealth Fund, Primary Care Roundtable: Strengthening Adult Primary Care: Models and Policy Options; October 3, 2006.
4Baicker K, Chandra A. Medicare spending, the physician workforce, and beneficiaries' quality of care. Health Aff (Millwood). 2004;suppl Web exclusives:W4-184–W4-W197.
5Bodenheimer T. Primary care—Will it survive? N Engl J Med. 2006;355:861–864.
6Phillips RL, Dodoo MS, Green LA, et al. Usual source of care: An important source of variation in health care spending. Health Aff (Millwood). 2009;28:567–577.
7Nutting PA, Miller WL, Crabtree BF, Jaen CR, Stewart EE, Stange KC. Initial lessons from the first national demonstration project on practice transformation to a patient-centered medical home. Ann Fam Med. 2009;7:254–260.
8Rosenthal TC. The medical home: Growing evidence to support a new approach to primary care. J Am Board Fam Med. 2008;21:427–440.
9Robert Graham Center for Policy Studies in Family Medicine and Primary Care. The Patient-Centered Medical Home: History, Seven Core Features, Evidence, and Transformational Change. Available at: http://www.adfammed.org/documents/grahamcentermedicalhome.pdf
. Accessed February 13, 2010. [No longer available.]
10Berenson RA, Hammons T, Gans DN, et al. A house is not a home: Keeping patients at the center of practice redesign. Health Aff (Millwood). 2008;27:1219–1230.
15Green LA, Jones SM, Fetter G Jr, Pugno PA. Preparing the personal physician for practice: Changing family medicine residency training to enable new model practice. Acad Med. 2007;82:1220–1227.
16Carney PA, Eiff MP, Saultz JW, et al. Aspects of the patient-centered medical home currently in place: Initial findings from preparing the personal physician for practice. Fam Med. 2009;41:632–639.
17Jones MD Jr, McGuinness GA, First LR, Leslie LK; Residency Review and Redesign in Pediatrics Committee. Linking process to outcome: Are we training pediatricians to meet evolving health care needs? Pediatrics. 2009;123(suppl 1):S1–S7.
18Holmboe ES, Bowen JL, Green M, et al. Reforming internal medicine residency training: A report from the Society of General Internal Medicine's Task Force for Residency Reform. J Gen Intern Med. 2005;20:1165–1172.
19DesRoches CM, Campbell EG, Rao SR, et al. Electronic health records in ambulatory care—A national survey of physicians. N Engl J Med. 2008;359:50–60.
20Gottlieb K, Sylvester I, Eby D. Transforming your practice: What matters most? Fam Pract Manag. 2008;15:32–38.
21Linder JA, Ma J, Bates DW, Middleton B, Stafford RS. Electronic health record use and the quality of ambulatory care in the United States. Arch Intern Med. 2007;167:1400–1405.
22Peled JU, Sagher O, Morrow JB, Dobbie AE. Do electronic health records help or hinder medical education? PLoS Med. May 5, 2009;6:e1000069.
23Rouf E, Chumley HS, Dobbie AE. Electronic health records in outpatient clinics: Perspectives of third year medical students. BMC Med Educ. 2008;8:13.
24Keenan CR, Nguyen HH, Srinivasan M. Electronic medical records and their impact on resident and medical student education. Acad Psychiatry. 2006;30:522–527.
25Aaronson JW, Murphy-Cullen CL, Chop WM, Frey RD. Electronic medical records: The family medicine resident perspective. Fam Med. 2001;33:128–132. Available at: http://www.stfm.org/Fullpdf/feb01/mi.pdf
. Accessed February 13, 2010.
26Sidorov J. It ain't necessarily so: The electronic health record and the unlikely prospect of reducing health care costs. Health Aff (Millwood). 2006;25:1079–1085.
* The nine schools that provided usable surveys from the family medicine ambulatory medicine clerkship sites were Albert Einstein College of Medicine, New York; Jefferson Medical College, Pennsylvania; New Jersey Medical School; Medical University of South Carolina; Southern Illinois University School of Medicine; Eastern Virginia Medical School; University of North Dakota School of Medicine & Health Sciences; University of Texas, San Antonio School of Medicine; and University of Washington School of Medicine. Cited Here...