Torre, Dario M. MD, MPH; Sebastian, James L. MD; Simpson, Deborah E. PhD
Dr. Torre is assistant professor of general internal medicine, Department of Medicine, and Dr. Sebastian is professor of medicine, Division of General Internal Medicine, both at the Medical College of Wisconsin, Clement J. Zablocki Veterans Affairs Medical Center; Dr. Simpson is professor of family & community medicine, associate dean for educational support & evaluation, School of Medicine Office of Educational Services, Medical College of Wisconsin.
Address correspondence and reprint requests to Dario M. Torre, MD, MPH, Assistant Professor, General Internal Medicine, Medical College of Wisconsin/Clement J Zablocki VA Medical Center, Froedtert East Office Bldg, Suite E4200, 9200 Wisconsin Avenue, Milwaukee, WI 53226; telephone: (414) 456-6871; e-mail: 〈email@example.com〉.
This project was partially funded by a grant from the Learning Resources Fund, Curriculum and Evaluation Committee, Medical College of Wisconsin, Milwaukee.
For research on related topics, see pp. 815–819 and pp. 820–825.
Purpose: To identify learning activities that students associate with high-quality teaching on a clinical rotation.
Method: From July to December 2001, data on patient encounters, learning activities, and teaching quality were collected via personal hand-held computers from 82 medical students during a required third-year internal medicine (IM) clerkship at the Medical College of Wisconsin, Milwaukee. Univariate (chi-square test) and multivariate analysis (stepwise multiple logistic regression) were performed to assess the association between learning activities and students' perceptions of teaching quality during this rotation.
Results: A total of 1,839 patient encounters were recorded: 62% of these occurred in the inpatient setting and 38% in outpatient clinics. In 80% of all encounters, students reported that they gave an oral case presentation and proposed a plan; they reported receiving high-quality feedback after 64% of these presentations. Univariate analysis demonstrated that the students' perception of high-quality teaching was associated with being on an inpatient rotation, formulating an assessment, proposing a plan, presenting to the attending physician with other members of the team present, giving an oral case presentation, and receiving high-quality feedback (p < .01). Multivariate analysis demonstrated that receiving high-quality feedback (odds ratio [OR] 4.5; 95% CI 3.57–6.25) and proposing a plan (OR 2.3; 95% CI 1.4–3.9) were the two strongest predictors of high, overall teaching quality.
Conclusion: Receiving high-quality feedback from faculty and proposing a plan were the learning activities most strongly associated with students' perception of high-quality teaching on a required third-year IM clerkship.
Academic health centers face significant challenges in fulfilling their educational role in today's constantly evolving price-competitive environment. The use of new methods of instruction, such as problem-based learning, has required a greater commitment of faculty time to teaching.1–4 However, previous research has demonstrated a decrease in the amount of physician time available to fulfill educational responsibilities to medical students and residents.5,6 Increased demands for clinical productivity have created considerable pressure and dissatisfaction among some clinical faculty who are trying to maintain their teaching commitments while performing an increasing number of clinical services.7 Furthermore, the decreased length of stay for hospitalized patients8 has contributed to the evolution of a learning environment in which students and residents often do not have enough time to learn from each case. To assist academic physicians in optimizing the way in which they use their limited teaching time, it is imperative that key learning activities associated with effective teaching in today's clinical environment are identified.
The use of new technology has proven to be an effective tool in medical education, although cost has been, at times, a barrier to its use.9,10 Recently, low-cost, hand-held computers or personal digital assistants (PDAs) have been used to collect data on patient encounters and trainee procedures,11,12 but there have been no published reports using students' real-time data collection about teaching/learning activities in the clinical setting. Our study was designed to assess the effect of specific learning activities occurring during a required third-year internal medicine (IM) clerkship on students' perceptions of high-quality teaching. More specifically, using real-time PDA-based data collection, we sought to identify those elements of the teaching program that students perceived as contributing most strongly to high-quality teaching encounters.
From July to December 2001, we collected data on patient encounters, learning activities, and teaching quality ratings from 82 medical students rotating on a required third-year IM clerkship at the Medical College of Wisconsin, Milwaukee. During the time our study was completed, students spent six weeks on an inpatient service and two weeks in an outpatient setting. Each day, student observations were logged in personal handheld computers loaded with a patient encounter/teaching activity questionnaire. For each patient with whom they had significant interaction, students were asked to record answers to the following questions: Did you make a case presentation?; To whom was the presentation made (to the whole team with attending physician present/to attending physician alone)?; Where was the case presented (hallway/clinic examination room/bedside)?; and What was the quality of oral feedback you received (high, medium, low, none)?
Students were also asked to record information as to whether they formulated an assessment, proposed a plan, or wrote a progress note. Students then rated the overall teaching quality of each encounter (outstanding, very good, good, marginal, unsatisfactory).
Univariate (chi-square test) and multivariate analyses (stepwise, multiple logistic regression) were performed to assess the association between students' learning activities and high-quality teaching. Results of the multivariate analysis are reported as odds ratios13 (OR), defined as the ratio of the odds of high-quality teaching as perceived by those who were exposed to a specific learning activity compared to the odds of high-quality teaching as perceived by those who were not.
A total of 1,839 patient encounters were recorded from July to December 2001. Of these encounters, 62% occurred in the inpatient setting and 38% occurred in the outpatient clinics. In 80% or more of all encounters, students reported proposing a plan (80%), writing a progress note (87%), and formulating an assessment (87%). Students gave an oral case presentation in 80% of recorded interactions, 53% of which were presented to the whole team with the attending physician present. Over half (53%) of the case presentations occurred in the hallway, 28% in clinic examination rooms, and 6% at the bedside. High-quality feedback was provided in 64% of those encounters.
Students reported high-quality teaching (rated as outstanding, very good, good) in 65% of encounters and low-quality teaching (marginal or unsatisfactory) in 35% of encounters. Chi-square analysis revealed that high-quality teaching was associated with being on an inpatient rotation, having a faculty member as the teacher (rather than a resident or fellow), formulating an assessment, proposing a plan, giving an oral presentation, giving an oral presentation to the attending physician with others present, receiving high-quality feedback, and not writing a progress note (p < .01). Results from the stepwise, multiple logistics regression revealed that receiving high-quality feedback (OR 4.5; 95% CI 3.57–6.25) and proposing a plan (OR 2.3; 95% CI 1.4–3.9) remained strong predictors of high, overall teaching quality after adjusting for all other teaching activities, type of rotation (inpatient versus outpatient), and teacher category (attending physician versus resident or fellow) (Table 1).
Receiving high-quality feedback and proposing a plan were learning activities that were strongly related to medical students' ratings of high-quality teaching. Traditional medical student learning activities (e.g., formulating an assessment, giving an oral presentation, and presenting to the attending physician while others were present) were also frequently occurring activities, but they were not perceived by students to be as highly associated with high-quality teaching.
Feedback is a common item on clinical teaching evaluation forms, and previous research has consistently revealed that feedback is a critical component of effective teaching.14 Our research also indicates that students perceive that they receive high-quality teaching when they are able to propose a plan for patient care. Although students reported that they presented a plan during 80% of their patient encounters, this activity has not typically been included on clinical teaching evaluation forms because it has not been considered to be the primary focus for third-year medicine clerkship students. For example, in Pangaro's RIME (Reporter-Interpreter-Manager-Educator) paradigm,15 which describes four stages of student learning activities/roles in clinical medicine, patient management skills, such as proposing a plan of care, are generally considered to be proficiencies expected of more senior level students. In the context of the RIME paradigm, it might be seen as premature for third-year clinical clerks still in their first six months of training to place so much emphasis on their ability to propose a plan. However, this emphasis may reflect the realities of teaching and learning in today's hospital environment. Many diagnoses are already established at the time patients are admitted to the hospital, limiting the role students perceive they can play in gathering data (R) and interpreting data and tests (I), leaving management (M) issues as the “real-time” focus for a student's patient-care activities.
Our study had several important limitations. Our study cohort consisted exclusively of third-year students from one institution. These students were rotating on their required internal medicine clerkship during the first half of the academic year; it's possible that end-of-the-year, third-year clinical clerks or more senior level students might have different perceptions about what constitutes important learning activities and high-quality teaching. Although previous research has shown excellent clinical teaching is associated with improved students' examination performances,16 the correlation between traditional teaching activities and student learning has not been demonstrated. This represents an opportunity for further investigation.
In summary, clinical teachers in today's high-pressure clinical environment must select those learning activities that are related to high-quality teaching. Our study shows that receiving high-quality feedback from faculty and proposing a plan were the learning activities most strongly associated with students' perception of high-quality teaching on a required third-year IM rotation. Further research is needed to explore the generalizability of these findings to other specialties and more senior level rotations. Given the association between high-quality teaching and performance, additional work is also needed to identify those elements of feedback, perhaps by RIME stage, that are perceived by learners as being of especially high quality.
1. Albanese MA, Mitchell S. Problem based learning: a review of literature on its outcomes implementation issues. Acad Med. 1993;68:52–81.
2. Vernon DT, Blake RL. Does problem-based learning work? A meta-analysis of evaluative research. Acad Med. 1993;68:550–63.
3. Neufeld VR, Woodward CA, MacLeod SM. The Mc Master M.D. program: a case study of renewal in medical education. Acad Med. 1989;64:423–32.
4. Kaufman A, Mennin S, Waterman R, Duban S, Hansbarger C, Silverblatt H. The New Mexico experiment: educational innovation and institutional change. Acad Med. 1989;64:285–94.
5. Nordgren R, Hantman JA. The effect of managed care on undergraduate medical education. JAMA. 1996;275:1053–8.
6. Bolognia JL, Wintroub BU. The impact of managed care on graduate medical education and academic medical centers. Arch Dermatol. 1996;132:1078–84.
7. Blumenthal D, Causino N, Campbell EG, Weissmann JS. The relationship of market forces to the satisfaction of faculty at academic health centers. Am J Med. 2001;111:333–40.
8. U.S. Department of Health and Human Services, Health Care Financing Administration. Medicare short-stay hospital statistics. Health Care Finance Rev. 1997;19(stat suppl):60.
9. Association of American Medical Colleges. Electronic teaching tools are transforming medicine. AAMC Reporter. 1999 June:1.
10. Gaba DM. Improving anesthesiologists' performance by simulating reality. Anesthesiology. 1992;76:495–501.
11. Alderson TS, Oswald NT. Clinical experience of medical students in primary care: use of an electronic log in monitoring experience and in guiding education in the Cambridge Community Based Clinical Course. Med Educ. 1999;33:429–433.
12. Garvin, RF, McRae OD. Using handheld computers to document family practice resident procedure experience. Fam Med. 2000;32:115–118.
13. Garrett JM. Odds ratios and confidence intervals for logistic regression models with effect modification. Stata Technical Bulletin. 1997;36:15–22.
14. Ende J, Pomerantz A, Erickson F. Preceptors' strategies for correcting residents in an ambulatory care medicine setting a qualitative analysis. Acad Med. 1995;70:224–9.
15. Pangaro L. A new vocabulary and other innovations for improving descriptive in-training evaluation. Acad Med. 1999;74:1203–7.
16. Griffith CH, Georgensen JC, Wilson JF. Six-year documentation of the association between excellent clinical teaching and improved students' examination performances. Acad Med. 2000;75(10 Suppl):S62–S64.