Secondary Logo

Journal Logo

Clinical Skills

A Prospective Study of the Longitudinal Effects of an Embedded Specialty Curriculum on Physical Examination Skills Using an Ophthalmology Model

Mottow-Lippa, Linda, MD; Boker, John R., PhD; Stephens, Frances

Author Information
doi: 10.1097/ACM.0b013e3181bb2d51
  • Free

Abstract

Current models of health care delivery, such as managed care, rely on generalist gatekeepers to examine, first-line treat, and triage patients who require subspecialty care. Traditionally, a generalist's comprehensive physical examination has included the screening ocular examination, and in some countries it still does.1,2 Earlier generations of medical students mastered essential physical examination skills that included funduscopy and proper screening of vision, pupils, extraocular muscles, confrontation fields, and gross anterior chamber depth.3 A recent position paper by the Association of American Medical Colleges Task Force on Clinical Skills validated the ophthalmic screening examination as a core skill set for the undifferentiated, newly graduated 21st-century physician.4 A survey of U.S. primary care residency directors found that 85% considered these eye examination screening skills essential in their incoming housestaff, but 90% felt that fewer than half of their interns had acquired those basic skills in medical school.3

Medical student exposure to subspecialty education in general, and ophthalmology in particular, has declined during the past 20 years.5 A 2004 survey of 126 U.S. medical schools found that 91% of 70 responding schools offered some formal ophthalmology training. However, only eight schools required an ophthalmology clerkship, and another eight offered both required and elective clinical experiences.6 Only approximately a quarter of U.S. students choose to do a clinical ophthalmology elective.5 Similarly, in Canada, as of 1998, only 7 of 16 schools required a clinical ophthalmology rotation.7 A study of Canadian students found that 47% self-reported either no or minimal confidence in their ability to examine an undilated fundus.8 The current prevailing marginalization of ophthalmology from the clinical curriculum inhibits students from developing ophthalmic knowledge and skills that are essential for future generalists, and even specialists, for providing quality patient care.

Our prior study9 objectively documented the degree of eye examination skill acquisition and decrement in one student cohort during the final three years of medical school. A companion observational funduscopy study conducted in three successive student cohorts indicated that observed skill deficits were not isolated to that single first class.10 We designed the present study to determine the effects of a sustained instructional intervention on skill retention and application in the clinical years of undergraduate medical training.

Method

We received exempt approval from the University of California-Irvine institutional review board to monitor the multiple outcome measures resulting from student assessments that were already in use in the curriculum, to evaluate prospectively the impact of a three-year, integrated ophthalmology skills curriculum. In academic year 2003–2004, Year 1 medical students at a medical school that is part of a California public university began a required ophthalmology curriculum aimed at expanding their ophthalmic knowledge base and developing their screening skills. Following the class cohort (N = 93) across its final three years of training provided longitudinal assessment of performing target skills.

Curriculum

We adapted the clinical skills objectives, training activities, assessment instruments, and timelines from our prior observational prospective study10 to this interventional prospective protocol. In the current study, an interwoven ophthalmology content theme enhanced core courses in Years 1 through 3, with ophthalmic elements essential to best medical practices in primary care (Table 1).

Table 1
Table 1:
Clinical Skills, Objectives, Training Activities Constituting a Longitudinal Curriculum for Ophthalmic Screening Examination Skills at the University of California, Irvine School of Medicine

Years 1 and 2.

The first phase of the embedded ophthalmology curriculum focused students on the fundamental knowledge acquisition necessary to recognize and interpret findings on the ophthalmic screening examination. Didactic sessions in Year 1 (2003–2004) centered on clinical correlates to anatomy (e.g., Hutchinson's sign in herpes zoster ophthalmicus, gingival hypesthesia in orbital floor fractures, and the anatomic basis for ocular deviations in cranial nerve palsies).

Lectures in Year 2 (2004–2005) focused on the screening eye examination, common eye manifestations of systemic disease, ocular adverse effects of systemic medications, differential diagnosis of the red eye, eye emergencies, sudden visual loss, and common eye diseases.

Students' initial training in fundamental screening eye examination skills occurred at the end of Year 2 during their physical diagnosis course. After a one-hour introductory lecture, the students split into small groups of about six with ophthalmology faculty preceptors for three hours to practice a defined set of examination skills—both on a funduscopic simulator (described below) and on their peers. To ensure consistent teaching and uniform evaluation criteria, the preceptors were previously debriefed in a 15-minute faculty development session that covered lecture content regarding technique and the “disc mantra” nomenclature the students would be expected to learn to use.11 The faculty then were introduced to the funduscopic simulator.

Year 3.

During the required Year 3 family medicine (FM) clerkship (2005–2006), a mandatory ophthalmology module reinforced core ophthalmic knowledge. The entire Year 3 class (N = 96) received two 3-hour blocks of interactive case-based teaching in small groups of about 10 students, with a single senior faculty ophthalmologist (L.M.-L.).

In the first three-hour session, one of us (L.M.-L.) reinforced the eye examination skills learned the previous year and reviewed the disc mantra algorithm for describing the fundus—“CCCMVR”—cup-color-contour-margins-vessels-retina.11 We then interactively explored three case scenarios: (1) third cranial nerve palsy in a diabetic, (2) papilledema of different etiologies, and (3) arterial occlusions in patients with temporal arteritis or embolic disease. Each case included correlative fundus slides, which the students viewed through an ophthalmoscope in a hand-built, optically correct funduscopic simulator. The simulator, known as “CLEO”—Clinical Learning Experience in Ophthalmoscopy—enhances the model first described by Dodaro and Maxwell,12 using one dilated and one undilated pupil. Refractive errors varying between each of our four CLEO simulators forced the students to make ophthalmoscope focus adjustments. The students practiced with different models of ophthalmoscopes viewing different fundus kodachrome slides that exhibited classic disease presentations from actual patients. With the preceptor, students then explored the effects of the eye examination, including findings from the CLEO funduscopy, on the thought processes involved in determining the pathophysiology, etiology, diagnosis, and differential diagnosis of the various diseases and in managing patients.

Students spent an additional half-day of self-study practicing ophthalmoscopy with a slide study set for CLEO, using disc mantra charts to assist in determining a diagnosis. In the second three-hour session with one of us (L.M.-L.), the students engaged in a personalized interactive session, often guided by one or more of the following Web sites:

The students were also urged to engage in complementary self-directed learning, based on both standard textbooks and the above Web links, the material of which stressed the applicability of ophthalmology to best medical practices in general medicine.

To ensure that the students understood that they were expected to implement their skills habitually, we engaged the FM faculty in the effort to boost skills performance. Selected FM faculty experienced a focused one-hour practice session with CLEO and the disc mantra algorithm, reinforcing their own skills confidence level and raising their expectations of the skills performed by the students they supervised in the hospital and clinics during the four-week clerkship.

Year 4.

During Year 4 (2006–2007), the standard ophthalmology elective remained the sole opportunity for formal retraining.

Assessment data collection

Year 2.

Year 2 students' performance assessments targeted the acquisition of core knowledge and fundamental skills underlying the screening eye examination. Knowledge assessment occurred during scheduled course examinations and consisted of embedded multiple-choice items covering eye-related content. A structured checklist of 10 defined skills completed by the preceptors in addition to a student self-assessment skills checklist and survey formed the basis for measuring skills acquisition (Table 1).

Year 3.

We assessed the effect of training during the ophthalmology module of the FM clerkship on the students' knowledge base and objective ophthalmoscopic skills immediately post clerkship. Then, at clerkship exit, students completed both a 15-item, multiple-choice quiz to assess their knowledge of basic principles underlying ocular disease processes and a practical funduscopy examination in which they viewed a single slide through a dilated pupil in the CLEO simulator. The slide represented the same diagnosis as, but was not identical to, one of the fundus slides in their self-study teaching slide set. They recorded their findings at the time they performed the funduscopy. The same ophthalmology preceptor (one of us, L.M.-L.) scored the student responses to measure the accuracy of the descriptions and the extent to which students followed the disc mantra algorithm. The total score of 25 points resulted from 3 points for correctly describing each disc feature (i.e., cup, color, contour, and margins) and the vessels and for providing a correct diagnosis and differential diagnosis. Correctly describing all features in the retina earned four additional points, with partial credit awarded according to a preset rubric.

To evaluate student charting performance during the FM clerkship, the clerkship director randomly selected three complete history and physical examination notes written by each student from among all those submitted for grading purposes. A research assistant masked the notes to hide any information that would identify either the patient or student involved. One of us, the ophthalmology preceptor (L.M.-L.), then reviewed the anonymous student notes, scoring them for the relevance, presence, and adequacy of eye examination. A standardized rubric measured the appropriate application of the 10 skills first taught in the Year 2 hands-on physical diagnosis course and reiterated in the Year 3 ophthalmology module.

Similarly, for comparison purposes, the Year 3 internal medicine (IM) clerkship director provided a triad of history and physical examination notes written by each student from among those submitted for grading purposes. Two of us (F.S. and J.R.B.) obtained the Year 3 clerkship schedule from the registrar to identify the timing of FM relative to IM training for each student in this cohort. We tracked each student's performance individually and longitudinally along the curriculum. However, to avoid bias on subsequent student grading, the ophthalmology preceptor (L.M.-L.) remained blinded to all individual performance data throughout the study except for the FM course knowledge and CLEO assessments.

At the end of Year 3, all medical students must take and pass the California Clinical Performance Examination (CPX). Immediately after a standardized patient diabetic case during the CPX, students again encountered the CLEO funduscopic simulator in a complementary, 10-minute, interstation exercise. One of us, the ophthalmology preceptor (L.M.-L.), scored the written responses obtained from this interstation encounter, applying the same rubric as for the post-FM clerkship CLEO exercise. The score represented the extent to which students in the CPX setting applied the disc mantra, provided proper differential diagnosis, and described appropriate management. Unlike in the post-FM clerkship funduscopy, the CPX examination required only a differential diagnosis and not the correct diagnosis, limiting the possible maximum score to 22 points. For the sake of parity, we recalculated the post-FM clerkship scores based on a 22-point maximum, reflecting the identical elements tested on the CPX examination.

Year 4.

During Year 4, we assessed the charting behavior of all students (N = 36) in the Year 4 IM subinternship (described below). We also surveyed all Year 4 students immediately before graduation (May 2007) about their experiences with, perceptions of, and values regarding their ophthalmology training and skills, but we did not provide any further required ophthalmic skills instruction during this year. We did, however, track students who experienced further training in ophthalmology electives at any time during Year 4 (and Year 3) in order to capture any possible confounding effect.

Data analysis

We entered all data into a spreadsheet for tabulation and preliminary calculation of descriptive statistics (e.g., rates and percentages) for each variable performed. We report rates as mean percentages (rounded up to the nearest whole number) and provide standard deviations. We performed subsequent analyses using SPSS version 13.0 for Windows (SPSS Inc., Chicago, Illinois). We used appropriate nonparametric (e.g., z test for proportions, McNemar test, χ2) or parametric (e.g., bivariate correlation coefficients, t test, analysis of variance) statistical tests to compare students' screening eye examination rates and to compare scores across training years on all pertinent measures.

Results

A total of 93 Year 2 medical students began the study as members of the class cohort. Exposure to the planned ophthalmic skills curriculum occurred in the second half of the academic year, immediately before the start of the required Year 3 clerkships.

Table 2 presents longitudinal (from Years 2 through 4) skills assessment results for evaluating visual acuity, pupils, extraocular muscle excursions, visual fields, and conjunctiva. Likewise, Figure 1 summarizes students' ability to perform funduscopy immediately after initial training in Year 2 and via charted funduscopy notes of clinical encounters in the Year 3 FM and IM clerkships. For comparison, the graph also includes charted funduscopy in workups audited in an earlier class cohort monitored in our prior observational study, as historical controls.10

Table 2
Table 2:
Summary of Longitudinal Eye Screening Examination Assessments for a Single Class Cohort Followed Across Three Years of Training at the University of California Irvine School of Medicine, 2004–2007
Figure 1
Figure 1:
Medical students' ability to visualize the fundus, from preceptor and self-assessed ratings post-Year 2 skills training, and their actual performance of ophthalmoscopy from chart audits during their Year 3 family medicine (FM) and internal medicine (IM) clerkships. Rates were calculated on the basis of charts in which the eye findings were relevant to the case, and which could have specified findings for ophthalmoscopy. Historical cohort represents Year 3 clerkship funduscopies charted in our prior companion observational study.10.

Preclerkship assessments

All Year 2 students (n = 93) passed their respective course examinations, which included items covering basic knowledge pertinent to ophthalmic screening (results not reported here). We captured data on 82 of those students after small-group, hands-on eye examination training (11 evaluation forms were either not completed or not turned in by the preceptors or students). Physical diagnosis course preceptors reported that 100% of those 82 students demonstrated acquisition of target skills at the completion of the small-group eye examination sessions. Students' self-assessments revealed that 82/84 (98%) and 80/84 (95%), respectively, could view the disc and the macula, but only a third (25/83, 30%) felt “very confident” and two thirds (55/83, 66%) “somewhat confident” about their skill level. The majority (56/74, 76%) endorsed the viewpoint that the physical examination was just as, or more important than, imaging studies or laboratory tests, but a mere 9 of 71 students (13%) owned an ophthalmoscope. Whereas 82/83 students (99%) wanted further ophthalmoscopic skills training, only 18 of 80 (23%) indicated an intention to definitely take an elective, 3/80 (4%) expressed no interest in enrolling in one, and the rest remained undecided.

FM ophthalmology module assessments

In 2005–2006, 96 students registered for Year 3. The Year 3 class size fluctuated from that reported for Year 2 because of a variety of scheduling factors, including students enrolling in dual-degree programs, going on or returning from a leave of absence, or placing into an extended preclinical curriculum for personal or academic reasons. We reported scores only for Year 3 students enrolled in the ophthalmology module assessments, excluding test scores for Year 4 students (i.e., those in a different graduating class) who had deferred the FM clerkship from the prior year and those Year 3 students in the study cohort who deferred FM until the following year. The mean basic knowledge score obtained by the 74 students completing the module was 67% (±10%; range 47%–100%) on the 15-item, multiple-choice test. The postclerkship CLEO exercise results measuring application of disc mantra pattern recognition for 72 of these students (two scores disappeared electronically and proved irretrievable) resulted in a mean score of 78% (±9%; range 36%–96%) on a 25-point scale. Forty-one students scored ≥80%. Knowledge and CLEO scores did not correlate significantly (r = 0.24; P = .06). A combined score of 82% or above placed students in the top 15%, meriting an honors grade. When we calculated the post-FM clerkship CLEO score using the same elements assessed in the year's end CPX CLEO exam, the honors students scored only moderately higher (19.9/22, 90%) than the nonhonors students (17.98/22, 82%).

Year 3 clerkship patient write-ups

We captured and reviewed a total of 456 chart notes from the FM and IM clerkships. For 441 of these charts (227 from FM and 214 from IM), the eye findings pertained to the disease process and management.

In the FM clinic and some IM clinics, the preprinted patient encounter forms contained a “WNL” (within normal limits) check-off box. This form design template impeded our ability to determine which elements of the examination the students had actually performed. If the students had checked the box without listing specific findings in the note, we assumed that the element was not performed. Although 146 chart notes (64% ± 6%) of 227 FM workups and 8 (4% ± 3%) of 214 IM featured a checked WNL box, no further findings accompanied the checked WNL box in only 42 FM and 4 IM workups.

Among the clinical notes in which eye findings would be potentially contributory, inadequate eye examinations and management plans occurred at a higher rate in IM clerkships (195/214, 91%) versus FM clerkships (98/227, 43%). However, the average overall score on the clinical notes did not vary as a result of the sequence in which students rotated through the FM clerkship in relation to the IM clerkship. That is, students who completed the FM ophthalmic training module before the inpatient and outpatient portions of the IM clerkship yielded no carryover effect on the quality of those analyzable IM clinical notes (F = 0.06; P = .94). Also, the dramatic difference between documentation in FM and IM clerkships made tracking the charts of a single student between the two clerkships a pointless exercise; within a clerkship, students' three charts were of consistent character—either sloppy or meticulous.

Despite the presence of potential eye findings relevant to the patients' diagnosis and management, 4/227 of FM workups (2% ± 2%) and 13/214 of IM workups (6% ± 3%) contained no written entry at all for the eye examination. Funduscopy charting occurred appropriately in 105 of 227 relevant FM notes (46%), in which the WNL box was not the sole documentation, as compared with 15 of 214 of similar IM notes (7%). Only 27 of 227 FM notes (12%) and 6 of 214 IM notes (3%) documented that students considered a need for an ophthalmology consult despite further scrutiny clearly being warranted given the apparent cursory nature of the examination and the relevance of eye examination findings to the patient's underlying disease process.

Post-Year 3 clinical performance examination

We collected data on 91 students in the class who, having passed all Year 3 courses and Shelf examinations, qualified to take the CPX. In the CLEO funduscopic simulator interstation exercise, 100% of those 91 students documented visualization of at least part of the fundus through a dilated pupil. However, only eight students scored ≥20. Students scores ranged from 1 to 22 of a maximum of 22 points, for applying the disc mantra to the fundus slide in the dilated eye, and the mean score for the entire group was 12/22 (55%). This mean score declined 27 percentage points below that measured post FM (t = 9.99; P < .0005).

In comparison to the CLEO score obtained post-FM clerkship, only 11 students performed as well or better on this identical post-Year 3 CPX CLEO examination. Furthermore, application of the disc mantra did not correlate between clerkship and CPX settings (r = 0.18; P = .13). Students who scored at the honors level in the ophthalmology module of their FM clerkship achieved a higher level of performance on the CPX CLEO than their peers who posted nonhonors module scores, but the difference was not statistically significant: 63% versus 54% (t = 1.18; P = .24). CPX funduscopic simulator performance did not significantly differ by the timing of the FM clerkship during Year 3 (F = 1.12; P = .347). Compared with peers completing the clerkship within six months of the CPX, students completing the FM clerkship more than six months before the CPX examination showed approximately a 10% lower funduscopic simulator performance score, a nonsignificant difference (t = 1.84; P = .07; Figure 2). However, prior simulator training succeeded. Among the 91 students taking the examination, 13 students had not yet retrained with CLEO, either having deferred FM to Year 4 or not yet having experienced the clerkship retraining session before the May 5 examination. Students who had been retrained during the ophthalmology module scored scored statistically significantly higher than those who had not (55% versus 19%; P = .0005).

Figure 2
Figure 2:
The time from the ophthalmology module during the family medicine clerkship to the California Clinical Performance Exam (CPX). The group at the end of the academic year shown on at the far right had either not yet undergone the funduscopy retraining session, or had deferred the FM clerkship to the following year.

More granular analysis of student data for the dilated pupil revealed that 45 of 91 students (49%) described findings supportive of their correct differential diagnosis of hypertensive or diabetic background retinopathy. In addition, 12 students (13% ± 7%) incorrectly called the dilated eye with nonproliferative diabetic retinopathy “normal”; 10 students (11% ± 6%) incorrectly diagnosed papilledema; and 26 students (29% ± 9%) described findings that either bore little resemblance to the slide or specifically mentioned the absence of represented pathology.

Thirty students (33%) outlined an appropriate and justified management plan. An additional 27 (30%) advised a reasonable plan despite some minor conceptual flaws. Of the 91 students, 18 (20% ± 8%) called for a consult in the correct time frame, albeit some for specious reasons; 10 students (11% ± 6%) called for a consult without specifying a time frame; 26 students (29% ± 9%) recommended grossly inappropriate medical management; and 62 students (68% ± 10%) did not consider a consult when appropriate or advised one in the wrong time frame.

In the undilated eye with background and some subtle proliferative diabetic retinopathy, 90 of the 91 students (99%) described at least some fundus features, indicating some level of visualization. Only 1 student of these 91 recognized subtle neovascularization at the disc, but 32 students (35%) specified the absence of neovascularization. Twenty-four students (26% ± 9%) requested an ophthalmology consult for the correct reason and in the proper time frame. Forty students (44% ± 10%) did not advise involving ophthalmology in consultation. Twenty-one students (23%) indicated that they did not complete the exercise in the allotted 10 minutes because they “ran out of time.”

Year 4 subinternship

We located only 24 patient clinical chart notes from 21 (of the 36) students who completed their required IM subinternship at our institution. However, we recognized a trend even among these relatively few relevant clinical notes: whereas 20 of 21 writeups relevant to the eye (95%) documented extraocular muscle exams and 15 (71%) reported pupil exams, only 1 reported that a confrontation field was performed, and only 1 chart described conjunctivae. No funduscopies appeared. Half of the charts did indicate that “cranial nerves 2 to 12 were intact.”

Post-Year 4 questionnaire

In this graduating class of 94 students, the majority matched into a first-year residency training program in which ophthalmic screening examination skills are of fundamental importance; 69 (73%) soon began their residency training in either a primary care program or a one-year preliminary medicine program. The number of future ophthalmologists in the cohort2 was too small to impact the results. Among 86 anonymous student respondents, 78 (91%) “felt comfortable” with taking a visual acuity, 100% with checking pupils, 100% with checking extraocular muscle excursions, and 71 (83%) with checking confrontation fields. Only 50 (58%) “felt comfortable” assessing anterior chamber depth. Of 85 respondents, only 64 (75%) felt comfortable assessing the disc, 49 (58%) the macula, 57 (67%) the vessels, and 57 (67%) the retina. Overall, 10 of 84 students (12%) reported being “very confident” in their ability to perform the screening eye examination, 55 (65%) “somewhat confident,” and 19 (23%) “not at all confident.” As for interest in refresher training, of 83 respondents, 24 students (29%) responded that they were “very interested,” 47 (57%) that they were “somewhat interested,” and 12 (14%) were “not at all interested.”

Of 84 student respondents, 69 (82%) recalled that they had practiced ophthalmoscopy during their Year 3 FM rotation. By the end of medical school, only 20/86 (23%) had purchased an ophthalmoscope at any time. Students' explanations (N = 83) for lack of ownership included prohibitive cost (38, 46%), ability to borrow one when needed (12, 14%), lack of importance for clinical duties (13, 16%), and simply a dearth of opportunities for use in clinical encounters (5, 6%). However, about one half intended to purchase an ophthalmoscope in the future.

Twenty-one of 86 students (24%) responding to the question stated that they took a one- to four-week elective in ophthalmology, an enrollment rate similar to that previously described.4 In contrast, data from medical education rosters indicated that only 16 of 91 students (18%) registered in at least one in-house or extramural elective. Many more students took an ophthalmology elective than those who enrolled in most marginalized subspecialty electives, with the major exception being dermatology: Among 91 students, 20 (22%) took one or more dermatology electives, 2 (2%) took at least one otolaryngology elective, and 6 (7%) took at least one orthopedic elective.

The majority (60 of 80 responding Year 4 students, 75%) considered physical examination findings of equal importance to laboratory testing or imaging in establishing a diagnosis; 8 (10%) felt the physical examination to be more important; and 6/79 (8%) thought the lab results more valuable.

Discussion

We prospectively evaluated the effects of a sustained, longitudinal curriculum for the acquisition and retention of ophthalmic screening examination skills in a single class cohort. The initial acquisition of target skills in Year 2 preclerkship training and the subsequent reinforcement in the Year 3 FM clerkship yielded positive skills learning outcomes. However, eye examination skills decayed when not routinely applied and practiced in subsequent clinical rotations. In this study, the 46% of FM clerkship chart notes documenting funduscopy represents a significant improvement over the 11% observed in our pilot study with a prior class cohort.9 This supports previous findings of successful skill transmission9,13 and suggests, at least on a short-term basis, the added value of skills reinforcement during clinical training.

Comparing the scores on the CLEO simulation exercise obtained immediately after the FM clerkship with those from the post-Year 3 CPX showed a dramatic decrement. Moreover, honors grades in the ophthalmology module (granted to the top 15% of the class) correlated less than we expected with post-Year 3 CPX simulator scores: Both honors and nonhonors performances between the post-FM clerkship funduscopy and the post-Year 3 CPX funduscopy differed from each other by approximately 10% at each time point. Scores dropped by a substantial 27 percentage points by the year's end for honors students and by 28 percentage points for nonhonors students. CPX performance tended to vary inversely with the time elapsed since completing the FM clerkship. The “break point” for skills retention seemed to be at about six months, after which performance decayed by about 10 percentage points. Although the performance measured against time from funduscopic training during the ophthalmology module to the time of the CPX was not statistically significant, the trend suggested good short-term retention as measured at the end of the FM clerkship, but poorer long-term skill retention, as measured by the end of Year 3. One explanation might be the emphasis on focused examinations on the wards and in the clinics, which counteract reinforcement of all the separate components of a complete screening examination. Because only two students took an ophthalmology elective during Year 3, the sample size was simply too small to make conclusions about the effect on CPX scores of taking—or not taking—an ophthalmology elective before the CPX. The medical school registrar listed 16 students having taken an intramural or extramural elective in Year 4; the discrepancy with the pregraduation questionnaire number likely resulted from five students mistakenly equating the ophthalmology module in the Year 3 FM clerkship to an elective. A simulator reevaluation at the end of Year 4 to assess the effect of elective training would have been illuminating, but because of the dispersion of the class across the country for electives, the logistics made reassessment of the entire cohort impractical.

The rates of documented funduscopies in IM chart workups (7% in the clerkship and 0% in the subinternship) sharply contrast with those in the FM clerkship (46%). These findings might represent the broader view of FM versus the more focused view of IM. Alternatively, the data could exemplify the transitory nature of the skills habituation and retention in clinical education settings where little or no ophthalmology integration had yet been implemented; ophthalmology integration and faculty development were already under way in FM but had not yet been extended to IM at the time of the study (the program has since expanded to multiple other departments in both the preclinical and clinical years). Despite the disparity between clerkships in documented ophthalmoscopy performance, completion of the ophthalmology module before the IM clerkship did not have a significant effect.

Other factors that may have contributed to the lack of carryover effects of ophthalmic training include faculty comfort level with the examination, faculty devaluation of the potential contribution of eye findings in physical diagnosis, and precepting by subspecialist internists who might disregard the eye entirely in their assessment and grading of student performance. Further, the low rate of ophthalmoscope ownership reported by graduating seniors compounded with the variable availability of ophthalmoscopes in clinics and on inpatient floors likely explained the low performance rate, because students needed to hunt down an ophthalmoscope to examine the fundus. (As a result of this study, ophthalmoscope ownership is now required of all students in our institution.) However, the extremely low rate of screening confrontation visual fields performed (2% in FM and 6% in IM) is clearly not an equipment-dependent phenomenon.

The low charted performance rates of visual acuity, fields, and disc assessments contrast sharply with the students' relative confidence levels in their evaluations and their attitudes toward the importance of the physical examination. The most likely explanation is that if housestaff and faculty do not model, reinforce, or grade the performance of ophthalmic exams in actual clinical situations, the relevance of the eye quickly fades. Because 82% of the class would experience no more ophthalmology than is required, regardless of its applicability to their chosen field, housestaff then cannot help but fall short of primary care residency directors' expectations for competency in this skill set.

Our findings must be interpreted in the context of the study design. A longitudinal study of this scope and complexity inherently depended on a large number of people. Faculty turnover and factors we had not anticipated, like the preprinted WNL box on the clinic encounter templates, negatively impacted data retrieval, but our numbers might underestimate actual performance if indeed the students performed the examination elements but chose the WNL checkbox as a convenient shortcut.

Another major assumption of ours concerns the accuracy of charting as an indicator of what students actually do in a patient encounter. Szauter and colleagues14 found clear and significant discrepancies between observed, videotaped patient physical examinations and the notes that supposedly record them. Their data found overdocumentation in 82% of charted notes for recorded standardized patient encounters. They found that the exam maneuver was either not performed at all or was improperly performed to the extent that the finding could not have been elicited. The underdocumentation rate of elements performed ranged from 31% to 55%, depending on the organ system examined. Szauter and coinvestigators did not study ocular elements. In our study, because the workup had been specifically prepared as a graded assignment, students most likely would record more extensively, rather than less so. This suggests that the performance rate perhaps is worse than that indicated by the chart note audit.

The ephemeral nature of examination skills is likely not unique to ophthalmology. In the face of a vast array of facts and skills to master, many of today's medical students probably adapt according to the information and tasks demanded by a rotation or upcoming written test, only to move on to the next graded task with little durable cumulative learning. The compartmentalization of fact and task processing by rotation or discipline (“silo effect”) serves only to undermine the incremental development of clinical reasoning and examination skills necessary in the holistic care of the patient. Only by resisting the urge to relegate the entire eye examination to subspecialists will generalists be armed with the skills to detect findings that enhance their ability to diagnose medical disease and to recognize conditions truly requiring subspecialty consultation in a timely fashion.

In February 2007, a task force on medical education under the auspices of the Association of University Professors of Ophthalmology (AUPO) identified a core of ophthalmic knowledge base and diagnostic and procedural skill proficiency necessary for best medical practices for the undifferentiated physician. The AUPO Board ratified these recommendations in the fall of 2007, and in November 2008, the board of trustees of the American Academy of Ophthalmology added its unanimous endorsement for its use as a national standard of core competency for generalists. We urge that interdisciplinary integration of this recently published standard15—as applies to best practices of general medicine—be incorporated into curricula and skills assessment programs so that their relevance and value continue to be reinforced. In so incorporating them, medical educators can ensure that skills are learned and applied so they accrue, mature, and contribute to proficient patient care overall.

We suggest that this model—but with increased embedded pulsed skills reinforcement and assessment—may be applicable to other specialties (e.g., dermatology, otolaryngology, orthopedics) which, like ophthalmology, may be marginalized in the current required curricula of our medical schools.

Acknowledgments

The authors wish to thank C. Sue Ahearn, RN, for collecting California Clinical Performance Examination data; Alpesh Amin, MD, MBA, for collecting internal medicine clerkship data; David Morohashi, MD, for collecting family medicine clerkship data; and Nam Kim, MD, for collecting Year 4 subinternship data.

This study was funded in part by a University of California Irvine Medical Center Dean's Scholar grant (L.M.-L.) and an unrestricted departmental grant to the Gavin Herbert Eye Institute by Research to Prevent Blindness, New York, New York.

References

1Jackson C, Hirst L, Ambler JA, Battisttutta D. GPs and eye skills. A brave new world? Aust Fam Physician. 1997;26:409–411, 413–415.
2Ah-Chan JJ, Sanderson G, Vote BJ, Molteno AC. Undergraduate ophthalmology education survey of New Zealand ophthalmologists, general practitioners and optometrists. Clin Experiment Ophthalmol. 2001;29:416–425.
3Stern GA. Association of University Professors of Ophthalmology Education Committee. Teaching ophthalmology to primary care physicians. Arch Ophthalmol. 1995;113:722–724.
4AAMC Task Force on the Clinical Skills Education of Medical Students. Recommendations for Clinical Skills Curricula for Undergraduate Medical Education. Available at: (https://services.aamc.org/Publications/showfile.cfm?file=version56.pdf&prd_id=141&prv_id=165&pdf_id=56). Accessed July 16, 2009.
5Jacobs DS. Teaching doctors about the eye: Trends in the education of medical students and primary care residents. Surv Ophthalmol. 1998;42:383–389.
6Haik B, Hardy L, Mottow-Lippa L. Survey of U.S. ophthalmology departments. Fall 2004. Presented at: Annual Meeting of the Association of University Professors of Ophthalmology; January 2005; Indian Wells, Calif.
7Bellan L. Ophthalmology undergraduate education in Canada. Can J Ophthalmol. 1998;33:3–7.
8Gupta RR, Lam WC. Medical students' self-confidence in performing direct ophthalmoscopy in clinical training. Can J Ophthalmol. 2006;41:169–174.
9Mottow-Lippa LS, Boker J, Duke A, Amin A. A novel 3-year longitudinal pilot study of medical students' acquisition and retention of screening eye examination skills. Ophthalmology. 2006;112:133–139. (Correction to figure: Opthalmology. 2006;113:1056.)
10Mottow-Lippa, Boker JR. Funduscopic simulator assessment in 3 successive classes. J Acad Ophthalmol. In press.
11Lippa L. The Disc Mantra: An Algorithm for Fundus Description. AAMC MedEdPORTAL. April 18, 2008. Available at: (http://services.aamc.org/jsp/mededportal/retrieveSubmissionDetailById.do?subId=1086). Accessed July 16, 2008.
12Dodaro NR, Maxwell DP. An eye for an eye. A simplified model for teaching. Arch Ophthalmol. 1995;133:824–826.
13Quillen DA, Cantore WA. Impact of a 1-day ophthalmology experience on medical students. Ophthalmology. 2006;113:2307–2309.
14Szauter KM, Ainsworth MA, Holden MD, Mercado AC. Do students do what they write and write what they do? The match between the patient encounter and the patient note. Acad Med. 2006;81(10 suppl):S44–S47.
15Mottow-Lippa L. Ophthalmology in the medical school curriculum: Reestablishing our value and effecting change. Ophthalmology. 2009;116:1235–1236.
© 2009 Association of American Medical Colleges