Secondary Logo

Journal Logo

Development and Validation of a Novel Ear Simulator to Teach Pneumatic Otoscopy

Morris, Elizabeth BS; Kesser, Bradley W. MD; Peirce-Cottler, Shayn PhD; Keeley, Meg MD

Simulation in Healthcare: The Journal of the Society for Simulation in Healthcare: February 2012 - Volume 7 - Issue 1 - p 22–26
doi: 10.1097/SIH.0b013e31822eac39
Empirical Investigations
Free

Introduction Otoscopy is an important skill in diagnosing conditions of the middle ear. This study evaluated the ability of a novel ear simulator to teach medical students diagnostic and pneumatic otoscopy. We hypothesized that exposure to this simulator improves the ability of medical students to apply an appropriate pneumatic pressure during insufflation and accurately identify the presence of a middle ear effusion in a simulated setting.

Methods An ear simulator was created to teach otoscopic skills to medical students. Third-year medical students attended a workshop on pediatric otoscopy, including a demonstration, videos, and verbal instruction on otoscope use. A cohort of these students then practiced pneumatic otoscopy with the simulator. All students, as well as a group of experts who had not been exposed to the trainer, then diagnosed the presence or absence of middle ear fluid in six simulator ears and pneumatic pressures generated were recorded.

Results Interaction with the simulator enabled students to insufflate with pressures in the proper range (0.4–20 in H2O) more often, apply an average pressure (12.7 in H2O) in the appropriate range, and diagnose middle ear fluid more accurately (79.2%) than students who were not exposed to the trainer (57.3%). The students exposed to the simulator also performed closer to the level of the experts who diagnosed presence of effusion 100% accurately with an average insufflation pressure of 3.8 in H2O.

Discussion Medical students trained with the ear simulator applied appropriate pneumatic pressure more consistently and diagnosed the presence of effusion more accurately and more like experts than students not exposed to the simulator. This ear trainer is a valuable tool for teaching pneumatic otoscopy.

From the Departments of Biomedical Engineering (E.M., S.P.-C.), Otolaryngology (B.W.K.), and Pediatrics (M.K.), University of Virginia, Charlottesville, VA.

Supported by Walter H. Coulter Foundation and the University of Virginia School of Medicine Academy of Distinguished Educators.

Drs. Keeley, Kesser, and Peirce patented the ear trainer and the University of Virginia licensed it to Nasco, Inc. They receive small royalties from its sale.

Reprints: Dr. Meg Keeley, Department of Pediatrics, University of Virginia Health System, PO Box 800745, Charlottesville, VA 22908 (e-mail: mmg4z@virginia.edu).

Acute otitis media is one of the most common infectious diseases in childhood and remains the most common reason antibiotics are prescribed for children in the United States.1 Otitis media, or inflammation of the middle ear, occurs frequently in young children and infants because of their shorter and more horizontal eustachian tubes in comparison to adults and because of a greater exposure to viruses and bacteria that cause upper respiratory infections.2,3 It is essential to correctly identify children with acute otitis media, especially in consideration of increasing antibiotic resistance; however, the diagnosis is often challenging. Lack of skill in this area leads to significant overdiagnosis of acute otitis media and resultant antibiotic overprescription, unnecessary surgical referrals, and significant associated costs.

Pneumatic otoscopy is a widely accepted procedure used to identify middle ear effusion and is specifically recommended in the American Academy of Pediatrics/American Academy of Family Practitioners clinical practice guidelines for diagnosis of acute otitis media.4 However, in a survey of practicing physicians in a research network, 60% never used pneumatic otoscopy or used it less than half the time.5 Researchers agree that pneumatic otoscopy is the most accurate way to diagnose otitis media,6–8 yet there currently exists no training device to teach students how to safely and effectively perform this important clinical skill.

Diagnosing conditions of the middle ear requires basic clinical skills performed most often in Pediatrics, Otolaryngology, Family Medicine, and Emergency Medicine. Inexperienced clinicians often misdiagnose middle ear pathology because of inadequate training and assessment.9–11 The small size of the ear canal creates a challenge for faculty who teach ear diagnosis to students and residents. Furthermore, pediatric patients are often uncooperative subjects; shared viewing is limited to video otoscopy. Existing simulators lack anatomic accuracy and use photographs of middle ear findings and therefore can teach only a limited number of diagnostic techniques [EAR Examination Simulator, Kyoto Kagaku (Kyoto, Japan), Life/Form Ear Examination Simulator and Basic Nursing Set, Nasco (Fort Atkinson, WI)]. The lack of a simulated tympanic membrane in these models prevents instruction and practice of pneumatic otoscopy in controlled settings. Therefore, teaching medical students and novice residents how to diagnose common conditions such as acute otitis media and otitis media with effusion is difficult and hampered by a lack of tools for teaching and learning assessment.

To perform pneumatic otoscopy, a clinician uses an insufflator bulb attached to an otoscope and applies air pressure to the tympanicmembraneto evaluate itsmovement.If the pressure in the middle ear space is the same as ambient pressure, a normal eardrum will visibly move, while an eardrum with an effusion in the middle ear space will have reduced or no mobility. The importance of pneumatic otoscopy is that when consistent pressures are used, it allows the examiner to assess presence of effusion in the middle ear space. This is often challenging in the pediatric patient who may be crying, poorly cooperative, and febrile. Their tympanic membranes are often injected or red, but if no effusion is present, there is no diagnosis of otitis media. Inexperienced clinicians who do not initially see movement might then apply increasing amounts of pressure to achievemobility. Thus, in an attempt to generate visible movement of the eardrum, excessive application of pressure may be applied, resulting in misdiagnosis and possible patient discomfort.

The difficulty experienced by students and trained medical practitioners in performing ear diagnosis is well documented in the literature. One study showed that only 5% of third-year medical students are confident with ear diagnosis upon completing their pediatric clerkships.7 Acomparison of the performance by otolaryngologists, pediatricians, and general practitioners from three countries on an otoendoscopic diagnostic video examination found that only approximately 50% of the diagnoses were correct.12 Another study found that trained individuals were able to accurately identify middle ear effusion in 94% of cases and accurately identify an ear without fluid in 78% of cases.13 These subjects often experienced difficulty in evaluating the hypomobile tympanic membrane. The threshold for visible movement of the eardrum ranges from 0.4 to 0.6 in H2O, but clinicians applied pressure pulses that varied from 6.7 to 20.5 in H2O.14 Thus, even the weakest insufflations were an order of magnitude higher than the threshold for detection of movement in a normal eardrum and middle ear space. A similar study found that skilled otoscopists generated pressures that ranged from 13.3 to 44.6 in H2O, with a mean of 29.4 in H2O with the bulb and 19.8 in H2O through the mouthpiece. The mean variation in pressure delivered was 7.8 and 4.9 in H2O with these two methods of insufflation, respectively; again, much higher pressures were used than are needed to move the tympanic membrane.15

To provide instructors with a realistic and interactive teaching simulator, our team has designed, prototyped, and licensed a novel ear trainer (Fig. 1), which can be used to teach both diagnostic and surgical otoscopic procedures to medical students, residents, and practitioners. Three of the investigators in this study receive small royalties from the sale of the simulator based on our work. Specifically, this trainer can be used to teach and evaluate students’ abilities to diagnose common ear pathologies, perform pneumatic otoscopy, extract ear canal foreign bodies, and perform myringotomy with ventilation tube insertion. Construct validity of this simulator has been established for myringotomy with ventilation tube insertion.16 We now focus on evaluating the simulator’s construct validity for pneumatic otoscopy and diagnosis of middle ear effusion.

Figure 1

Figure 1

We hypothesize that medical students with access to training on the novel ear simulator will perform better than medical students without training and closer to expert performance when assessed on the simulator. Our long-term expectation is that more effective teaching of proper pneumatic otoscopy technique with the use of a simulator will enable students to make more accurate diagnoses in clinical settings.

Back to Top | Article Outline

METHODS

Ear Trainer Design, Development, and Commercialization

The development of the ear simulator was a multidisciplinary effort by undergraduate engineering students, medical students, medical residents, and faculty from the Departments of Biomedical Engineering, Otolaryngology, and Pediatrics at the University of Virginia. The engineering design process, which spanned a 4-year time period, proceeded through an iterative manner and included the following steps: needs identification, conceptual design, prototyping, testing, design for manufacturing, and intellectual property protection. The trainer (US Patent #7,654,321 granted December 2010) was licensed to Nasco, Inc. (Fort Atkinson, WI) and is commercially available (Figs. 1 and 2). Design and construction of the basic elements of the ear trainer, which consists of amannequin head, auricle, ear canal, and cartridge containing the eardrum and “middle ear space” (Fig. 2), are presented in the article by Volsky et al.16 Air pressure and fluid mimicking effusion was applied to the fluid-tight chamber in the removable cartridge that simulates the middle ear space (Fig. 2B) through a small port at the back of the chamber that is connected to tubing and a syringe.

Figure 2

Figure 2

The inclusion of a pressure gauge (D1005PS Digital Pressure Gauge; part number 25D1005PS; Ashcroft, Inc., Huntington Beach, CA), which is also commercially available as an add-on item for the ear trainer through Nasco, Inc., was vital in this study for the measurement of pressure applied during pneumatic otoscopy. The “max pressure” feature of the pressure gauge was used to display the maximum pressure applied during the simulated examination. The “ideal range of pressure” was determined to be 0.4 to 20 in H2O based on the literature.

Back to Top | Article Outline

Ear Trainer Validation: Controlled Cohort Study With Medical Students

This study was approved by the University of Virginia Institutional Review Board (#2009-0176-00). All third-year medical students at the University of Virginia School of Medicine attended a workshop (supervised by M.K.) on the pediatric ear examination during their pediatric clerkship. This 90-minute nongraded workshop, in groups of 10 to 12 students, is taught by M.K. and includes anatomy review, patient positioning, instruction in equipment use, cerumen removal, discussion of findings and diagnoses, and video of pneumatic otoscopy examinations. All students are taught to identify middle ear effusions and how to distinguish between acute otitis media and otitis media with effusion. Attendance at this workshop has been shown to improve performance on a standardized clinical examination, but the skill of pneumatic otoscopy was not previously practiced.17 Groups of students are preassigned to workshop dates based on rotation location. The first group of one rotation and the second group of the next rotation (n = 21) used the simulator for 15 minutes of the workshop. These students examined simulated tympanic membranes with and without effusions. They received immediate feedback from the instructor regarding position and equipment use as well as feedback regarding pressures generated from the manometer readings. The second group of the first rotation and first group of the second rotation (n = 20) served as the control cohort. They did not practice with the simulator and spent that time watching videos of pneumatic otoscopy examinations with instructor explanation. The two cohorts represent 30% of their medical school class.

All students in both groups were then asked to determine the presence of an effusion in six ears of three ear simulators. The nonintervention group received a brief introduction to the mechanics of the simulator. The six simulator ears were filled with varying amounts of water, representing an effusion. Two ears contained no fluid, two ears contained 1.0 mL of fluid (partial effusion), and two ears were filled with 1.5 mL of fluid (complete effusion) (Fig. 2B, left, middle, and right images, respectively). Each head had different findings, which did not vary for each group. The students progressed through the heads in random order. The subject’s determination of whether or not an effusion was present (yes or no) was recorded for each ear by E.M., and the maximum pressure applied during pneumatic otoscopy was recorded for three ears, including one representing each of the three conditions. The pressure sensor has a setting that displays the maximum pressure achieved over a period, and this was reset before each subject performed pneumatic otoscopy on the ear simulator.

Back to Top | Article Outline

Ear Trainer Validation: Expert Validation

The same test as described above was then administered to a group of five experts by E.M. These subjects were all faculty attending pediatricians in a General Pediatric teaching clinic with 4 to 25 years of experience. The experts did not attend a workshop on ear diagnosis or practice on the ear trainer but did receive a brief introduction to the mechanics of the simulator. The experts were asked to diagnose the presence or absence of an effusion in the ear trainer, and the maximum pressures they generated were recorded.

Back to Top | Article Outline

Statistical Analysis

Data were compiled by E.M. for three groups of subjects: medical students with exposure to the ear trainer (n = 21), medical students with no exposure to the ear trainer (n = 20), and experts (n = 5). Each subject made six diagnoses (six trials), and their maximum insufflation pressures were recorded for three of these trials. Maximum pressures were averaged for each group and reported_standard deviations. Because the data failed the equal variance test, a nonparametric Kruskal-Wallis one-way analysis of variance on ranks was performed and the data passed with P < 0.001. To determine whether the different study groups were statistically different from one another, the Dunn’s Method for multiple comparisons was subsequently performed.

Back to Top | Article Outline

RESULTS

Instruction and practice with the ear trainer enabled students to identify an effusion with significantly greater accuracy (79.2%) than students who did not practice with the ear trainer (57.8%, P = 0.02). Notably, experts made 100% accurate diagnoses.

The group of students that trained with the ear simulator also demonstrated an enhanced ability to insufflate the ears with a pressure in the proper range (0.4–20 in H2O) more often (83% of the time) than the group of students that did not practice on the ear trainer (32% of the time). Experts insufflated with an average pressure in the appropriate range 100% of the trials. Exposure to the ear trainer enabled students to apply an average pressure of 12.7 in H2O (Fig. 3). This value is lower than the average pressure applied by students who were not exposed to the ear trainer (27.1 in H2O), and closer to the average value achieved by experts (3.8 in H2O), and within the proper range (0.4–20 in H2O). Importantly, medical studentswhowere not exposed to the ear trainer insufflated with significantly more pressure than medical students who were exposed to the ear trainer (P < 0.05).

Figure 3

Figure 3

Back to Top | Article Outline

DISCUSSION

Accurate diagnosis of middle ear conditions has a significant impact on pediatric healthcare in terms of costs, antibiotic use, surgical intervention, and societal burden. The literature affirms that distinguishing between acute otitis media and otitis media with effusion is difficult even for experienced practitioners. Moreover, teaching students how to diagnose middle ear effusion using safe pneumatic otoscopy technique has been an unmet challenge with long-term consequences that only amplify the frequency of misdiagnoses and antibiotic overuse in clinical practice. Until our ear trainer became commercially available in 2009, there were no simulation tools for teaching pneumatic otoscopy. This study suggest that students who received simulation training with our device performed more like experts than those who did not receive simulation training and implies that the trainer is capable of delineating between experts and nonexperts. This delineation supports construct validity of the simulator for the procedure of pneumatic otoscopy.

Specifically, students trained on the ear simulator applied a more appropriate insufflation pressure and diagnosed the presence of middle ear effusion more accurately and more like experts than students who did not practice on the ear trainer. Moreover, experts performed better than either student group across all metrics. These results support construct validity of the simulator but with a cautionary word: our study did not evaluate students’ abilities to make accurate diagnoses in clinical settings (ie, predictive validity). Nevertheless, this study represents an important first step toward enhancing the teaching of pneumatic otoscopy and improving students’ diagnostic acumen in common ear pathology.

The use of this novel simulator includes many aspects of best simulation education practices. Students receive immediate feedback from the instructor regarding position and equipment use as well as feedback regarding pressures generated from the manometer readings. The simulator offers the opportunity for repetitive practice in a controlled environment, which is not possible in the pediatric patient. Use of the simulator was integrated into the clinical skills workshop curriculum as one of multiple learning strategies. Although this study focused on the learning outcome of identification of effusion and use of pneumatic otoscopy, the simulator also allows learners to practice surgical skills and foreign body/cerumen removal and to make diagnoses based on photographs. This study was performed to ensure validity as a teaching tool.18

Future research is needed to validate the impact of the ear trainer on student performance in clinical settings and on patient outcomes. A limitation of this study is that students trained on the simulator were then tested on the same tool. The familiarity and repetition with the trainer likely contributed to the observed enhancement in performance over that of their naive classmates. Although, the two groups should have had similar previous experience with the ear examination, we did not do any baseline comparison, which is a limitation. We tried to make the overall workshop time the same for both groups. The exposure group had time practicing on the simulator with instructor feedback while the nonexposure group spent that time watching more videos of pneumatic otoscopy with instructor explanation. The nonexposure groups were introduced to the mechanics of the model but they did not practice with it. We did not document performance during the exposure group practice session, but by observation, their performance did improve as they practiced and received feedback from the instructor and manometer. Performance in testing likely reflects some of this familiarity with the simulator but we would then hope that this translates to familiarity in patient care, which is why we developed the model.

Also, because otitis media was deliberately chosen as our disease focus, we simply tested for the ability to diagnose the presence or absence of middle ear fluid. We did not assess students’ abilities to distinguish acute otitis media from otitis media with effusion. Thus, future validation studies designed to evaluate the effectiveness of the trainer to teach diagnosis of additional disease states should obviously incorporate a broader range of ear pathologies. Another limitation is that, as noted, three of the study investigators (M.K., B.K., and S.P.) received small royalties from the sale of this simulator. However, the investigator who performed all the assessments, data collection, and data analysis (E.M.) has no financial interest in the model.

Construct validation of this novel ear trainer sets the stage for future studies. We are now conducting a multi-institutional study to determine whether the student demographic or institutional culture impacts its utility and to significantly increase the subject numbers. Future research should also include predictive studies where students’ diagnostic skills are assessed in the clinical setting as opposed to in the model itself.

This study suggests that our novel ear trainer is an effective simulation-based didactic tool with a broad target audience in medicine, nursing, and audiology. Because of the widespread occurrence of ear complaints in the pediatric population in the setting of increasing antibiotic resistance, it is imperative that practitioners at all experience levels perform the appropriate procedural skills, such as pneumatic otoscopy, to deliver accurate ear diagnoses. The American Academy of Pediatrics/ American Academy of Family Practitioners guidelines recommend “instruction in the proper examination of the child’s ear should begin with the first pediatric rotation in medical school and continue throughout postgraduate training.”4 Effective introductions to ear diagnoses, particularly through simulator-aided training experiences at an early point in medical education, will strategically improve the diagnosis and management of otitis media.

Back to Top | Article Outline

REFERENCES

1. Rovers M, Schilder A, Zeilhuis G, Rosenfeld R. Otitis media. Lancet 2004; 363: 465–473.
2. Paradise JL. Otitis media in infants and children. Pediatrics 1980; 65: 917–943.
3. Bluestone CD, Shurin PA. Middle ear disease in children. pathogenesis, diagnosis, and management. Pediatr Clin North Am 1974; 21: 379–400.
4. American Academy of Pediatrics Subcommittee on Management of Acute Otitis Media. Diagnosis and management of acute otitis media. Pediatrics 2004; 113: 1451–1465.
5. Vernacchio L, Vezina RM, Mitchell AA. Knowledge and practices relating to the 2004 acute otitis media clinical practice guideline: a survey of practicing physicians. Pediatr Infect Dis J 2006; 25: 385–389.
6. Takata GS, Chan LS, Morphew T, Mangione-Smith R, Morton SC, Shekelle P. Evidence assessment of the accuracy of methods of diagnosing middle ear effusion in children with otitis media with effusion. Pediatrics 2003; 112 (6 Pt 1): 1379–1387.
7. Jones WS, Kaleida PH. How helpful is pneumatic otoscopy in improving diagnostic accuracy? Pediatrics 2003; 112 (3 Pt 1): 510–513.
8. Legros JM, Hitoto H, Garnier F, Dagorne C, Parot-Schinkel E, Fanello S. Clinical qualitative evaluation of the diagnosis of acute otitis media in general practice. Int J Pediatr Otorhinolaryngol 2008; 72: 23–30.
9. Marchisio P, Mira E, Klersy C, et al.. Medical education and attitudes about acute otitis media guidelines: a survey of Italian pediatricians and otolaryngologists. Pediatr Infect Dis J 2009; 28: 1–4.
10. Steinbech W, Seetish T, Benjamin D, Chang K, Messner A. Pediatric residents clinical diagnostic accuracy of otitis media. Pediatrics 2002; 109: 993–998.
11. Coker T, Chan L, Newberry S, et al.. Diagnosis, microbial epidemiology, and antibiotic treatment of acute otitis media in children: a systematic review. JAMA 2010; 304: 2161–2169.
12. Pichichero ME, Poole MD. Comparison of performance by otolaryngologists, pediatricians, and general practioners on an otoendoscopic diagnostic video examination. Int J Pediatr Otorhinolaryngol 2005; 69: 361–366.
13. Gates GA. Cost-effectiveness considerations in otitis media treatment. Otolaryngol-Head Neck Surg 1996; 114: 525–530.
14. Clarke LR, Wiederhold ML, Gates GA. Quantitation of pneumatic otoscopy. Otolaryngol Head Neck Surg 1987; 96: 119–124.
15. Cavanaugh RM Jr. Pediatricians and the pneumatic otoscope: are we playing it by ear? Pediatrics 1989; 84: 362–364.
16. Volsky PG, Hughley BB, Peirce SM, Kesser BW. Construct validity of a simulator for myringotomy with ventilation tube insertion. Otolaryngol Head Neck Surg 2009; 141: 603–608.e1.
17. Corbett EC Jr, Payne NJ, Bradley EB, Maughan KL, Heald EB, Wang XQ. Enhancing clinical skills education: University of Virginia School of Medicine’s Clerkship Clinical Skills Workshop Program. Acad Med 2007; 82: 690–695.
18. Issenberg SB, et al.. Features and uses of high-fidelity simulations that lead to effective learning: a BEME systematic review. Med Teacher 2005; 27: 10–28.
Keywords:

Otoscopy; Pneumatic otoscopy; Task trainer; Acute otitis media; Otitis media with effusion

© 2012 Society for Simulation in Healthcare