See “Considerations for Physician Training in a New Era of Medical Education” by Robson on page 111.
Educating residents and medical students while simultaneously balancing patient care responsibilities is an arduous task faced by many clinician-educators. Pediatric residents acquire training experience in pediatric gastroenterology through core pediatric subspecialty rotations, and medical students are exposed to intestinal and liver disease while they participate in pediatric inpatient services within their pediatric clerkships. Subspecialists responsible for teaching these learners on the inpatient pediatric gastroenterology rotation must do so while also addressing the care and management of the current inpatients, typically in the face of significant time limitations. Therefore, an effective and efficient curriculum is imperative to the success of these clinician-educators. The potential curriculum for a pediatric gastroenterology rotation is broad and ranges from rare conditions to gastrointestinal aspects of general pediatrics. There is an array of common gastrointestinal conditions, such as functional constipation and uncomplicated infantile gastroesophageal reflux, which are typically within the scope of practice of general pediatrics (1,2). Trainees should become comfortable in identifying and managing these conditions but also must learn to identify many other gastrointestinal conditions, which require appropriate urgent or emergent evaluation and subspecialty referral. Educating pediatric residents and medical students in a core gastroenterology curriculum is important to their future competence in managing patients with gastrointestinal complaints.
There are several obstacles to teaching on an inpatient rotation, with faculty time and resident availability being 2 important factors. In addition, timing of resident availability for learning may not correlate well with faculty availability. Face-to-face faculty teaching typically occurs during the daytime, whereas residents may be more available and ready to learn at other times. Availability of a flexible, well-organized curriculum framed within a clinical context can help overcome scheduling conflicts. Resident work hour restrictions implemented by the Accreditation Council for Graduate Medical Education (3) in 2003 and further restricted in 2010 have made the task of educating residents more time constrained. Residents and program directors report that with decreased work hours, there have been decreases in educational time and satisfaction (4,5) with some programs noting decrease in educational activities by up to 50% (6). Therefore, novel ways to educate residents must be explored.
Computer technology such as Web-based modules can present reliable, reusable content in a format that is convenient to the learner (7). It allows residents to learn the material at their own pace and convenience and can make the material more relevant to learners than the isolated reading of text (8). A variety of teaching methods such as multimedia, cases, questions, and other techniques can be used (9). Multiple studies have shown that Web-based learning is both effective and well received by learners (8,10–12). A combination of teaching methods using computer modules and faculty debriefings constitutes a blended approach to medical education. The use of blended learning has even further advantages by supporting the dissemination of information in a form preferred by the learner and then promoting an opportunity for discussion and practice in a group environment (13).
Little is written on resident education and curriculum development during the pediatric gastroenterology, hepatology, and nutrition rotation. Pediatric guidelines for core rotations focus on suggested goals and objectives but do not direct the specific teaching methodology (14). The Accreditation Council for Graduate Medical Education Outcomes Project Toolbox includes recommendations that the use of simulations, models, and cases are preferred choices in the assessment of medical knowledge (15).
We developed a case-based, online curriculum for the pediatric gastroenterology rotation with a focus on primary care pediatric medical knowledge. Components of the curriculum include 8 case-based modules, such that 2 modules are scheduled to be completed each week by the learner and subsequently reviewed at a weekly didactic session with an instructor. During these faculty-facilitated discussions, key points from the modules are discussed and the learners have the opportunity to use the knowledge gained from the modules in new clinical case scenarios. The curriculum is completed during a 4-week period. The curriculum was evaluated for effectiveness, learner satisfaction, and effect on evaluations of the rotation.
The process of curriculum development followed the Kern model: problem identification and needs assessment, establishing objectives and educational strategies, implementation, and evaluation (16). The author (M.F.), along with gastroenterology faculty and fellows, identified the need to improve learning on the gastroenterology rotation and to make the educational experience more consistent from month to month. A formal needs assessment was conducted, consisting of a survey of 30 residents who had recently completed the pediatric gastroenterology rotation. Residents were asked to score the amount of teaching received on 16 different topics as well as their comfort level in managing these topics after completing the rotation, using 5-point Likert scales. Survey topics were chosen based on the content in the Educational Guidelines (14). The results of the needs assessment, along with the changing educational environment, supported the need for educational reform at our institution and led to the development of this case-based, Web-delivered curriculum.
Goals were set to meet the needs of primary care pediatricians when encountering patients with gastrointestinal complaints, based on Academic Pediatric Association subspecialty content goals, our institution's rotation goals, and the results of the resident survey. Learners should acquire the knowledge necessary to recognize, diagnose, and treat common gastrointestinal conditions and to recognize conditions that require urgent care and/or consultation with a pediatric gastroenterologist. The curricular strategy used blended learning by combining independent use of computer modules with in-person didactic sessions with faculty to reinforce the material.
The curriculum consists of 8 units: gastroesophageal reflux, constipation, diarrhea, gastrointestinal bleeding, abdominal pain, nutrition, liver disease I (neonatal liver disease, acute hepatitis, hepatic failure), and liver disease II (chronic liver disease). Each unit consists of a Web-based module hosted on the university server and a corresponding half-hour, didactic session. The online modules were developed using EDACTIC, an institution-based authoring software platform that packages content (text, graphics, pictures, questions) into a user-friendly computer module accessible via the Internet. Every module begins with learning objective and ends with key points, which reinforce the learning objectives. Within each module, there are 2 or 3 cases followed by interactive questions, which provide learner feedback for both correct and incorrect answers. Multiple choice, true-false, and matching questions are used. There are annotated radiographic images embedded within the modules and links that provide additional information about topics within the modules. Each module can be completed in approximately 30 minutes, based on observed sessions of learners completing the modules as well as user feedback. During the rotation, learners are expected to complete 2 modules per week at their own convenience. This self-study is accompanied by a weekly interactive hour-long didactic session with an instructor. During these sessions, key points from that week's 2 assigned modules are discussed and new clinical cases are presented to reinforce the learners’ newly acquired knowledge. The learners formulate an assessment and plan for the patient in the new case scenarios. A power-point presentation as well as a script for each session is provided by the curriculum designer (M.F.) to the instructors to promote consistency and ease of delivery. The entire curriculum is completed during a 4-week period.
Evaluation of Curriculum
The gastroesophageal reflux module was developed first and pilot tested in April 2005, using a 1-group pretest/posttest design and a satisfaction survey. Based on the encouraging results of the first module and didactic session pilot, the remainder of the course content was developed. The entire curriculum was pilot tested using a similar format from February to May 2006. Pilot testing of the curriculum included an evaluation of both efficacy and learner satisfaction. Efficacy was tested using a 1-group pretest/posttest design, and learner satisfaction was tested using a satisfaction survey completed at the end of the educational experience. The pretest and posttest, which were identical and consisted of 33 multiple-choice questions, were developed using 4 to 5 learning objectives from each of the modules. Test questions were written by the curriculum designer (M.F.) with the help of the department faculty content experts who also edited course content of each module. The learners were not permitted to keep the pre-test after completion and were not given feedback or results before the post-test. Scores were compared using paired t test. The satisfaction survey consisted of 8 questions using a 5-point Likert scale, which asked about teaching methods, course format, relevance to primary care, comfort in managing gastrointestinal complaints, and satisfaction with the educational experience. All of the learners were also asked whether the content was appropriate for their level of training, and to list strengths and weaknesses of the educational experience (survey available upon request).
To assess effect on learner perceptions of the gastroenterology rotation at our institution, rotation evaluations from the year before implementation of the modules and for the 3 years after implementation were compared in multiple categories. The evaluations from the gastroenterology rotation were also compared with data from all of the pediatric rotations. During those years, the residency program employed E*Value Healthcare Education Solutions, an electronic residency evaluation system (www.e-value.net). All of the residents were assigned a postrotation evaluation, which included 11 items on a Likert scale of 1 (poor) to 5 (excellent), including assessments of clinical care and learning within the rotation (instrument available upon request). An unpaired 2-sample t test was performed for all of the comparisons. P value of <0.05 was considered significant.
This project was submitted to the Indiana University-Purdue University Indianapolis institutional review board and was approved as an exempt research study.
Fifteen of the 30 residents completed the needs assessment survey. The results indicated that the educational experience varied greatly from month to month, and residents were not comfortable managing most gastrointestinal conditions after completing the rotation, with only 2 of 15 residents indicating that they felt comfortable in managing more than half of the surveyed topics after completing the rotation. Resident comments suggested that didactic teaching was often limited by conflicting schedules of the faculty and residents.
The pilot test of the gastroesophageal reflux module was completed by 6 learners (4 pediatric residents and 2 medical students). Posttest scoring showed acquisition of knowledge, and the learner satisfaction survey revealed that all of the learners were satisfied with the educational experience, including the didactic session. All of the learners reported that they felt comfortable managing patients with gastroesophageal reflux after completing the pilot.
Twenty-one learners (6 medical students, 7 interns, and 8 upper-level residents) were studied in the implementation of all of the 8 modules. The data for pre- and posttest scoring were assessed using paired t testing and mean scores and standard deviation were calculated. The difference of the means for pretest scoring (73% ± 7.3%) and posttest scoring (92% ± 6.2%) showed a significant increase in knowledge acquisition (P ≤ 0.001) with a 95% confidence interval for the difference of the means of 15.8 to 22.7 (Fig. 1). For the satisfaction survey, mean and standard deviation were tabulated for each question using the 5-point Likert scale. The learners’ response to how well they liked the course format was extremely positive (4.90 ± 0.30), and learners reported that the didactic sessions with the instructor were important (4.62 ± 0.59) to the educational experience. The learners reported that the content was highly relevant (4.76 ± 0.44) to the practice of primary care pediatrics, and that they were extremely satisfied (4.90 ± 0.30) with the educational experience. When asked to compare the teaching methods used for this curriculum to more traditional teaching methods such as lectures or required reading, learners unanimously preferred this method to more traditional methods. Perhaps most important, learners reported that they felt comfortable taking care of patients with gastrointestinal complaints (4.00 ± 0.45). All of the learners, including the third-year medical students, indicated that the content was appropriate for their level of training.
Gastroenterology rotation E-values evaluations were then compared with those for all of the pediatric residency rotations during 4 years; each year there were 42 to 45 gastroenterology rotation evaluations and 1076 to 1460 evaluations of all of the rotations. At baseline, when gastroenterology was compared with all of the rotations during the preintervention year, there were no differences between gastroenterology and all rotations in any of the evaluation categories. Three postintervention years were then analyzed. The overall score for the gastroenterology rotation improved from 4.13 before the introduction of the curriculum to 4.31, 4.51, and 4.49 in the subsequent years, whereas the overall scores for all of the pediatric rotations were 3.96, 3.95, 4.07, and 4.08, respectively. Overall scores in each of the postintervention years were significantly better for the gastroenterology rotation than for all of the rotations (P < 0.05) (Fig. 2A). Further analysis of changes in each domain of the evaluations showed that the most improvement was seen in “rounds, presentations, and discussions” (Fig. 2B), “teaching by faculty” (Fig. 2C), and “self-directed learning” (Fig. 2D); however, benefits were also seen in “learning from patients” and “setting expectations.” gastroenterology improved significantly over time in all of the categories with the exception of “faculty coordination of workload.” Qualitative review of resident comments by year demonstrated positive mention of the modules by 29% to 57% of residents. Of note, the overall gastroenterology rotation ranking among the core residency rotations sequentially improved from 10th among 35 to 3rd of 33 in the 4 years of observation.
We describe a pediatric gastroenterology curriculum that uses Web-based education to deliver sound instruction to learners while limiting the amount of formal teaching time for the instructor. The results of the pilot test are encouraging and suggest that this curriculum is both effective and well received by learners. After completing the curriculum, the learners performed better on knowledge testing, perceived the experience as beneficial, and felt comfortable managing patients with gastrointestinal complaints. After implementation of the curriculum into the pediatric gastroenterology rotation at our institution, learner evaluations of the rotation improved significantly not only in domains directly related to the intervention but also in other aspects of the rotation.
The instructional methods within the curriculum are 2-fold. Learners have the opportunity for self-directed learning in both timing and pace during completion of the online modules, and key concepts are reviewed and reinforced by the instructor during the didactic sessions. During these small group sessions, learners readily demonstrate their degree of familiarity with the modules, creating accountability for module completion. They also have the opportunity to practice and further manipulate the knowledge gained from the modules with new clinical cases. The instructional design allows learners to be self-regulated and active participants in the educational process. The availability of the comprehensive curriculum to complement varied clinical activities ensures a uniform educational experience for the learners and contributes to a positive view of this rotation.
Other studies have evaluated the effects of both asynchronous learning and case-based instruction. The flexibility of online curricula results in increased use and knowledge acquisition (17,18), and learners have shown a preference for case-based teaching over non–case-based curricula (19). Bridgemohan et al (20) compared online multimedia teaching of childhood learning disorders with facilitated case discussions. Residents who completed the online teaching enjoyed the flexibility of the experience and the increased opportunities for active learning. Facilitated case discussion participants reported that the method made the material more relevant to clinical practice. Interestingly, our curriculum uses blended learning with both of these teaching methods, capitalizing on the strengths of both. Our study showed the benefits of blended learning on knowledge acquisition as well as learner attitudes.
Further work will need to be done to validate these findings. This pilot study involved small numbers of learners with no control group, and both residents and students were studied in combination. We did not establish whether all of the modules had been completed before testing/surveys; however, all of the learners did attend the didactic sessions. Because the same test was used for the pre- and postassessment, subjects in theory may have learned from the first test, although they were not given feedback on their performance or access to the test. Another limitation was that long-term retention of knowledge was not assessed. Other unmeasured factors may have affected the improvement in learner evaluations of the gastroenterology rotation: changes in faculty workload, rotation design, or educational approaches may have occurred and contributed to the improvement in evaluations.
Several key factors contributed to the successful development and implementation of the curriculum, although there were some challenges as well. The Academic Pediatric Association Educational Guidelines provided valuable curricular goals and content information. The EDACTIC software provided an excellent and user-friendly platform for module design. Significant time was required for both development and testing of the curriculum, requiring about 18 months for completion. Tasks included developing the modules, tests, and material for the interactive didactic session, piloting and subsequent editing of the components based on user feedback, and finally evaluation of the results. The main logistical challenge was organizing the didactic sessions for optimal learner attendance. Morning rounds, postcall regulations, and resident continuity clinics made scheduling difficult. Excellent administrative support from the pediatric gastroenterology office for scheduling of the sessions with e-mail reminders and pages was helpful.
In conclusion, we describe a unique method of educating residents and medical students about pediatric gastroenterology, hepatology, and nutrition. This curriculum, which uses Web-based education, was both effective and well received by learners and positively affected learner ratings of their educational experience on the gastroenterology rotation. Although the curriculum was initially intended to teach pediatric residents, it appears that the educational experience is also suitable for third- and fourth-year medical students. This type of curriculum uses less resident-to-instructor time with perhaps greater yield than a more traditional educational approach. It effectively addresses the limitations that work hour restrictions have placed on didactic opportunities. We believe that this curriculum could serve as a model to clinician-educators in a variety of subspecialties. We are presently exporting this product from our institutional EDACTIC format to an Adobe Presenter format so that the curriculum is readily available as a resource for others. Based on our success, we believe that this curriculum could be a valuable tool for any pediatric gastroenterologist who works as a clinician-educator as time for teaching becomes increasingly limited.
1. Baker SS, Liptak GS, Colletti RB, et al. Constipation in infants and children: evaluation and treatment. A medical position statement of the North American Society for Pediatric Gastroenterology and Nutrition. J Pediatr Gastroenterol Nutr
2. Orenstein SR. Gastroesophageal reflux. Pediatr Rev
3. Residency Review Committee for Pediatrics. Program Information for Continued Accreditation
. Chicago: Accreditation Council for Graduate Medical Education; 2002.
4. Jagsi R, Shapiro J, Weissman JS, et al. The educational impact of ACGME limits on resident and fellow duty hours: a pre-post survey study. Acad Med
5. Peterson LE, Johnson H, Pugno PA, et al. Training on the clock: family medicine residency directors’ responses to resident duty hour reform. Acad Med
6. Gelfand DV, Podnos YD, Carmichael JC, et al. Effect of the 80-hour workweek on resident burnout. Arch Surg
7. Choules AP. The use of elearning in medical education: a review of the current situation. Postgrad Med J
8. Rawson RE, Quinlan KM. Evaluation of a computer-based approach to teaching acid/base physiology. Advanc Physiol Edu
9. Cook DA, Garside S, Levinson AJ, et al. What do we mean by web-based learning? A systematic review of the variability of interventions. Med Educ
10. Bata-Jones B, Avery MD. Teaching pharmacology to graduate nursing students: evaluation and comparison of Web-based and face-to-face methods. J Nurs Educ
11. Kearns LE, Shoaf JR, Summey MB. Performance and satisfaction of second-degree BSN students in Web-based and traditional course delivery environments. J Nurs Educ
12. Marchevsky AM, Relan A, Baillie S. Self-instructional “virtual pathology” laboratories using web-based technology enhance medical school teaching of pathology. Hum Pathol
13. Wood BP. Blended learning in medicine: trouble in paradise? Am J Roentgenol
14. Baldwin C, Kittredge D, Baron M, et al. Academic Pediatric Association Educational Guidelines for Pediatric Residency
. Washington, DC: MedEdPORTAL; 2009.
15. Toolbox of Assessment Methods: A Product of the Joint Initiative of the ACGME Outcomes Project, Accreditation Council for Graduate Medical Education, American Board of Medical Specialties (ABMS), Version 1.1. www.acgme.org/Outcome/assess/Toolbox.pdf
, Accessed June 13, 2011.
16. Kern DE, Thomas PA, Hughes MT. Curriculum Development for Medical Education: A Six-step Approach
. Baltimore: Johns Hopkins University Press; 1998.
17. Baker RC, Klein M, Samaan Z, et al. Effectiveness of an online pediatric primary care curriculum. Acad Pediatr
18. Burnette K, Ramundo M, Stevenson M, et al. Evaluation of a web-based asynchronous pediatric emergency medicine learning tool for residents and medical students. Acad Emerg Med
19. Cook D, Thompson W, Thomas K. Case-based or non case-based questions for teaching post-graduate physicians: a randomized crossover trial. Acad Med
20. Bridgemohan CF, Levy S, Veluz AK, et al. Teaching paediatric residents about learning disorders: use of standardized case discussion versus multimedia computer tutorial. Med Educ