See “The Gastrointestinal Endoscopy Competency Assessment Tool for Pediatric Colonoscopy” by Walsh et al on page 474.
Assessing medical knowledge has been at the forefront of medical education since its beginning, from medical licensing examinations to specialty and subspecialty board examinations. Assessing medical and procedural skills, however, is often more difficult than assessing medical knowledge. The most critical skill in medicine is the physical examination, yet the objective structured clinical examination (OSCE) as an assessment tool was first described only in 1975 (1). During the past 40 years, the OSCE has become the criterion standard for physical examination skills assessment and is used throughout medical schools. In pediatric gastroenterology practice, endoscopy is a critical skill, yet only recently did Walsh et al develop a pediatric colonoscopy assessment tool (2).
In this issue of the Journal of Pediatric Gastroenterology and Nutrition, Walsh et al describe the validation of their previously developed pediatric colonoscopy assessment tool (3). Three hospitals and 56 endoscopists participated in the validation. Both interrater reliability and test–retest reliability were excellent. Meanwhile, the Gastrointestinal Endoscopy Competency Assessment Tool for pediatric colonoscopy (GiECATKIDS) demonstrated a clear differentiation among novice, intermediate, and advanced endoscopists. As present and past chairs of the NASPGHAN training committee, we applaud this work.
In the present validation study, not only was there differentiation between skill levels but there was also a positive correlation between the GiECATKIDS assessment score and ileal intubation rate, as well as the number of previous colonoscopies. The primary indication for pediatric colonoscopy remains evaluation of bleeding/anemia and suspected inflammatory bowel disease (4), both of which generally require ileal intubation. Therefore, it would make sense that a pediatric assessment tool would correlate with ileal intubation rate and these findings provide additional face validity to the tool. Perhaps, as in adult endoscopy with acceptance of adenoma detection rate as a quality measure, ileal intubation that is a part of the GiECATKIDS will be a future quality measure in pediatric colonoscopy.
The GiECATKIDS offers the first robust pediatric colonoscopy assessment tool that was rigorously developed and now validated with excellent validity and reliability in the assessment of trainees’ colonoscopy skills. The validation of this tool provides further evidence to support its use in training programs, and consideration for ongoing credentialing of pediatric gastroenterologists.
Another tool in the ACGME assessment toolbox (5), or another tool for training program directors and endoscopy directors, however, is hardly useful without the widespread adoption. The benefits of widespread use in training institutions include improved feedback, development of a learning curve, improved identification of those requiring remediation, and eventual development of teaching tools for all of the aspects of endoscopy. Furthermore, widespread use may provide additional data for use in ongoing credentialing of colonoscopy.
Feedback is an essential aspect of teaching and learning, and is vital for performance improvement. Yet, often in training programs, there is little feedback, and the little feedback that is given is most often negative instead of positive. In endoscopy, feedback is rarely given because there is a focus on outcome or completion of the colonoscopy instead of development of necessary skills. The GiECATKIDS provides individual checklist items and technical and cognitive skills necessary for completion and correlated with success, and thus defines expectations of colonoscopy. These expectations include skills to manipulate the colonoscope using control knobs, ability to use a variety of skills for colonoscope advancement or mucosal visualization, and knowledge of complications, findings, and safety. The GiECATKIDS can provide trainees and mentors with a useful framework for formative feedback and can identify opportunities for trainee improvement and alteration of teaching/learning techniques to obtain all of the skills necessary for colonoscopy. In summary, the GiECATKIDS provides a road map of important skills in colonoscopy and can be used for useful feedback and training techniques.
Furthermore, the widespread use of the GiECATKIDS could provide benchmarks for a learning curve for colonoscopy among trainees. Development of colonoscopy skills can vary, just as growth does in children, and thus a standard learning curve based on the number of colonoscopies performed would be ideal. Present recommendations for the number of colonoscopies during training to meet competence are based on expert opinion, but the use of the GiECATKIDS may be able to provide data to support or modify such recommendations. Interestingly, an adult study demonstrated that no trainee was able to complete greater than 90% of colonoscopies after the recommended 140 colonoscopies, although only 11 fellows at a single institution participated in this study (6). Widespread use of the GiECATKIDS assessment tool could provide hundreds of data points in a short time to address this concern. In addition, a colonoscopy learning curve could identify trainees who require remediation or additional training such as hands-on courses or simulators. Finally, a learning curve could be used to compare trainees from one institution with those from another and help better understand institutional differences in teaching colonoscopy.
Additionally, the GiECATKIDS could be used to drive teaching aids for specific skills, techniques, and knowledge. Faculty members at training programs can easily demonstrate colonoscopy skills, but teaching these skills can be difficult. The GiECATKIDS identifies specific skills, and the development of teaching aids for each skill would improve teaching and learning across the country, at the institutional level and the national level through hands-on endoscopy courses. Many faculty members will admit that instead of being taught individual endoscopy skills and techniques, they were taught to complete endoscopy. Training aids would change the landscape of endoscopy training, with a focus on skill acquisition instead of colonoscopy completion.
It is likely that ongoing credentialing of colonoscopy and other procedures will be required. The use of the GiECATKIDS for assessment of competency and credentialing could provide useful feedback and help identify practitioners who would most benefit from additional training. Pediatric cardiopulmonary resuscitation is an excellent parallel to colonoscopy. Many pediatric gastroenterologists maintain cardiopulmonary resuscitation certification with regular classes, and most would admit that every time they take the class at least 1 technique has changed. Likewise, colonoscopy equipment, techniques, and available devices have changed. By using the GiECATKIDS assessment tool as a screen for those best suited for a refresher course or perhaps approaching endoscopy as a skill requiring maintenance of certification, pediatric endoscopy can be enhanced and technologic improvements can be translated to improved care.
The GiECATKIDS, although specifically designed for colonoscopy, could have an extensive effect on advanced procedures by applying similar principles. A recent publication suggested that a majority of fellowships are unable to meet the suggested number of advanced endoscopy procedures during their fellowship training (7). By using the GiECATKIDS as a basis for general endoscopy skills, advanced endoscopy skills can be identified and potentially included in an additional or revised assessment tool. Thus, the focus can change from obtaining the required number of procedures to developing the appropriate skills required for a procedure. For example, a polypectomy procedure requires the skills within this colonoscopy assessment tool, as well as skill with a snare and medical knowledge of electrocautery. Likewise, endoscopic injection is a skill that spans multiple procedures regardless of whether epinephrine, botulinum toxin, or a sclerosant is injected, but must be combined with the medical knowledge of the condition being treated and the substance being injected. By distilling advanced procedures into skills and medical knowledge, however, both can be assessed and individual skills can be taught and enhanced through advanced endoscopy courses, hands-on training, and skill simulators specifically designed for advanced endoscopy to improve advanced endoscopy training.
The future of medicine rests on improved quality and outcome. One, however, must first admit that improvement in the quality of teaching and assessment is essential to the improvement in outcome. This validated assessment tool is the one step toward improved outcome, and its entirety or specific elements such as ileal intubation rate may become the criterion standard for quality measurement in pediatric gastroenterology. The ideal quality marker for endoscopy would be a simple assessment of all of the skills and be a focus of training. Thus, ileal intubation rate or other aspects of this assessment tool are ideal markers and provide an exceptional opportunity for a data-driven approach to quality of pediatric endoscopy. By improving quality, outcome must also improve.
It has been said that research findings are often lost in translation in clinical practice, resulting in a delay in the application of clinical research (8). Similarly, research in medical education often takes years or decades to trickle down to application to trainees, highlighted by the OSCE that became the standard of physical examination skills assessment only after decades. The validation of the GiECATKIDS is a call to action for training programs, endoscopy programs, and their directors to apply this well-developed tool across the country, and truly advance the teaching of endoscopy and to critically look at other procedures that would benefit from a similar approach.
1. Harden RM, Stevenson M, Downie WW, et al. Assessment of clinical competence using objective structured examination. Br Med J
2. Walsh CM, Ling SC, Walters TD, et al. Development of the Gastrointestinal Endoscopy Competency Assessment Tool for pediatric colonoscopy (GiECATKIDS). J Pediatr Gastroenterol Nutr
3. Walsh CM, Ling SC, Mamula P, et al. The Gastrointestinal Endoscopy Competency Assessment Tool for pediatric colonoscopy. J Pediatr Gastroenterol Nutr
4. Squires RH Jr, Colletti RB. Indications for pediatric gastrointestinal endoscopy: a medical position statement of the North American Society for Pediatric Gastroenterology and Nutrition. J Pediatr Gastroenterol Nutr
5. Swing S, Bashook PG. Toolbox of Assessment Methods
. Chicago, IL: Accreditation Council for Graduate Medical Education and American Board of Medical Specialties; 2000.
6. Spier BJ, Benson M, Pfau PR, et al. Colonoscopy training in gastroenterology fellowships: determining competence. Gastrointest Endosc
7. Lerner DG, Li BU, Mamula P, et al. Challenges in meeting fellowship procedural guidelines in pediatric therapeutic endoscopy and liver biopsy. J Pediatr Gastroenterol Nutr
8. Lenfant C. Shattuck lecture—clinical research to clinical practice—lost in translation? N Engl J Med