Outcomes From Pediatric Gastroenterology Maintenance of Certification Using Web-based Modules

Abstract

Objectives: 

Beginning in 2013, the North American Society of Pediatric Gastroenterology, Hepatology and Nutrition (NASPGHAN) sponsored and developed subspecialty field–specific quality improvement (QI) activities to provide Part 4 Maintenance of Certification (MOC) credit for ongoing certification of pediatric gastroenterologists by the American Board of Pediatrics. Each activity was a Web-based module that measured clinical practice data repeatedly over at least 3 months as participants implemented rapid cycle change. Here, we examine existing variations in clinical practice among participating pediatric gastroenterologists and determine whether completion of Web-based MOC activities improves patient care processes and outcomes.

Methods: 

We performed a cross-sectional and prospective analysis of physician and parent-reported clinical practice data abstracted from Web-based MOC modules on the topics of upper endoscopy, colonoscopy, and informed consent collected from pediatric gastroenterologists from North America from 2013 to 2016.

Results: 

Among 134 participating pediatric gastroenterologists, 56% practitioners practiced at an academic institution and most (94%) were NASPGHAN members. Participating physicians reported data from 6300 procedures. At baseline, notable practice variation across measured activities was demonstrated. Much of the rapid cycle changes implemented by participants involved individual behaviors, rather than system/team-based improvement activities. Participants demonstrated significant improvements on most targeted process and quality care outcomes.

Conclusions: 

Pediatric gastroenterologists and parents reported baseline practice variation, and improvement in care processes and outcomes measured during NASPGHAN-sponsored Web-based MOC QI activities. Subspecialty-oriented Web-based MOC QI activities can reveal targets for reducing unwarranted variation in clinical pediatric practice, and can effectively improve care and patient outcomes.

What Is Known

• Board-certified pediatric gastroenterologists participate in American Board of Pediatrics–approved Part 4 Maintenance of Certification activities to maintain their certification.
• There is limited documentation of variations in clinical practice among pediatric gastroenterologists in varied settings.
• There has been little evidence of the effect of participation in Web-based quality improvement activities on patient care processes and outcomes.

What Is New

• This work demonstrates that there is significant practice variation among pediatric gastroenterologists across practice settings.
• Completion of North American Society of Pediatric Gastroenterology, Hepatology and Nutrition sponsored Web-based quality improvement activities demonstrated both physician and parent-reported improvements in measures of care processes and outcomes.

See “Do Maintenance of Certification Activities Promote Positive Changes in Clinical Practice?” by Heitlinger on page 655.

In 2000, the American Board of Medical Specialties mandated a 10-year recertification cycle for all 24 specialty affiliates ⁽¹⁾. Subsequently, the American Board of Pediatrics (ABP) and the 23 other certifying boards specified maintenance of certification (MOC) programs, which engage participants in learning and self-assessment and improving professional practice. For ABP Diplomates, meeting the improvements in professional practice requirement can be demonstrated through participation in quality improvement (QI) activities created or approved by the ABP. Practice-initiated QI activities are organized by either individuals or institutions, or by collaborative networks, such as the California Perinatal Quality Care Collaborative, Solutions for Patient Safety and Improve Care Now, and are often expensive to perform and maintain ^(2–4). More individually affordable and accessible Web-based QI activities have also been developed and approved by the ABP. These, however, initially primarily targeted primary care physicians and were not generally viewed as relevant to subspecialists, including pediatric gastroenterologists. In response, the North American Society of Pediatric Gastroenterology, Hepatology and Nutrition (NASPGHAN) sponsored the development of Web-based QI activities in 2013 to provide content-relevant opportunities for its members to obtain MOC. As of 2016, NASPGHAN has independently sponsored 5 and cosponsored 3 unique Web-based QI activities. Data from 3 of these activities, reviewed and approved by the ABP, are presented in this report.

Although the ABP has made efforts to demonstrate the value of its MOC program to its Diplomates ⁽⁵⁾, practitioners remain concerned that participation in such activities has no demonstrated benefit ^(6,7) and have called for evaluations of the effect of MOC on clinical outcomes ⁽⁸⁾. Emerging data from practice-initiated QI activities for MOC have established improvements in care delivery and patient outcomes ^(2,4). In addition, experience and data from the American Board of Internal Medicine's Web-based Performance Improvement Modules and other activities have demonstrated participant satisfaction and improvements in knowledge and care processes ^(9,10).

In 2012, NASPGHAN's Executive Council approved the development of MOC Part 4 (QI) modules that would allow physicians to improve their performance or documentation of certain tasks deemed important to the practice of pediatric gastroenterology. In this report, we report the data collected via the NASPGHAN-sponsored MOC modules focusing on pediatric gastroenterology practices regarding endoscopy and on the clinical practice of obtaining informed consent before performance of procedures. We specifically examined variability in the practice of pediatric gastroenterology, and the effect of MOC module participation on positive changes in selected care and patient outcomes.

METHODS

Study Design

We performed a cross-sectional and prospective analysis of data collected for MOC operational purposes and data entered into NASPGHAN's MOC Web-based modules (upper endoscopy, colonoscopy, and informed consent) as of June 2016.

Maintenance of Certification Completion and Module Participant Demographics

NASPGHAN tracks completion of sponsored Web-based modules and collects limited demographic data (practice setting and NASPGHAN membership status) for operational use.

Web-based Maintenance of Certification Modules and Physician-reported Data

NASPGHAN enlisted internal content experts to develop the MOC Web-based upper endoscopy, colonoscopy, and informed consent QI modules for ABP MOC Part 4 credit. Each module identified standard best clinical practices and guidelines that were introduced in an educational video. Participants engaged in 3 data collection periods over a period of at least 4 months. During each data cycle collection (occurring at baseline and at ≥2 months and at≥3 months after baseline), practitioners self-reported their performance and/or obtained parental survey responses on specified quality measures (Supplemental Digital Content, Appendix, https://links.lww.com/MPG/A864) and entered data into the NASPGHAN Web module.

Most data entry elements consisted of binary (yes-no) items indicating whether a given quality measure was met or documented during an individual patient encounter, based on practitioner documentation or parent report.

Physician-selected Quality Improvement Methods and Feedback

After each of the initial 2 data collections, practitioners identified areas for improvement and stated implementation of change interventions on specified performance and/or quality measures. QI methods promoted in the Web-based modules were based upon PDSA (Plan-Do-Study-Act) cycle methodology ⁽⁴⁾. After each data collection period, practitioners received a performance report on specified outcome measures in relation to themselves (comparing performance to baseline in subsequent data reports) and to a stipulated ideal.

Statistical Analysis

For each Web-based module, we analyzed the change in practice documentation by participants, expressed as the proportion meeting best practice guidelines across select process and quality care measures. QI plans were classified into common categories and according to whether they represented individual or system-based efforts. Change in practice documentation over data cycles were assessed via chi-squared tests. All data were analyzed using JMP professional version 12.0 (SAS Institute, Inc., Cary, NC).

RESULTS

Practitioner Demographics

A total of 134 pediatric gastroenterologists completed at least 1 of 3 ABP-approved, Web-based QI modules for MOC Part 4 credit during 2013 to 2016: upper endoscopy, colonoscopy, and informed consent. Most completed the upper endoscopy module, which had 116 participants. The colonoscopy module had 73 participants, and the informed consent module had 21.

Overall, 56.0% participants reported practicing at an academic medical center, 23.1% in group private practice, 7.5% in hospital-based practice, 12.7% in individual private practice, and 0.7% in an institution affiliated with the armed forces. The majority of participants (94.0%) reported being members of NASPGHAN.

Individual Maintenance of Certification Web-Based Quality Improvement Module Data

Upper Endoscopy (N = 116 Participants)

At baseline (Table 1), the majority of reported procedures were upper endoscopy with biopsy (95.7%), most commonly performed for the indication of dyspepsia with or without abdominal pain (45.4%). Baseline procedural process data showed that gastroenterologists most commonly obtained biopsies at 2 levels within the esophagus (34.1%), and 3 to 4 tissue samples in the duodenum (23.5%). Baseline care process data demonstrated that most endoscopy results were communicated to families by phone (66.2%) within a week after the procedure (69.2%). Overall, baseline documentation data demonstrated that most practitioners documented measured elements inconsistently, with <80% documenting esophageal level biopsies, number of duodenal biopsies, American Society of Anesthesiologists (ASA) status, and estimated blood loss (Table 2 ).

TABLE 1: Descriptive statistics of baseline upper endoscopy practices (N = 1160 upper endoscopies)

TABLE 2: Maintenance of Certification activity quality measures and performance over time (N = 134 participating pediatric gastroenterologists)

TABLE 2 (Continued): Maintenance of Certification activity quality measures and performance over time (N = 134 participating pediatric gastroenterologists)

Overall, upper endoscopic procedures documented to involve esophageal biopsies at ≥2 levels were more likely to be associated with a report of change in clinical management when compared with procedures with fewer esophageal biopsies (81% vs 73%, ≥2 or <2 esophageal biopsies, P < 0.0001). Similarly, upper endoscopic procedures that documented >4 duodenal biopsies were more likely to lead to changes in clinical management than procedures in which fewer biopsies were taken (87.0% vs 75.7%, >4 duodenal biopsies vs less, P < 0.0001).

Compared with baseline data, practitioners reported significant improvements in later data collection periods in documentation of informed consent, performance of time out, duodenal and esophageal biopsy locations and numbers, patient ASA status, procedural complications, estimated blood loss, discharge plans, and communication of procedural reports to primary care physicians and biopsy results to patients during the upper endoscopy QI activity (Table 2 ).

Colonoscopy (N = 73 Participants)

At baseline, the majority of procedures performed were colonoscopy with biopsy (86.3%), most commonly performed for the indication of abdominal pain (52.4%). Baseline procedural process data demonstrated that gastroenterologists reported adequate to excellent bowel preparations (66%), total colonoscopy performance (anal intubation through withdrawal of the scope) within 30 minutes (42.9%), cecal intubation within 15 minutes (31.2%), reaching the terminal ileum (89.6%) when terminal ileum intubation was a procedural goal, and normal findings (55.7%). Baseline care process data showed that performance of colonoscopy in a majority of cases (68.5%) resulted in a change in management. Baseline documentation data demonstrated that most practitioners documented measured elements inconsistently, with <80% documenting bowel prep quality, adverse events and complications, ASA status, procedure duration, and estimated blood loss (Tables 2 and 3 ).

TABLE 3: Descriptive statistics of baseline colonoscopy practices (N = 730 colonoscopies)

Across colonoscopies evaluated at baseline and final data collections (N = 1460 colonoscopies), bowel prep quality was found to be significantly associated with terminal ileum intubation rates (45% vs 90% successful terminal ileum intubation, inadequate vs adequate bowel preparation, P < 0.0001) and with change in management plan (58% vs 72% change in management, inadequate vs adequate bowel preparation, P = 0.03). ASA status was not associated with bowel prep quality and ileal intubation rates. Procedure duration was significantly increased by fellow participation (36% vs 73% ≤30 minutes, fellow participant vs not, P < 0.0001) but not by inadequate prep quality (71% vs 59% ≤30 minutes, adequate vs inadequate prep, P = 0.06), ASA status (62% vs 71%, ASA I/II vs ASA III/IV, P = 0.21), or related to change in management (71% vs 71%, ≤30 vs 30 minutes or more, P = 0.90). Colonoscopies with documented biopsies obtained from various locations throughout the colon and small intestine were more likely to lead to changes in management than procedures without such documentation (71% vs 58%, biopsy location documented vs not documented, P = 0.02).

Compared with baseline data, practitioners reported significant improvements in subsequent data collection periods in documentation of performance of a time out, bowel prep quality, procedure duration, biopsy locations, ASA status, procedural complications and estimated blood loss, and discharge plans (Table 2 ).

Informed Consent (N = 21 Participants)

At baseline, the majority of procedures performed for which informed consent was obtained was upper endoscopy (75.2%). Baseline care process data demonstrated that all consent discussions were performed in the patient's preferred language and the majority of informed consent discussions were performed by the proceduralist or primary gastroenterologist (97.6%), signed, dated, and timed (88.6%), performed within 24 hours of the procedure (91%), and signed by the patient's legal guardian (91.7%). The majority of parents (>90%) reported understanding the main indications for their child's procedure, the risks involved, and potential benefits of the procedure (Tables 2 and 4 ).

TABLE 4: Descriptive statistics of baseline informed consent practices (N = 210 informed consent events)

Compared with baseline data collection periods, significant improvements were demonstrated in later data collection periods of the informed consent activity in regards to parental knowledge of whether a trainee was involved (and personal introduction to the trainee if involved), parental understanding of alternatives to the procedure, parental understanding of how complications would be managed should they occur, and clinician request of assent from the child.

Selected Quality Improvement Methods

Methods of QI that participants were encouraged to use in the didactic sections of the activities included both individual and team-based initiatives. In accordance with the promoted PDSA cycle methodology, participants commonly elected to monitor and review data over the various data cycles as part of their QI methodology. Across all modules, most QI initiatives could be characterized as involving only the participating physician (65% upper endoscopy, 66.5% colonoscopy, and 66.7% informed consent), whereas a minority of participants described engaging a medical practice team and/or office staff (35% upper endoscopy, 33.5% colonoscopy, and 33.3% informed consent).

DISCUSSION

We reviewed data from ABP-approved subspecialty-specific MOC Part 4 QI activities and found evidence of practice variation at baseline, and improved documentation of processes of care and patient outcomes in later data collections by pediatric gastroenterologists performing endoscopic procedures and obtaining informed consent. Identifying and reducing unwarranted practice variation is one of the cornerstones of using QI methodologies to change patient care for the better ⁽¹¹⁾. Although clinical documentation may be viewed as a secondary target for QI initiatives, prior work has demonstrated that poor documentation is associated with poor patient outcomes ⁽¹²⁾. In turn, QI activities that target documentation, such as the ones we describe, have been shown to improve general performance on quality measures, including decreased length of hospital stay ⁽¹³⁾. Across ABP-approved MOC modules sponsored by NASPGHAN, improved procedural documentation was associated with likelihood of leading to a change in patient clinical management, in addition to improved patient-reported comprehension of the informed consent discussion related to pediatric endoscopy, particularly procedural staffing, alternatives, and management of complications.

Evidence of practice variation among pediatric gastroenterologists was found across all analyzed Web-based modules. For upper endoscopy, the greatest variations were seen in documentation of performance of a preprocedural “time out”: acquisition of biopsies from multiple esophageal levels, number of duodenal biopsies obtained, assessment and documentation of ASA status and patient comorbidities, estimated blood loss; and when and how pathology results were communicated to patients. For colonoscopy, the greatest variations were seen in documentation of performance of a time out, biopsy locations, colon preparation quality, ASA status and patient comorbidities, procedural complications, estimated blood loss during the procedure, and procedural timing. For informed consent, the greatest variations were seen in the acquisition of assent from minor and in parent reports of their own comprehension of critical elements of informed consent for a procedure to be performed on their child. Parents particularly reported variable degrees of understanding available alternatives to the procedures being performed, and whether or not a trainee would be performing the procedure. If a trainee was participating in the procedure, there was also variation in whether or not the trainee was introduced to the family before the procedure. Prior studies among adults have also demonstrated differences in endoscopy practices including biopsy performance ^(14,15) and documentation, and practices around communication of pathology results to patients ⁽¹⁶⁾. Prior clinical procedural informed consent studies have also demonstrated notable variability in the comprehension of the informed consent discussion by patients and parents, including studies evaluating informed consent for youth undergoing endoscopic procedures ^(17,18).

There has been growing interest in identifying quality measures for pediatric endoscopy ⁽¹⁹⁾. Thakkar et al ⁽²⁰⁾ recently published an article exploring quality indicators documented during pediatric colonoscopy that evaluated more than 21,800 colonoscopy reports from a multicenter consortium, and that included many of the documentation variables measured in the NASPGHAN MOC colonoscopy module. In our smaller cohort, we found similar issues with poor documentation of bowel prep quality and patient ASA classification. We demonstrate that engaging pediatric gastroenterologists in QI methods, such as PDSA cycles of rapid change as part of this MOC activity, can improve documentation outcomes for both bowel prep quality and ASA status.

Improvements in documentation of key quality measures in pediatric endoscopy appear to be related to important clinical outcomes. Specifically, those endoscopic procedures that followed endoscopy biopsy guidelines published by the American Society of Gastrointestinal Endoscopy ⁽²¹⁾ were more likely to result in changes in patient management when compared to those that did not. Bowel prep quality has been shown to affect diagnostic yield and colonoscopy completion rates in adults ^(9,22). We similarly found significantly lower terminal ileum intubation rates in children with poor bowel prep quality. Although higher ASA status has been associated with poor bowel preparation in both adults and children ^(9,20), we did not demonstrate a similar relation in our cohort. Our results may reflect differences in participant demographics; prior studies demonstrating an association between patients with higher ASA scores and poorer bowel preparations were performed at tertiary-care center practices, whereas our procedural data were reported from practices in a variety of settings. Although it is not clear whether prolonged colonoscopy duration should be used as an indicator of poorer procedural quality ⁽²⁰⁾, we demonstrated improvements in colonoscopy procedural efficiency among activity participants over time.

Prior studies evaluating informed consent in the setting of pediatric endoscopy have demonstrated suboptimal understanding by parents ⁽¹⁸⁾, specifically in regards to alternatives to the procedure, what would happen in the event of a complication, and whether a trainee was involved in the performance of the procedure ⁽¹⁷⁾. Our findings validate such reports and demonstrate that targeted QI efforts by pediatric gastroenterologists can improve these outcomes over time. In our cohort, improvements in parent comprehension during the informed consent process were achieved primarily via improved personal communication efforts to ensure recipient understanding. Other studies have used the teach-back method, videos/multimedia, simplification of language used on the informed consent document, and educational handouts as means by which to improve recipient understanding of informed consent ^(23–25).

Acquiring assent from age-appropriate youth for medical and surgical procedures acknowledges the minor's development of autonomy and is increasingly promoted ^(26–28). Other studies have confirmed that youth desire to be informed about endoscopic and surgical procedures ^(17,29). The majority of pediatric gastroenterologists participating in the informed consent Web QI module at baseline did not report obtaining assent from their pediatric patients at the time of the procedure. After completing the informed consent activity, a statistically significant increase in the goal behavior of obtaining assent from appropriate children was reported by participants.

Improvements in care processes and patient outcomes reported by participants who completed the NASPGHAN-sponsored Web QI modules reflect those that have been demonstrated in other Web-based MOC QI activities ^(10,30). QI for the 3 modules was affected most commonly via changes in individual physician behaviors, as compared to systematic or institutional changes that may have involved behaviors of colleagues or staff. Personal action may have been a natural target for QI initiatives in our study due to our focus on physician documentation, but this tendency may also reflect a lack of understanding by providers regarding the use of practice data feedback to redesign office work processes, and the benefits of focusing on clinical team involvement to sustain change ⁽⁹⁾. System and information management methods have been reported in other QI activities targeting clinical documentation, including use of an electronic reporting system utilizing drop-down menus ⁽³¹⁾ and clinical governance ⁽³²⁾, with varying outcomes.

The primary limitation of our evaluation of outcomes from use of MOC activities involves the self-reported nature of the data collected. Participating physicians selected cases to report at each data cycle for the endoscopy-targeted modules (upper endoscopy and colonoscopy). Although parents completed the survey collected for the informed consent activity, physicians still had the option of selecting which cases to report. Therefore, there was the potential for misrepresentation, bias, and/or error in the evaluated data. Nevertheless, our data concur with findings of other studies involving pediatric endoscopy and informed consent in which data inclusion was more comprehensive ^(17,18,20).

In conclusion, we found significant improvements in clinical care documentation, as well as reported improvements in clinical processes of care and their outcomes among pediatric gastroenterologists who completed NASPGHAN-sponsored Web-based QI activities focused on endoscopy. In addition, parent surveys collected by participants who completed the NASPGHAN-sponsored Web QI activity on informed consent demonstrated significant improvements in parental comprehension of critical elements of the informed consent process. Subspecialty-focused QI activities may have clinical relevance and value for participants engaging in such activities for MOC Part 4 credit. Over time, investments by NASPGHAN and other pediatric subspecialty societies in developing MOC activities may yield important information about opportunities for individual member practice change, and for overall improvements in the healthcare of children.